Skip to main content

Hourly log scanner reported a healthy OpenClaw gateway as INACTIVE

lesson · created 2026-09-11 · actor hermes · confidence high · status active tags: openclaw gateway cron monitor false-positive

The hourly log scanner (~/.hermes/scripts/hourly_log_scanner_report.py, wrapping ~/clawd/scripts/openclaw-log-scanner.sh) tested the gateway with ps -axo pid,command | grep -E "[g]ateway run|[h]ermes_cli". OpenClaw's gateway matches neither string: it runs as node .../openclaw/dist/index.js gateway --port 18789. So a healthy gateway printed Gateway: INACTIVE, the wrapper's service_problem() saw that token, and every hourly run raised a "Service issue detected" alert with 0 errors.

Widening the grep to [o]penclaw.*gateway was not enough: the process line sits past the ~1.4KB column where ps can clip a very long command, so detection flaked (about 1 run in 5 still read INACTIVE). Fix applied 2026-09-11: a gateway_active() helper probes http://127.0.0.1:${OPENCLAW_GATEWAY_PORT:-18789}/health first and falls back to pgrep -f "gateway run|hermes_cli|openclaw.*gateway" for legacy or non-default-port setups. After the change the scanner reported ACTIVE in 12/12 runs and under COLUMNS=40, the wrapper stayed [SILENT] in 10/10 runs, and the health endpoint returned 200 with launchd job ai.openclaw.gateway (PID 78899) loaded.

Rule: a health monitor must probe the service's actual endpoint, not infer liveness from a process-name grep. A ps command line is width-limited and regex-fragile. When a monitor reports a service down, confirm with an independent probe (health endpoint, launchctl, lsof) before alerting or repairing, and rerun the check several times so an intermittent false reading cannot pass as fixed.


Sources: ~/.hermes/scripts/hourly_log_scanner_report.py · ~/clawd/scripts/openclaw-log-scanner.sh · ps -axo pid · command · http://127.0.0.1:18789/health