Skip to main content

Lesson: report wrappers must parse their source, not guess its wording

entry

Date: 2026-09-16 Lane: hermes Confidence: high

Context​

~/.hermes/scripts/state_of_union_report.py (cron job 4fa833abcfab, daily 5 20 * * * Australia/Sydney) aggregates five child scripts and prints a card. Two independent defects made its output unusable and mislabelled the fleet as degraded.

What was actually wrong​

  1. Noise, not signal. The Nexus leg printed the child monitor's raw progress lines verbatim. The 20:05 AEST card carried [crypto] Fetching health...| [crypto] Fetching pnl...| [crypto] Fetching engines... and the HA leg carried the literal string NO_REPLY from a wrapper that had nothing to say. The card was built from first_lines(child_output), which takes whatever the child happens to print, including progress chatter.

  2. Missing script misread as a healthy check. run_script ran /Users/dwayne/clawd/scripts/matter_health_report.py, deleted in commit daa641d (2026-08-06, "Claw Automated Sync") along with matter_healer.py, fleet-healer.py and fleet_healer_report.py. The probe therefore returned a non-zero exit, and the caller only tested for "ACTION_REQUIRED" or "Nodes: 0". A failed 2 - can't open file line fell through to the else branch and was reported as "Matter server check is healthy." HA has held zero Matter config entries since the retirement; the Pro Matter server at ws://100.87.212.111:5580/ws now refuses connections outright (ConnectionRefusedError, verified live this session). The check had been vacuous for six weeks while reporting green.

  3. Substring matching on health keys. The Nexus fault test used any(word in nexus.lower() for word in ("blackout","failed","error","unreachable")). The wrapper's own healthy evidence line ends changed=api_error unknown to False, so a clean run tripped the "error" keyword and was escalated as CRITICAL.

  4. Wrong schema assumptions in two places. portfolio_summary() read total_value_aud / portfolio_value_aud / total; the real file (clawd/memory/portfolio-live-prices.json) nests one level deeper under totals.combined_est_aud, so the portfolio line never rendered. The research and Gmail metric lines used memory_tail(), which prints the last N lines of a file and so emitted closing boilerplate (- None from this bounded scan., }) instead of a headline.

  5. Cron status checked against an old format. The guard tested "ERRORS: None" in cron and "DELIVERY FAILURES: None" in cron. cron-hourly-report.py now prints the Errors: and Delivery: lines only when there is a problem, and prints OK: n/n enabled otherwise. The old string is never present, so a clean fleet was raised as a critical error on every run.

  6. Section bleed in the HA count. Counting - bullets after Critical candidates: without stopping at the next header swallowed the Top actionable non-noise: section too, producing "50 critical candidates" where the true figure is 15.

Rule​

A summary wrapper must parse a child's structured output, or re-read the child's own inputs. first_lines() over human-facing stdout is not parsing: it inherits progress chatter and sentinels. When a probe script is missing, that is an unknown, never a pass, and never a silently green "healthy" branch.

Also: never keyword-match on a health report's raw text. Emit and compare explicit status keys (api_error=False, Nodes: 0), because evidence lines routinely contain the words being searched for.

Fix as landed​

state_of_union_report.py:

  • resolve_script() searches ~/.hermes/scripts/ then ~/clawd/scripts/ and returns a distinct ": not found" marker instead of running a missing path.
  • nexus_fault() matches whole-word signals (swarm sync failed, api_error=true, unreachable, timed out) instead of substrings.
  • matter_entities_present() reads /Users/dwayne/.homeassistant/.storage/core.config_entries; with no matter domain entry the retired probe is reported as Matter is not in use: no HA Matter entities, probe retired. When the registry cannot be read it returns True, so an unknown state still pages rather than passing silently.
  • Cron status is read from the presence of Errors: / Delivery: / OK: lines.
  • HA candidates are counted within their own section only.
  • portfolio_summary() reads totals.combined_est_aud; research_scan_summary() reads the file's Updated: header; gmail_scan_summary() reads scanned_at and curated_findings_count.
  • --self-test guards each of the six behaviours with synthetic fixtures.

nexus_swarm_sync_report.py (both the ~/.hermes/scripts/ copy that cron actually resolves and the ~/clawd/scripts/ copy): on an API error or a fingerprint change it prints named status lines (Nexus crypto API unreachable (crypto_api=None)., change_lines()) instead of echoing the monitor's stdout. crypto.errors stays excluded from the error test: its name is misleading, it is a routine INFO/WARN log tail.

Verification​

  • python3 -m py_compile clean on both files.
  • python3 state_of_union_report.py --self-test prints SELF-TEST: PASS, exit 0.
  • Live run 2026-09-16 20:34 AEST: 0 critical, 1 watch, 4 stable, replacing the pre-fix 3 critical with no actionable content. HA count now 15, matching the child report. Portfolio line renders as Portfolio approx A$10,142.
  • Falsification for the Nexus parser: synthetic healthy line containing api_error None to False is not flagged; Nexus swarm sync failed. and crypto_api=None are flagged.
  • Matter retirement confirmed three independent ways: no matter entry in core.config_entries (75 entries total), no matter-named cron job in jobs.json (ced6e4b366be matter-seal-healer is gone), and ConnectionRefusedError on ws://100.87.212.111:5580/ws.

Files (absolute, for the next agent)​

  • /Users/dwayne/.hermes/scripts/state_of_union_report.py
  • /Users/dwayne/.hermes/scripts/nexus_swarm_sync_report.py
  • /Users/dwayne/clawd/scripts/nexus_swarm_sync_report.py
  • /Users/dwayne/clawd/memory/portfolio-live-prices.json (schema: totals.combined_est_aud)
  • /Users/dwayne/clawd/memory/nexus-engine-status.json
  • /Users/dwayne/.hermes/cron/jobs.json

Residual​

  • ~/.hermes/scripts/ is gitignored (.gitignore:2:*), so the SoU fix is not version-controlled; the sibling ~/clawd/scripts/state_of_union_report.py is an older unreferenced copy and was deliberately left untouched.
  • matter_self_heal.py still exists in ~/.hermes/scripts/ but no cron job references it. Delete or repurpose when Matter is next touched.
  • 23 prompt-only enabled cron jobs remain (85.2% script-backed).