Skip to main content

Cron output quality

entry

Now​

Observed 2026-09-08 at 23:20 AEST, Hermes repair: bedtime schedules are 19:30/20:00 Sunday through Thursday; event-chains uses CT1 alone; voice sources require same-day successful completed output; voice schedules now follow source completion. Deterministic voice/readiness jobs preserve script failure status. Tech Leaders source now distinguishes capture dates from publication dates and ties headings to cited evidence. Source: /Users/dwayne/.hermes/artifacts/jobs-repair-2026-09-08/repair-result.md and verification.json (172 targeted tests; not full-fleet certification).

Bedtime schedule superseded 2026-09-20: kids-bedtime-lightsout (f700b9defbeb) moved from 0 20 * * 0-4 to 0 0 * * 1-5 and is enabled, next run 2026-09-21 00:00 AEST; kids-bedtime-winddown (2f46dc57b611) is paused at 30 19 * * 0-4 and its job no longer appears in ~/.hermes/cron/jobs.json (re-verified 2026-09-20: only kids-bedtime-lightsout, bedtime-reminder, showcase-lights-left-on and pergola-lights-midnight-off match a bedtime/lights name). The "noon lights-out and 12:30 wind-down" wording in the unrepaired list below is stale: the live lights-out job is midnight. Detail: lessons/2026-09-20-cron-midnight-move-weekday-gate.md.

MindRoom per-schedule timezone support is locally tested but not activated. Existing jobs retain UTC by default. The personal brief needs a named-zone decision plus service restart before an API timezone edit. No live timezone change is claimed. Source: the same repair evidence, observed 2026-09-08. This supersedes the earlier lack of a prepared timezone implementation, not the unresolved intent boundary.

Observed 2026-09-08, repair continuation: OSIRIS-REx video was delivered with attachment in Telegram 5893 at 19:03:45 AEST (parent-provided channel evidence; original platform receipt not independently located by repair workers), despite a blocked continuation ledger. The undelivered claim in the OpenClaw audit is superseded by its dated addendum; never resend/regenerate it. Air fryer succeeded at 20:00 (5899), establishing 1256.108 kWh baseline; no missing-script restoration needed. Source: /Users/dwayne/.openclaw/workspace/artifacts/job-repairs-2026-09-08/media-reconciliation.md and repair-matrix.md. <!-- project: path:/Users/dwayne/.openclaw/workspace -->

Observed 2026-09-08: scoped local repairs cover forecast/number validation, pass timing/buckets, sourced insights, status-helper fallback and same-snapshot ops metrics; eight OpenClaw payloads patched and Hermes unified brief changed to deterministic no-agent execution. These are offline-tested source/payload repairs, not proof of next scheduled delivery. Runtime final-selection/media reconciliation and memory index remain expert/approval-blocked: local configured roster has only main, so requested openclaw expert handoff was rejected. MindRoom timezone intent remains unresolved (local/Los_Angeles comments versus UTC dispatch); no timezone changes. Source: /Users/dwayne/.openclaw/workspace/artifacts/job-repairs-2026-09-08/repair-matrix.md. <!-- project: path:/Users/dwayne/.openclaw/workspace -->

Observed 2026-09-08. Scheduler success is not task success in this fleet. Hermes can record a completed run when an agent reports a failed prerequisite, missing source, or script exit 1. OpenClaw media jobs can complete their initial cron lifecycle before background media completion. Source: /Users/dwayne/.hermes/artifacts/jobs-audit-2026-09-08/audit.pdf, associated run inventories and output transcripts.

OpenClaw automation history is in ~/.openclaw/state/openclaw.sqlite, table task_runs, runtime cron. Read detail_json.status for native status: the generic task status marks an intentionally skipped Pro heartbeat as failed. Live gateway openclaw cron list --all --json is the job inventory authority. Source: live schemas, gateway lists and matched run records, observed 2026-09-08.

Hermes output retention defaults to 50 files per job (cron/jobs.py:_CRON_OUTPUT_DEFAULT_KEEP). Early same-day output for frequent jobs can disappear even though execution records remain. In this audit, cron sessions in ~/.hermes/state.db retained all 24 pruned early Home Integration Sentinel results. Source: cron/jobs.py:3849-3888 and recovered session records, observed 2026-09-08.

Observed 2026-09-27 15:30-15:50 AEST (weekly retune): a job can complete with last_status=ok while never producing its contracted output. The Australian AI Talent Watch job (fe0814a7945e, every 130m, deepseek-v4.1-flash:cloud on ollama-launch) hit the per-turn loop_web_search_cap on 30 of the 50 saved runs on disk, including 16 guardrail halts in the 24 hours to 15:42 AEST, and every one still recorded last_status=ok with last_error=null. Root cause is two-part: the configured search backend ddgs (web.search_backend: ddgs) returns DDGSException: No results found on most queries, so the model re-queries; and the job's own "Search budget: at most 30" block contains a spliced sentence fragment (never keep searching past it.\nrun. Use the evidence already gathered) whose truncated second line no longer reads, so the per-job budget never bounded the loop. Repairs applied 2026-09-27, all Tier 0-2: (1) prompt repaired via hermes cron edit fe0814a7945e --prompt (fragment joined, explicit hard stop after five consecutive empty results), backup at ~/.hermes/backups/cron-fe0814a7945e-prompt-20260927-154449.txt; (2) ~/.hermes/scripts/engagement_scout.py gained guardrail_halt_lines() and output_ended_in_guardrail_halt() so the scout reads the newest stored cron output instead of trusting last_status, and reports the halt class; verified by python3 -m py_compile, a live run that now lists the job, and confirmation it was invisible before the change (scout previously returned zero opportunities at 15:30 while the job was halting five times a day). A direct ddgs probe from the Hermes venv (hermes-agent/venv, ddgs 9.16.0) returned 5 results per query, so the backend itself works; the failure is per-query, not per-backend, and no keyless fallback is configured. Source: ~/.hermes/cron/output/fe0814a7945e/*.md (50 runs), ~/.hermes/logs/gateway.error.log, ~/.hermes/cron/jobs.json, live scout run 2026-09-27 15:5x AEST., hermes.

OpenClaw has no ~/.openclaw/logs/gateway.err.log, so the scout's third log path is inert on this host; the Hermes-side paths carry the signal. Source: filesystem check 2026-09-27.

Unrepaired at audit time, observed 2026-09-08:

  • ~/clawd/scripts/event-chains.py:435 calculates mains total as CT1 + CT2. CT1 is already whole-house power and CT2 is the aircon subset. The 18:30 power-anomaly output therefore double-counts aircon. This is distinct from the correct power-baseline report.
  • Hermes kids-bedtime schedules include noon lights-out and 12:30 wind-down. Their wrappers discard successful stdout and return silence, while ~/.openclaw/workspace/scripts/kids-bedtime.py performs light/media actions and calls Telegram directly. Silence is not proof of no side effect.
  • MindRoom's Personal Ops Autopilot schedule is 07:30 UTC, which is 17:30 AEST, despite its morning description. Source: authenticated schedules API and Matrix task event at 17:30 on 2026-09-08. Moot from 2026-09-19: MindRoom was removed, so this finding has no live subject.
  • Portfolio voice at 07:08 precedes the day's portfolio report completion at 07:20. Research voice at 07:53 precedes the day's summary completion at 08:07. Both selected September 7 sources on September 8. Source: recorded source filenames in voice-job outputs.
  • Tech Leaders HTML has semantic heading/body mismatches despite HTML QA passing. Latest September 8 page is live; explicitly named previous-date archive URL for September 7 returns 404 during audit verification. Source: generated HTML and publication-checks.json.

Detailed per-job findings and proposed repair order are in the audit artifacts. No schedules, scripts, credentials or runtime services were changed during the audit.

Log​

  • 2026-09-08: Hermes verified priority schedule/power/voice/editorial repairs and a real Pro inference probe. The probe returned PROBE_OK; do not keep reporting the old disabled-plugin finding as current for that tested route. Preserved the OpenClaw recovery report's generator/runtime/media/publication limitations and no-resend correction. Source: /Users/dwayne/.hermes/artifacts/jobs-repair-2026-09-08/repair-result.md.

  • 2026-09-08: Recovery clarified evidence provenance above, preserved the no-resend decision, and completed scoped local verification: 37 Python tests, 209 payload assertions, 196 current-inventory comparisons; PDF fixture text extracted from one page, not live delivery/layout certification. Added bounded satellite missing/invalid-data handling without schedule changes. Final report includes source hashes, rollback and unresolved runtime/generator/public-UI/timezone boundaries. Source: /Users/dwayne/.openclaw/workspace/artifacts/job-repairs-2026-09-08/report.md and provenance-recovery.md. <!-- project: path:/Users/dwayne/.openclaw/workspace -->

  • 2026-09-08: Superseded OpenClaw undelivered-video inference with authoritative Telegram attachment receipt; preserved contradictory ledger evidence. Recorded scoped offline-tested repair matrix and outstanding expert/timezone/public-page approval boundaries. Existing findings below remain historical until individually revalidated. Source: /Users/dwayne/.openclaw/workspace/artifacts/job-repairs-2026-09-08/repair-matrix.md. <!-- project: path:/Users/dwayne/.openclaw/workspace -->

  • 2026-09-08: Read-only multi-runtime audit created this entry. Reviewed saved outputs, recovered pruned session results and checked selected artifacts. Treat dated failures as historical until reverified before repair.

  • 2026-09-28 07:10 AEST: A cron job can deliver the tool-guardrail halt text as its brief, move discovery into a pre-run script. Australian AI Talent Watch (fe0814a7945e, every 130m) delivered, instead of its brief, the sentence "I stopped retrying because I kept running web_search 50 times without making progress. The last result above shows what blocked it. Tell me how you'd like to proceed, or send continue and I'll try a different approach." That is not the job's output: it is the runtime's controlled-halt text (run_agent.py::_toolguard_controlled_halt_response, rendered by agent/turn_tool_round.py when _tool_guardrail_halt_decision is set), and it becomes the turn's FINAL RESPONSE, which the cron then delivers. Root cause chain: unattended platforms enable hard stops by default (tool_guardrails.py::ToolCallGuardrailConfig.non_interactive_hard_stop_enabled = True; platform for cron is not in _ATTENDED_PLATFORMS); the per-turn search cap (loop_web_search_cap, _DEFAULT_MAX_WEB_SEARCHES_PER_TURN = 50) and the no-progress detector both route through _guardrail_block_result -> _set_tool_guardrail_halt, so once a block fires the turn ENDS and the halt sentence is the whole reply. The job's own prompt forbade exactly that ("Never ask a question... Never offer to continue"), and its own search instruction ("use at most 30 web_search calls") was above the runtime's hard-stop threshold, so complying with the prompt guaranteed the halt. Measured damage: 36 halted runs across 4 jobs, all time; 33 of them this one job; the run is still logged status=completed, delivery=delivered, so nothing flags it, the job looks healthy while emitting junk. Fix (applied): new deterministic collector ~/.hermes/scripts/talent_watch_scout.py attached as the job's pre-run script; it performs <=12 bounded ddgs searches (venv ~/.hermes/hermes-agent/venv/bin/python, which has ddgs 9.16.0 and is the interpreter the gateway runs, so sys.executable resolves correctly), rotates employers/sectors via ~/.hermes/state/australian_ai_talent_watch_rotation.json, reports per-source failures in sources, and NEVER judges or writes brief prose, it emits candidate pages only, with a top-level try/except so it cannot fail the job. The agent now extracts (<=8 web_extract) instead of discovering, so there is no search loop to trip. Job prompt amended: the 30-search budget and the "five consecutive failures" escape hatch removed, bounded verification budget and a zero-candidates-plus-failures -> coverage-failure rule added; the dedup ledger path, output contract, [SILENT] token, external-content guard and delivery target are all preserved. Search-shape finding: wrapping the employer in quotes ("Canva" AI engineer jobs) returns ads and how-to pages; unquoted natural phrasings ("Canva chief AI officer appointed 2026") return real signals (it surfaced the Nirmal Govind appointment on the first run). Backup ~/.hermes/cron/jobs.json.bak-talentwatch-guardrail-20260928-065247. Ad-hoc harness hermes-verify-iuio8ho1.py 54/54 exit 0, including artefact-shape and falsification checks. Other 3 halt occurrences (0db38ffac926, 12b037b23e7c, 8c8e98e8fd1c) are one-off and not yet fixed.