A declared output key that nothing computes makes a report look complete
reporting cron honesty verification
Lesson
A report that declares what it will output, but produces only part of it, reads as complete while carrying no data. Nothing errors. The gap is invisible unless someone counts.
leader_project_suite.py gave every project a metrics_expected list. The code
wrote only source_count, evidence_count, dollar_mentions and
gw_mentions. The other declared keys were never computed:
| project | declared but never produced |
|---|---|
| agent-eval-harness | samples, pass_rate, blocked_items |
| openclaw-hardening-pass | controls_seen, gaps, risk_score |
| openclaw-namespace-audit | mentions, ambiguous, files |
| enterprise-agent-adoption-monitor | signals, vendors, dated_items |
| ai-capex-dashboard | signals |
| autonomy-reliability-auditor | failures, modes, lessons |
Six of six projects affected. The report looked healthy for months.
Rule
When a record declares its output keys, assert that every declared key is
present in the produced record. A missing key is a silent failure, not an
absent value. Where a key cannot be derived, write None explicitly so the
reader sees a known gap rather than an implied omission.
Related trap: a frozen "next action"
The same report set next_action = checklist[0] with no progress tracking. The
same six actions repeated daily under the wording "now has a next action queued",
which implies movement that never happened.
Check the subject of an action against its actual source before repeating it. The adoption monitor's action reads "extract adoption numbers from leader briefings", but the corpus contains zero mentions of Gemini Enterprise, Meta Business AI and Claude connectors. Only Copilot appears. The instruction cannot be carried out from the material it names.
Trap inside the fix
Do not flag every action whose subject is missing from one corpus. Five of the six actions read live systems (the session DB, OpenClaw config, wiki/memory), so a term absent from the briefing corpus does not block them. Flagging all six produced six lines of noise and one true finding. Apply the coverage check only where the action declares that source.
Trap inside the metric
Match the format the data is actually in. Evidence stored as raw fragments
carried dates as "Apr 11", not ISO. An ISO-only pattern reported dated_items: 0
for visibly dated material, which misrepresents the corpus in the opposite
direction from the original silence. Same for vendors, which appear as
| Microsoft | table cells rather than JSON keys.