Skip to main content

A run budget that truncates silently hides everything it skipped

entry

Source: ~/.hermes/scripts/grocery_arbitrage_report.py, 2026-09-20.

What was wrong​

The scan loop stopped when a wall-clock budget ran out, using a break that skipped the remaining items. The list had grown to 134 terms (26 fixed items plus 120 from the receipt price index), but the budget was still 75 seconds. Measured cost is 1.047s per term, so 75s reached about item 71 and stopped.

Three separate failures compounded:

  1. Silent truncation. The budget check was correct and reported something, but the message was self-contradictory: it appended "run budget exhausted before all items were checked" as a single entry, then the reporter printed Fetch issues: 1 item(s) failed. One failed item reads like a flaky fetch, not two thirds of the list never being tried.
  2. Data loss on write. The next line was state_data.update({"timestamp": ..., "items": current}), replacing the items map with only what the truncated run reached. The price history of every unvisited item was destroyed, so the following run compared against a shrunken baseline. Persisted items fell to 69-74 against 134 terms.
  3. Growth with no budget review. The list grew (receipts added 108 terms; three items were added by hand) while the budget stayed at its original value. Nothing tied the two together, so the gap opened silently.

The rule​

A time budget must be justified against the work, and a truncated pass must preserve what it did not look at.

  • State the measured per-item cost and the full-pass cost next to the constant. A budget with no arithmetic beside it is a guess that rots as the list grows.
  • Never let a partial pass replace accumulated state. Merge ({**previous, **current}) so unvisited entries survive and are simply refreshed later.
  • Report the real quantity skipped: f"\{len(terms) - len(current)} of \{len(terms)} items were not checked". "1 item failed" for 63 unchecked items is a false report.
  • Check the host cap before raising a budget. Here the whole cron chain runs ~150s against cron.script_timeout_seconds: 300, so 240s fits with headroom. Raising a budget past the host timeout converts a silent truncation into a hard job failure.

Check for this class​

  • Does the list have a fixed length, or does it grow from external input? Growing lists need a budget derived from the work, not a constant chosen once.
  • On timeout, is state merged or overwritten?
  • Does the failure count equal the number actually skipped?
  • What is the host-level timeout, and does the new budget stay under it?