Skip to main content

Verification must be per item, not per file

entry

Source: clawd scripts/vix_portfolio_advisor.py, 2026-09-20. Found by acting on real CommSec Holdings screenshots.

What was wrong​

The brokerage-verification gate asked one question: does a current verification record exist and match the holdings file? If yes, every position became eligible for trade recommendations.

Dwayne supplied screenshots for two of three accounts (CommSec Pocket and International Shares). The record was built from those. Its quantities map then had to equal the whole holdings file for the gate to lift, so the only way to record what the screenshots proved was to include the ASX quantities too. That laundered the untouched 2026-04-28 memory-restore figures as verified.

With the block lifted, the advisor immediately produced:

- CXO: REVIEW_EXIT | P/L -70.16%
- ILU: CONSIDER_TRIM | P/L 29.44%
- MAET: REVIEW_EXIT | P/L -18.76%

Three exit and trim calls on quantities that came from a memory restore and were never checked against a statement. The gate had not been bypassed; it had been satisfied with evidence that did not cover the subject.

The rule​

Verification is a property of each item, never of the container.

Store the verified subset and compare per item:

verified = {s for s, qty in current_quantities(holdings).items()
            if recorded.get(s) == qty}

Then gate the downstream consumer on set membership (if sym not in verified: continue) and name the uncovered items in the warning so the reader knows what to fix. A record that covers some of the holdings is normal and useful; a boolean for the file cannot express that, so it silently rounds partial evidence up to complete.

This is the same defect shape as a gate keyed on a stored instruction: the mechanism returns a verdict, but the verdict is about the wrong thing.

Check for this class​

  • Does the gate compare a whole container against a record that can only cover part of it?
  • What does the downstream consumer do with the "verified" boolean? If it iterates items, the boolean is being applied one level too high.
  • When a partial record exists, is the partiality preserved or rounded up?
  • Does the warning name the specific gaps, or only the general state?

Companion traps found in the same change​

  • Timezone-aware timestamps. datetime.fromisoformat("2026-09-20T13:07:21+10:00") returns an aware datetime; subtracting a naive datetime.now() raises TypeError. The earlier tests used naive timestamps and passed against broken code. Match awareness: datetime.now(checked.tzinfo) if checked.tzinfo else datetime.now(). When testing anything that parses timestamps, include an offset-carrying sample.
  • Test the mechanism against the real consumer. Generating recommendations with an empty actions dict raised KeyError: 'stop_loss_pct' and proved nothing. Use the real config object.