OSS-Singularity/zcode-dispatch
OSS-Singularity/zcode-dispatch
Watch
5
0
Fork
You've already forked zcode-dispatch
 0
Code Issues 3 Pull requests 2 Projects Releases 129 Packages Wiki Activity Actions

v1.0.20 (VEE-6): torn-ledger defense + admission fail-open #51

Merged
ClaudiuSchuster merged 1 commit from oss-oo/pinky-vee6-torn-ledger into main 2026-10-02 04:46:44 +00:00
Conversation 0 Commits 1 Files changed 7 +396 -11
ClaudiuSchuster commented 2026-10-02 04:35:16 +00:00
Owner

Cross-lane defect fix VEE-6 (reported by Brain-B's budget-consumer validation, observatory v1.0.35-era 51-check parity run): budgets.window_entries crashed on torn ledger records — non-dict entries inside a valid JSON array in usage-ledger.json. Every budgets reader shares that read, and the unwrapped admission call meant a torn usage ledger did not just lose accounting — it halted every dispatch at admission.

The fix (design contract, binding)

  1. budgets._read_entries filters to dict entries — the one choke point every reader shares (window_entries, root_of, usage_summary, check, the rollup payload builders): torn entries behave as if absent.
  2. _append_pruned heal-on-write — non-dicts are filtered from the incoming entries first, so the next recording drops torn entries from the file permanently.
  3. dispatch.py admission fail-open — budgets.check (and its lane_for argument) wrapped: on ANY exception BUDGET_DECISION goes inert ({"enabled": false, "soft": null, "hard": null, "error": …}) with a loud stderr NOTE (failing OPEN, the dispatch proceeds). A guardrail crash must never block a dispatch — the budgets module docstring already promised "a guardrail only ever acts on numbers it could honestly read"; the code now matches the promise. Documented in docs/BUDGETS.md §Admission fail-open.
  4. ledger_io.drop_torn_items — shared-seam hardening for every stream-family ledger: _read_with_salvage (locked reads) and stream.load_json (plain sweep reads) drop non-dict items from list ledgers, one stderr NOTE per filtered read naming the dropped count (no MQTT — a persistently torn file would spam the salvage topic; the file heals on its next write). Parse-tear salvage semantics (backup, notice, default_factory) unchanged.

Suite legs (podman shim first — zero real invocations; rig config dirs, fake app-server, never the live broker/units)

  • Torn usage ledger (string / int / null / nested-list entries): every budgets reader safe, a rig dispatch completes, its recording heals the file, --usage-summary + retained rollups recompute honestly (14 + 29 = 43 rig-proven).
  • Admission fail-open with a real crash class: an undecodable ledger (raw invalid UTF-8 → UnicodeDecodeError, beyond the fail-safe read's OSError/JSONDecodeError catches) — the dispatch proceeds with the NOTE even under a would-refuse hard limit, result JSON free of budget verdicts.
  • Shared seam: a torn stream-pending.json does not crash a rig sweep (pre-fix: AttributeError in the oldest-first sort); the sweep's writes heal the file; the valid expired record is honestly dead-lettered.
  • No regression: the full existing suite green (clean-ledger semantics, dedupe, caps, windows, CC-round legs, salvage legs untouched).

Docs (same-round rule)

  • docs/BUDGETS.md: torn-record defense + admission fail-open subsections.
  • docs/USAGE.md: ledger-family torn-record filtering line (filtered on read, noted on stderr, healed on next write).

No new upstream assumptions: zcode-cli/app-server surfaces untouched; usage numbers still come from the same turn_usage / session/usage reads the result JSON already reports.

MERGE HOLD: the CC round's independent validation (valcc) runs against main @ 6f79f00 — merge only on the brain's merge-go steer.

Cross-lane defect fix **VEE-6** (reported by Brain-B's budget-consumer validation, observatory v1.0.35-era 51-check parity run): `budgets.window_entries` crashed on **torn ledger records** — non-dict entries inside a valid JSON array in `usage-ledger.json`. Every budgets reader shares that read, and the unwrapped admission call meant a torn usage ledger did not just lose accounting — it **halted every dispatch at admission**. ## The fix (design contract, binding) 1. **`budgets._read_entries` filters to dict entries** — the one choke point every reader shares (`window_entries`, `root_of`, `usage_summary`, `check`, the rollup payload builders): torn entries behave as if absent. 2. **`_append_pruned` heal-on-write** — non-dicts are filtered from the incoming entries first, so the next recording drops torn entries from the file permanently. 3. **`dispatch.py` admission fail-open** — `budgets.check` (and its `lane_for` argument) wrapped: on ANY exception `BUDGET_DECISION` goes inert (`{"enabled": false, "soft": null, "hard": null, "error": …}`) with a loud stderr NOTE (*failing OPEN, the dispatch proceeds*). A guardrail crash must never block a dispatch — the budgets module docstring already promised "a guardrail only ever acts on numbers it could honestly read"; the code now matches the promise. Documented in `docs/BUDGETS.md` §Admission fail-open. 4. **`ledger_io.drop_torn_items`** — shared-seam hardening for every stream-family ledger: `_read_with_salvage` (locked reads) and `stream.load_json` (plain sweep reads) drop non-dict items from list ledgers, one stderr NOTE per filtered read naming the dropped count (no MQTT — a persistently torn file would spam the salvage topic; the file heals on its next write). Parse-tear salvage semantics (backup, notice, `default_factory`) unchanged. ## Suite legs (podman shim first — zero real invocations; rig config dirs, fake app-server, never the live broker/units) - **Torn usage ledger** (string / int / null / nested-list entries): every budgets reader safe, a rig dispatch **completes**, its recording **heals** the file, `--usage-summary` + retained rollups recompute honestly (14 + 29 = 43 rig-proven). - **Admission fail-open** with a real crash class: an undecodable ledger (raw invalid UTF-8 → `UnicodeDecodeError`, beyond the fail-safe read's `OSError`/`JSONDecodeError` catches) — the dispatch proceeds with the NOTE **even under a would-refuse hard limit**, result JSON free of budget verdicts. - **Shared seam**: a torn `stream-pending.json` does not crash a rig sweep (pre-fix: `AttributeError` in the oldest-first sort); the sweep's writes heal the file; the valid expired record is honestly dead-lettered. - **No regression**: the full existing suite green (clean-ledger semantics, dedupe, caps, windows, CC-round legs, salvage legs untouched). ## Docs (same-round rule) - `docs/BUDGETS.md`: torn-record defense + admission fail-open subsections. - `docs/USAGE.md`: ledger-family torn-record filtering line (filtered on read, noted on stderr, healed on next write). **No new upstream assumptions**: `zcode-cli`/app-server surfaces untouched; usage numbers still come from the same `turn_usage` / `session/usage` reads the result JSON already reports. MERGE HOLD: the CC round's independent validation (valcc) runs against main @ 6f79f00 — merge only on the brain's merge-go steer.
v1.0.20 (VEE-6): torn-ledger defense + admission fail-open
All checks were successful
Checks / Functional checks (pull_request) Successful in 9m15s
Details Explain this check
67efbe7108 
Cross-lane defect fix (brain-B budget-consumer validation, observatory
v1.0.35-era parity run): budgets.window_entries crashed on TORN ledger
records — non-dict entries inside a valid JSON array in
usage-ledger.json — and every budgets reader shares that read; worse,
the unwrapped admission call meant a torn usage ledger halted EVERY
dispatch at admission.

- budgets._read_entries filters to dict entries: the one choke point
  every reader shares (window_entries, root_of, usage_summary, check,
  the rollup builders) — torn entries behave as if absent.
- _append_pruned filters incoming entries before appending: the next
  recording heals the file (torn records never survive a write).
- dispatch.py admission FAIL-OPEN: budgets.check (and its lane_for
  argument) wrapped — on ANY exception the decision goes inert
  (enabled False, no soft/hard, an error string), a loud stderr NOTE
  fires, and the dispatch proceeds. A guardrail crash never blocks a
  dispatch — the module docstring's "a guardrail only ever acts on
  numbers it could honestly read" made executable.
- ledger_io.drop_torn_items: shared-seam hardening for the whole
  stream family — _read_with_salvage (locked reads) AND
  stream.load_json (plain reads) drop non-dict items from list ledgers
  with ONE stderr NOTE per filtered read (no MQTT — a persistently
  torn file would spam the salvage topic; files heal on next write).
  Parse-tear salvage semantics (backup, notice, default_factory)
  unchanged.
- suite legs: torn usage ledger (string/int/null/nested-list entries)
  — every reader safe, a rig dispatch COMPLETES, its recording heals
  the file, summary + retained rollups recompute honestly; admission
  fail-open proven with a real crash class (an undecodable ledger —
  UnicodeDecodeError) even under a would-refuse hard limit; a torn
  stream-pending.json does not crash a rig sweep and the sweep's
  writes heal the file (the expired bell dead-lettered honestly);
  full-suite no-regression.
- docs: BUDGETS.md torn-record defense + admission fail-open
  subsections; USAGE.md ledger-family torn-record filtering line.

No new upstream assumptions: zcode-cli/app-server surfaces untouched;
usage numbers still come from the same turn_usage / session usage
reads the result JSON already reports.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
OSS-Singularity/zcode-dispatch!51
No description provided.