v0.2.2: the on-the-wire guard, a heal that cannot crash, the plan re-derived under the lock #4

Merged
claude merged 5 commits from oss-oo/appserver-send-v022 into main 2026-10-27 23:17:49 +00:00
Member

v0.2.2 — the second read-only subagent's findings on v0.2.1

That report (on ae3e69d, pinned to a git archive, 83/83 twice, every prior finding verified fixed) arrived minutes after v0.2.1 was merged; its three should-fixes and the nits are folded here.

Should-fixes

  1. The on-the-wire boundary had a hole. Between the request going out and delivered: true, only Timeout/Closed/OSError became ack-unknown; any other exception (a non-object JSON frame during the ack wait, a non-object result/turn) fell to the generic handler → refused, reservation released, the caller resends. Now an on_wire flag makes EVERY failure after the request ack-unknown; replies are parsed defensively (a non-object result still counts as acknowledged); the client skips non-object JSON frames; the verdict is set the moment the acknowledgement arrives, before anything else can fail.
  2. heal() could crash before its try. The lock's makedirs/os.open ran unguarded: an unwritable state dir meant a traceback, no receipt line, a throttle reservation never released. Now the lock acquisition is guarded (heal-skipped, lock-unavailable:…, the send goes on through an open door), and the sender never dies without a receipt: an unexpected exception prints a refused row and gives the reservation back.
  3. The heal action was not re-derived under the lock. Only app/stale/door were re-checked; the verb came from the pre-lock plan (a unit failed/disabled meanwhile would still be restarted; a unit someone else just started would be started again). Now decide(fresh) runs under the lock: disabled/failed → skipped (door-closed), the app appeared → skipped and the send answers held-by-app, somebody started the unit (activating, or active and young) → wait instead of a second start, healed meanwhile → already-healed / door-open-now.

Nits and follow-ups in this PR

  • thread not found (exactly) is the writer-lock verdict, no longer any "not found"; a buffered turn/completed counts only for a known turn id.
  • WebSocket client: a message under reassembly survives a deadline (fragment state on the instance); wait_for_completion honours notifications buffered during the ack wait.
  • station.writers() takes the service pid (hermetic); fake server: EOF-safe length headers, close-before-ack, non-object JSON frame, non-object result; the fake systemctl also opens the door.
  • doctor names the threads the shared server holds while the app runs in fallback mode and points at release (from the counter-proof on the owner's real app, 2026-10-28: the locked chat shows "open in another app", release heals in 658 ms, Retry brings the prompt back).
  • README: observation points, attached-but-door-closed row, heal classes and exit codes, the measured lock-out behaviour, two new known gaps — loaded threads keep their memory (backlog: nightly gated release) and the bundled Codex app tools need a bridge in the shared server (the owner's requirement; design with Z).
  • The two parallel heal legs accept the second sender observing an already-healed station (a flake, not a defect).

Tests

Suite 92 → 100 legs (green on every commit, three runs in a row on the last code change): closed-before-ack → ack-unknown, garbage frame ignored, non-object result still delivered, lock unavailable → skipped + delivered, parallel door-closed heals start once (door-open-now), plan re-derived under the lock (app appeared → held-by-app; unit disabled → door-closed; unit activating → waited).

Independent review

The subagent that wrote the findings verified d66cad0 on a frozen export: "delta clean", its three findings fixed and probed (non-object frame / string result → delivered with the mark kept; lock unavailable → skipped with a receipt; decide(fresh) under the lock), 100/100 twice; three nits remained and are folded in 6543def. The later commits (819ceaa test relaxation, 5ecdd97 doctor hint + README, 36222ad README) carry no contract change. CI green on every commit (runs 13–17). Texts: v021-review/review-subagent-ae3e69d.md, review-subagent-v022-delta.md in the evidence folder.

🤖 Generated with Claude Code

## v0.2.2 — the second read-only subagent's findings on v0.2.1 That report (on `ae3e69d`, pinned to a git archive, 83/83 twice, every prior finding verified fixed) arrived minutes after v0.2.1 was merged; its three should-fixes and the nits are folded here. ### Should-fixes 1. **The on-the-wire boundary had a hole.** Between the request going out and `delivered: true`, only `Timeout`/`Closed`/`OSError` became `ack-unknown`; any other exception (a non-object JSON frame during the ack wait, a non-object `result`/`turn`) fell to the generic handler → `refused`, reservation released, the caller resends. Now an `on_wire` flag makes EVERY failure after the request `ack-unknown`; replies are parsed defensively (a non-object `result` still counts as acknowledged); the client skips non-object JSON frames; the verdict is set the moment the acknowledgement arrives, before anything else can fail. 2. **`heal()` could crash before its `try`.** The lock's `makedirs`/`os.open` ran unguarded: an unwritable state dir meant a traceback, no receipt line, a throttle reservation never released. Now the lock acquisition is guarded (`heal-skipped`, `lock-unavailable:…`, the send goes on through an open door), and the sender never dies without a receipt: an unexpected exception prints a `refused` row and gives the reservation back. 3. **The heal action was not re-derived under the lock.** Only app/stale/door were re-checked; the verb came from the pre-lock plan (a unit `failed`/`disabled` meanwhile would still be restarted; a unit someone else just started would be started again). Now `decide(fresh)` runs under the lock: disabled/failed → skipped (`door-closed`), the app appeared → skipped and the send answers `held-by-app`, somebody started the unit (`activating`, or active and young) → wait instead of a second start, healed meanwhile → `already-healed` / `door-open-now`. ### Nits and follow-ups in this PR - `thread not found` (exactly) is the writer-lock verdict, no longer any "not found"; a buffered `turn/completed` counts only for a known turn id. - WebSocket client: a message under reassembly survives a deadline (fragment state on the instance); `wait_for_completion` honours notifications buffered during the ack wait. - `station.writers()` takes the service pid (hermetic); fake server: EOF-safe length headers, close-before-ack, non-object JSON frame, non-object result; the fake systemctl also opens the door. - `doctor` names the threads the shared server holds while the app runs in fallback mode and points at `release` (from the counter-proof on the owner's real app, 2026-10-28: the locked chat shows "open in another app", `release` heals in 658 ms, Retry brings the prompt back). - README: observation points, attached-but-door-closed row, heal classes and exit codes, the measured lock-out behaviour, two new known gaps — loaded threads keep their memory (backlog: nightly gated `release`) and the bundled Codex app tools need a bridge in the shared server (the owner's requirement; design with Z). - The two parallel heal legs accept the second sender observing an already-healed station (a flake, not a defect). ### Tests Suite 92 → 100 legs (green on every commit, three runs in a row on the last code change): closed-before-ack → `ack-unknown`, garbage frame ignored, non-object result still delivered, lock unavailable → skipped + delivered, parallel door-closed heals start once (`door-open-now`), plan re-derived under the lock (app appeared → `held-by-app`; unit disabled → `door-closed`; unit activating → `waited`). ### Independent review The subagent that wrote the findings verified `d66cad0` on a frozen export: **"delta clean"**, its three findings fixed and probed (non-object frame / string result → delivered with the mark kept; lock unavailable → skipped with a receipt; `decide(fresh)` under the lock), 100/100 twice; three nits remained and are folded in `6543def`. The later commits (`819ceaa` test relaxation, `5ecdd97` doctor hint + README, `36222ad` README) carry no contract change. CI green on every commit (runs 13–17). Texts: `v021-review/review-subagent-ae3e69d.md`, `review-subagent-v022-delta.md` in the evidence folder. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
claude agent added 1 commit 2026-10-27 22:57:56 +00:00
The second read-only subagent's report on v0.2.1 (it arrived after that merge) — all folded:

- on the wire: once the turn/start or turn/steer request has been written, every failure before the
  acknowledgement — not only Timeout/Closed/OSError — is `ack-unknown`; replies are parsed
  defensively (a non-object `result`/`turn` still counts as acknowledged), non-object JSON frames
  are skipped by the client, and the verdict is set the moment the acknowledgement arrives, before
  anything else can fail.
- heal: the lock acquisition cannot crash the sender (`heal-skipped`, `lock-unavailable:…`, the send
  goes on through an open door); under the lock the station is observed again and the plan
  RE-DERIVED (`decide(fresh)`): a unit disabled or failed meanwhile → skipped, the app appeared →
  skipped and the send answers `held-by-app`, somebody started the unit → wait instead of a second
  start, healed meanwhile → `already-healed` / `door-open-now`; `activating` units are waited for.
- the sender never dies without a receipt: an unexpected exception still prints a `refused` row
  and gives the throttle reservation back.
- `thread not found` (exactly) is the writer-lock verdict, no longer any "not found".
- WebSocket client: a message under reassembly survives a deadline (fragment state on the instance);
  `wait_for_completion` honours notifications buffered during the ack wait.
- station: `writers()` takes the service pid (hermetic tests); fake server: EOF-safe length headers,
  close-before-ack, non-object JSON frame, non-object result; the fake systemctl also opens the door.
- suite 92 → 100 legs (closed-before-ack, garbage frame, non-object result, lock unavailable,
  parallel door-closed heals start once, plan re-derived: app appeared / unit disabled / switch to
  wait). README: observation points, attached-but-door-closed row, heal classes and exit codes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude agent added 1 commit 2026-10-27 23:02:32 +00:00
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude agent added 1 commit 2026-10-27 23:11:31 +00:00
The fake systemctl makes the station fresh (like a real restart); when the second sender's first
observation comes after that, no heal is due at all (`heal: None`) — correct, not a second restart.
The legs now assert exactly one systemctl call, both deliveries, one `healed` and either no heal or
`already-healed` / `door-open-now` for the other sender.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude agent added 1 commit 2026-10-27 23:13:06 +00:00
Counter-proof on the owner's real app (v0.2.1, 2026-10-28): the locked chat shows "open in another
app" with Retry/Fork, other chats work, `oo-appserver release` heals in 658 ms and Retry brings the
prompt back. `doctor` now says how many threads are locked for the app and points at `release`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude agent added 1 commit 2026-10-27 23:15:36 +00:00
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude agent merged commit f5d10603ff into main 2026-10-27 23:17:49 +00:00
claude agent deleted branch oss-oo/appserver-send-v022 2026-10-27 23:17:50 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
OSS-OO/oo-appserver!4
No description provided.