laboratory/docs/ACCESS-VALIDATION-ROUTES.md
Claudiu Schuster b23fc26fd3
All checks were successful
access-validation-contract / contract (push) Successful in 3m23s
ci / ci (push) Successful in 2m27s
Validate Git SSH and public MCP target contracts (#134)
Add explicit edge and direct-gateway access contracts, SSH server-side refusal/canary checks, and public MCP continuation/integrity checks.

Validated by the isolated e2e gate, 502 contract tests, independent native review and both actual PR CI checks on the exact candidate. Production Stage3 remains separately planned.
2026-10-15 21:39:44 +00:00

25 KiB

Access-route contract inventory at 4cc434a

Focused inventory of the source-defined access contracts on Laboratory main 4cc434a80a0f70ea3a448f387a09b699f04e8979 (2026-10-14, the PR #126/#127 lane state), with the redirect/cancellation round deltas (commits 3da0c13, 64f40ab), this inventory itself (73403b7), and the review-fix round deltas (commits 5c53b1c, 06136f8) marked explicitly. Public-MCP integration round (2026-10-15, on top of a02eb9d): the anonymous public catalog section below replaces the former "planned, NOT deployed, NOT accepted" placeholder — the endpoint shipped as ad99aae, was Root-live-accepted on 2026-10-15, and the lane now carries its contract. Every asserted contract carries an exact source anchor (file:line at the named commit) or a timestamped accepted-evidence pointer. Companion docs: the owner requirement and access matrix live in ACCESS-VALIDATION.md; the implementation record in ACCESS-VALIDATION-IMPLEMENTATION.md; lane usage in scripts/access-validation/README.md.

Anchors to files untouched by the redirect/cancellation round are identical at 4cc434a and on top of it.

Public HTTPS Git

  • Edge: apex catch-all proxies everything unmatched to the forge (caddy/Caddyfile:177 at 4cc434a); git smart-HTTP is served by the forge behind it. Canonical-shape redirects the edge really answers: /about → /about/ (caddy/Caddyfile:56), /agents (68), /agents/console (74), /contribute (118), /observatory (146) permanent; www.oss-oo.io → apex permanent (caddy/Caddyfile:188); /donate → donate.oss-oo.io (114).
  • Public ref advertisement: scripts/verify-live.sh:33 runs git ls-remote --heads https://oss-oo.io/<repo>.git (read-only); watchdog/watchdog.py:45 probes /OSS-OO/laboratory.git/info/refs?service=git-upload-pack expecting 200.
  • Lane proof: stage 2 git-https/public-anonymous-read (legitimate anonymous read of a PUBLIC fixture — native VM1005 evidence), stage 3 git-https/public-fetch.
  • Every HTTPS Git probe, including anonymous reads, requires the final canonical URL. Redirects are refused so an HTTP fallback cannot count as a successful HTTPS transport proof (policy below).

Authenticated HTTPS Git

  • Credentials reach git only through a generated GIT_ASKPASS helper under a neutral env name, with station/system git config neutralized (GIT_CONFIG_GLOBAL=/dev/null, GIT_CONFIG_NOSYSTEM=1, GIT_TERMINAL_PROMPT=0, GIT_ALLOW_PROTOCOL=http:https) — scripts/access-validation/av_common/clients/git_https.py:63 (_run_git) at 4cc434a.

  • Acceptance correction (2026-10-15): Git executes with an owned neutral temporary cwd and HOME, with ancestor repository discovery fenced. Caller-local credential helpers, askpass settings and extra headers and caller .netrc credentials are not inherited. SSL_CERT_FILE and SSL_CERT_DIR references remain explicitly preserved. Pushes take refs/objects from the explicit caller repository via an owned local shared mirror, whose configuration and hooks are independent; no source configuration/files are changed. Preparation and transport share the existing per-check Deadline. Tests exercise general and URL-scoped synthetic credential bait, anonymous refusal without a secret, selected-token authentication, and pushes from ordinary and linked-worktree subdirectory sources. All explicitly addressed local ref namespaces are preserved, including remote-tracking and notes refs. Even with the preparation mirror's own remote.origin.mirror=true, the explicit target URL/refspec push updates only the requested destination refs.

  • Redirect seam pin (this round, commit 3da0c13): every git invocation initially carried -c http.followRedirects=initial (git_https.py:63 _git_argv on 3da0c13). Acceptance correction (2026-10-15): HTTPS targets, including anonymous reads, and all credential-bearing probes now pin http.followRedirects=false, both generally and for the exact operator-named URL. All redirects are errors; the operator must supply the final canonical repository URL, including for a same-origin path redirect. A matching URL-scoped command-line pin is necessary because a URL-scoped local setting overrides a general command-line setting. Only anonymous reads from an explicitly named HTTP target retain initial. Verbatim from git-config(1) (the manual page shipped with git 2.43.0 on the workstation, man git-config — the official git documentation):

    http.followRedirects Whether git should follow HTTP redirects. If set to true, git will transparently follow any redirect issued by a server it encounters. If set to false, git will treat all redirects as errors. If set to initial, git will follow redirects only for the initial request to a remote, but not for subsequent follow-up HTTP requests. Since git uses the redirected URL as the base for the follow-up requests, this is generally sufficient. The default is initial.

    initial is not an origin or hop bound: one initial request may traverse a redirect chain. Real-Git acceptance probes against the native candidate c0b6ad7 followed A→B→C and supplied the synthetic askpass credential after C challenged; a verified HTTPS→HTTP redirect did the same. The current false pins prevent either request from reaching a redirect destination. These are real-binary loopback regressions in stage1_contract/tests/test_redirect_boundaries.py, including the URL-scoped local override and the compatible anonymous redirect chain. No live hostile-route drill is claimed.

  • Native e2e proof shape: scripts/e2e-local.sh:551-555 clone+commit+ push over loopback HTTP with user/pass in the URL (e2e-only fixture; the lane's transport-honest id git/e2e-loopback-http-push).

  • Lane proofs: stage 1 git-https/askpass-fetch, wrong-token, push; stage 2 git-https/authenticated-fetch, wrong-token (on the distinct private fixture, auth-shaped refusal required), canary-push (synthetic write + commit.id readback + verified cleanup); stage 3 git-https/canary-push behind the fail-closed canary gate.

Pinned-host SSH Git (deployed public port 2222)

  • Deployed port: host sshd keeps 22, git-over-ssh lands on 2222 — compose.prod.yml:31-32 (${FORGEJO_SSH_PORT:-2222}:22), compose.pve.yml:17-19, compose.yml:63 (FORGEJO__server__SSH_PORT, default 2222).
  • Known-hosts normalization: non-22 ports write the host field as [host]:port in the temp UserKnownHostsFile under StrictHostKeyChecking=yes — av_common/clients/git_ssh.py (ls_remote/push/_run_git_ssh) at 4cc434a; pinned by stage1_contract/tests/test_git_ssh.py (test_known_hosts_host_field_port_shapes). accept-new/first-use is never used anywhere in the seam; the captured keyscan is verified against the approved host_key_pins fail-closed BEFORE any subprocess exists.
  • Historical accepted evidence (timestamped, see the table below): Root 2026-10-14T05:47:19Z verified the deployed pinned host and invalid-key rejection; authenticated SSH clone was deliberately NOT repeated then.
  • Lane proofs: stage 1 banner/keyscan parse + hermetic transport contract; stage 2 optional SSH overlay (authorized fetch, wrong-key, unauthorized-repo, synthetic canary push); stage 3 ssh/authorized-fetch, ssh/wrong-key, gated ssh/canary-push.

Public REST reads

  • /api/v1/version anonymous: scripts/verify-live.sh:23, watchdog/watchdog.py:42.
  • Registry presence /v2/ → 401: watchdog/watchdog.py:44.
  • Client: av_common/clients/http.py rest_request (redirect policy below applies).

Authenticated MCP (https://oss-oo.io/mcp)

  • Route + edge guard: caddy/mcp.caddy — @mcp path /mcp /mcp/* (line 18), anonymous die at the edge with 401 (line 31), liveness GET /mcp/healthz anonymous (line 22), OPTIONS preflight 204; imported by caddy/Caddyfile:162.
  • Stateless POST JSON-RPC, MCP 2025-06-18: one JSON-RPC 2.0 object per POST; GET /mcp → 405 and DELETE /mcp → 405 (no SSE stream, no server-side sessions) — agent-gateway/remote_server.py:11-13 (contract header), :206-217 (do_GET → 405), :223-225 (do_DELETE → 405).
  • Batch arrays → 400 (remote_server.py:303), wrong Content-Type → 415 (:296), budget exhaustion → HTTP 429 with Retry-After (:283-285; per-token sliding window 600 read-class / 60 write-class per minute — caddy/mcp.caddy header notes, owner decision D7).
  • Anonymous → 401 before any JSON-RPC processing (edge guard above; gateway do_POST auth).
  • Acceptance path the lane demands everywhere: initialize → tools/list → real tools/call read (get_file) — a health 200 proves liveness only (IMPLEMENTATION doc, "Truthfulness first").
  • Unit/protocol proofs: agent-gateway/test_mcp.py (local stdio surface), agent-gateway/test_remote.py (remote door: auth/scope/ revocation negatives, budgets, caddy snippet). Lane proofs: stage 1 mcp/* six legs; stage 2 mcp/surface + 405/400/415 + anonymous; stage 3 mcp/surface.
  • Client: av_common/clients/mcp.py mcp_raw_post — same redirect policy as REST (below).

MQTT over WebSocket

  • Endpoint wss://events.oss-oo.io/mqtt (any path upgrades; non-upgrade → mosquitto 403), MQTT 3.1.1/5 over subprotocol mqtt — docs/EVENTS.md:94-98; edge proxy caddy/Caddyfile:199-204; plain TCP 8883 deliberately not opened (owner decision D9).
  • Probe client (CONNECT/CONNACK rc semantics): watchdog/mqtt_probe.py:53 (connect).
  • Lane proofs: stage 1 CONNACK + rc 5 negative + non-upgrade refusal; stage 2 CONNACK + authorized SUBSCRIBE + canary event through the product path + private-topic silence; stage 3 CONNACK + negative.

LFS / registry presence probes (presence-only, explicitly)

  • LFS: protocol batch route POST <repo>.git/info/lfs/objects/batch with the native empty-download body and application/vnd.git-lfs+json media types; anonymous 401 proves the endpoint. Native evidence: 401 on VM1005 (v4 round). NO LFS content transfer is attempted or claimed.
  • Registry: package API list route GET /api/v1/packages/{owner} (scope-gated by read:package; native 403→200 diagnosis in the IMPLEMENTATION doc) plus the public /v2/ 401 presence shape. NO OCI content round-trip is attempted or claimed.
  • Lane proofs: stage 2/stage 3 lfs|registry/endpoint-presence.

Anonymous public MCP catalog — deployed, accepted (ad99aae)

POST https://oss-oo.io/mcp/public is LIVE and independently accepted: merged as ad99aae ("feat(mcp): add anonymous public repository exploration", PR #132), Root live acceptance 2026-10-15 against the production edge. Source of truth: agent-gateway/mcp_public.py (981 lines at a02eb9d) + its serving side in agent-gateway/remote_server.py (_public_post, and do_GET/do_DELETE/do_OPTIONS for /mcp/public). Offline protocol proofs: agent-gateway/test_public.py (81 cases).

  • Exactly four tools — public_search_repositories, public_repo_info, public_list_tree, public_get_file — one coherent find → orient → browse → read flow. Every other name (authenticated-surface, registrar, garbage) is refused with the bounded unknown-tool error and NO upstream call (mcp_public.call_public_tool).
  • Anonymous by construction: the server strips any inbound Authorization header before work (remote_server._public_post); the upstream client never attaches one and never reads credential material; it also never follows redirects (any 3xx is a refusal — server-side behavior; the lane client keeps the shared lane redirect policy).
  • Uniform refusal: private repo, nonexistent repo, and missing path all answer ONE identical bounded isError refusal — "not found or not public" (mcp_public.NOT_FOUND_MSG) — never differentiated.
  • Complete chunked retrieval: public_get_file serves base64 chunks over HTTP Range with per-chunk verification (206, content-range offset/total, exact delivered length, ETag equal to the git blob sha); continuation cursors are HMAC-signed and opaque; concatenating decoded chunks to eof reproduces the file byte-for-byte (git blob digest sha1("blob <len>\0" + bytes) plus total-length check). A 0-byte file is complete at offset 0. A completeness-checked full-body /git/blobs fallback exists behind the upstream blob-size knob; no arbitrary size cap exists on the Range path.
  • Pagination continuity: public_list_tree walks byte-wise-sorted windows pinned to one immutable commit via signed cursors (has_more ⟺ next_cursor, exact termination; continuations must repeat owner/repo and MUST match the cursor — path/ref are ignored on continuations, the cursor is authoritative; tampered/invalid cursors are refused). public_search_repositories pages with an unknown total: a FULL page never reads as complete.
  • Method/transport contract — TWO serving shapes, bound to the DECLARED target (Root-observed live edge, 2026-10-15): at the public edge (caddy in front of the service) every non-POST — GET, DELETE, OPTIONS included — answers 405 with a bounded text/plain body, structurally NO Allow header, and no Access-Control-Allow-* headers; at the direct gateway (the service itself) GET/DELETE /mcp/public → 405 with Allow: POST, and OPTIONS /mcp/public → 204 with deliberately NO Access-Control-Allow-* headers. The stage-2/3 public_mcp sections carry a REQUIRED target key (direct-gateway | public-edge) and the evaluator binds statuses, the Allow-header expectation, and the receipt wording to that target — one surface's answer can never be relabeled as the other's proof (mis-pairing fails in BOTH directions, pinned by tests and runner rigs). Per-client-IP budget before any upstream work → HTTP 429 with Retry-After (mcp_public.check_budget). The authenticated /mcp surface is unchanged (regression rows above).
  • Client seam: av_common/clients/mcp.py public_* helpers — the anonymous path has NO token parameter at all (credential-freedom is structural; stage 1 proves it from the receiving journal), and the public_read_file/public_list_tree walkers turn the cursor contracts into complete verified results — a truncated transfer, an offset discontinuity, or a digest mismatch raises and can never pass.
  • Lane proofs (route public-mcp, all read-only): stage 1 pins the full contract against the in-process fake (stage1_contract/fakes.py FakePublicMcp, hostile knobs for truncated/corrupt-transfer and snapshot-expiry drills, plus an edge_mode that mirrors the caddy public edge) — 15 rows incl. exact tool scope, search-page semantics, cursor tamper/mismatch refusals, the 429 budget, BOTH method/CORS surfaces (direct gateway and public edge), credential-freedom, truncated-transfer-fails, and the distinct snapshot-expired refusals failing the walk. Stage 2 carries an OPTIONAL public_mcp config section (inert without it; config names fixture NAMES only — no credential envs exist on this surface; the REQUIRED target key declares which surface the URL names) — 7 rows against the configured stack. Stage 3 carries the same 7 rows against a real named environment, inert without the public_mcp section.

No SSE, no server-side sessions, and no authentication exist or are claimed on the public surface. Codex-native client proof exists but is a separate package; this lane makes NO client-acceptance claim (ChatGPT/ZCode/Codex client receipts remain their own stage-3 slots).

Redirect & authentication-boundary policy (2026-10-14 round, 2026-10-15 acceptance corrections)

The HTTP clients no longer rely on urllib's default redirect behavior (at 4cc434a rest_request/mcp_raw_post used bare urllib.request.urlopen; git pinned no http.followRedirects — no test anywhere pinned cross-origin Authorization stripping). The explicit policy (av_common/clients/http.py open_with_policy on 3da0c13, applied identically by mcp.py):

  • Authenticated REST/MCP requests (any Authorization header present, with case-insensitive header names) NEVER follow a cross-origin redirect (scheme, host, or port differ): the client refuses with a deliberate unsafe-redirect error carrying the 3xx answer, and the header is never sent to the other origin (proven by recording sinks in the tests). Never a silent retry without the header.
  • https→http downgrade is refused for ANY request, authenticated or not.
  • Same-origin redirects are followed, Authorization preserved (journal-pinned on the followed hop by the review round 06136f8), bounded by DEFAULT_MAX_REDIRECTS = 3 hops with a per-call max_redirects override for callers that need it. 303 (and 301/302 for body-carrying requests) downgrade to GET without a body; 307/308 preserve method and body exactly (both semantics journal-pinned from what the server actually received, review round 06136f8). Rewriting to GET removes Content-Type, Content-Length, and Transfer-Encoding regardless of header-name casing. Per-hop socket timeouts derive from the remaining netutil.Deadline.
  • Loopback carve-out, precisely: there is none for credentials — the cross-origin refusal applies on loopback exactly as anywhere else. What IS allowed is an UNAUTHENTICATED same-scheme cross-origin redirect (no credential boundary exists to cross — nothing to exfiltrate); the tests prove this on loopback because that is where the fakes live (test_unauthenticated_cross_origin_loopback_followed).
  • Refused Locations: userinfo credentials (scheme://user:pass@...), non-http(s) schemes, syntactically invalid or out-of-range ports (review round 5c53b1c — previously such Locations crashed with an uncaught ValueError at the origin comparison instead of refusing), malformed URL authorities (including invalid IPv6 brackets and missing hosts), and 3xx answers without a Location header. Locations must be ASCII URI references: spaces and non-ASCII characters must be percent-encoded. Raw 3xx answers reach the policy before urllib can parse a malformed Location; refusals therefore return a ClientResult rather than escaping as a parser exception. Refusal texts never quote the Location.
  • Hop-limit rationale: a small bound (3) is a SAFETY bound against redirect loops — it terminates what would otherwise spin until the deadline; it is not an invented performance cap, and callers can override it explicitly when a legitimate route needs more hops.
  • The followed chain lands in ClientResult.meta["redirects"] in scrubbed form (scheme://host[:port]/path — no query strings, no userinfo, no header material; IPv6 literal hosts re-bracketed, review round 5c53b1c), safe under av_common.report.scrub. MCP retains that chain on HTTP-error results as well as successes.
  • TLS probes (av_common/clients/tls.py tls_probe) hit the named host:port directly over a raw socket — there is no HTTP layer and hence no redirect surface; pinned by stage1_contract/tests/test_tls.py (test_probe_connects_to_the_named_host_directly).
  • HTTPS Git and authenticated HTTP Git refuse every redirect with general and URL-scoped http.followRedirects=false pins (citation above). Canonical repository URLs are required. Only anonymous, explicitly named HTTP targets retain initial; they do not carry an askpass secret, and that setting does not promise a single-hop or single-origin bound.

Operator cancellation (round 64f40ab)

SIGINT (KeyboardInterrupt) and SIGTERM (converted by av_common/cancellation.py install()) interrupt a runner at the next interpreter boundary; the runner finalizes the report immediately (in-flight row and every never-run inventory row → notrun/cancelled-by-operator; completed rows keep their real verdicts; pre-existing honest notrun reasons are never relabeled), writes it, and exits 130 (the 128+SIGINT shell convention, carried for both signals). Cancellation never fabricates a passed or failed verdict — OperatorCancelled derives from BaseException precisely so no except Exception probe boundary can swallow it into a "failed check". Temp hygiene survives (askpass helpers, known_hosts cleanups run in finally blocks).

What each existing validator proves today

Validator Proves Pointers
scripts/e2e-local.sh disposable Forgejo/DB: REST repo/PR changes, actual git clone/push over loopback HTTP, local stdio + remote MCP, registrar/join/renewal, cookie console/CSRF, mock mail, broker/bridge, internal WS tests scripts/e2e-local.sh (git legs at 548-560); coverage table in ACCESS-VALIDATION.md
agent-gateway/test_mcp.py, test_remote.py MCP initialize/discovery/read/write, protocol/auth/scope/revocation negatives, budgets, shared MCP caddy snippet agent-gateway/test_mcp.py; agent-gateway/test_remote.py (sections incl. budget_section, caddy_section)
scripts/verify-live.sh apex A record, HTTPS apex, /api/v1/version, www redirect, optional public ls-remote scripts/verify-live.sh:12-34
watchdog/watchdog.py seven expected HTTP statuses incl. git ref advertisement, MCP healthz, registry 401; its own no-redirect HTTP prober watchdog/watchdog.py:41-46, :396-408
watchdog/mqtt_probe.py authenticated MQTT-WS CONNECT/CONNACK watchdog/mqtt_probe.py:53
Lane stage 1 client contracts against in-process loopback fakes incl. the redirect policy, its refusals, and the anonymous public-MCP catalog contract (exact four-tool scope, chunk/digest verification, uniform refusal, method/CORS, budget) scripts/access-validation/README.md; IMPLEMENTATION doc stage table
Lane stage 2 authenticated E2E on the isolated stack (identity, scopes, private separation via an independent outsider user, canary writes with native-schema readback + verified cleanup, MQTT product-path events) + optional anonymous public-MCP rows behind the public_mcp config section same
Lane stage 3 exact-release real-route public checks + gated canary writes behind the fail-closed gate chain + optional anonymous public-MCP rows behind the environment's public_mcp section same

Historical accepted evidence (timestamped — NOT current observations)

Evidence At (UTC) What it proved Pointer
Root live acceptance of deployed PR131 == 4cc434a 2026-10-14T05:47:19Z 18/18 public cases; native files/containers; HTTPS Git + authenticated MCP read; SSH pinned host + invalid-key rejection; authenticated SSH clone deliberately NOT repeated ~/.codex-logs/oss-oo-masterbrain-2/20261014-execution/laboratory-pr131-live-v1/ROOT-LIVE-ACCEPTED.json (outside git; sha256s of its proofs inside)
Root VM1005 final acceptance of the v4 lane tree 2026-10-13 stage-2 board incl. the five-scope owner token; native refusal shapes (wrong-token auth-shape, commit.id, LFS 401, registry scope gate) IMPLEMENTATION doc, v4 sections
v4.1 native release-shape captures 2026-10-13 tag/ref/commit API shapes on the real public v1.74.0 release (read-only) IMPLEMENTATION doc, v4.1 section

Historical results are evidence with a timestamp, never a claim about the live platform at read time.

Honest gaps (registered, not silent)

  • Real SSH handshake proof is VM1005-only: the git-SSH transport has hermetic injected-runner proof on workstations; a real ssh handshake / git-over-ssh has never executed there (by constraint). Wrong-key/unauthorized negatives fail (not pass) on local refusals, so a fail-closed state cannot fake a server-side verdict.
  • Browser proofs unsupported: cookie-jar e2e evidence can never mark a browser check passed; a rendered-browser proof does not exist in the lane yet.
  • LFS/registry content round-trips not attempted: presence probes only (see above); object upload/download/pull proofs are a future configured-target round.
  • Client receipts pending: Codex/ChatGPT/ZCode rows pass only on verified client-run receipts, which do not exist yet.
  • Public MCP: the contract is deployed and accepted (see its section) and stage 1 pins it against fakes; the CONFIGURED stage-2 execution against a standing isolated stack and any stage-3 real-route run of the new rows had not been executed yet at integration time (2026-10-15) — those rows report notrun until a configured round runs them.
  • Redirect policy proven against fixtures, not the live edge: the refusal semantics are pinned by loopback contract tests; the live Caddy redirects are canonical same-origin path redirects (anchored above) and stage 3 exercises the real routes, but no live cross-origin refusal drill has been run (it would require a deliberately hostile route Root has not authorized).