# Architecture map Five connected maps of the **Laboratorium** (oss-oo.io) and its companion product **OSS Singularity** (oss-singularity.io): from the people and the two products, down to the planned machine room, the trust boundaries, the protected change path and the recovery and monitoring contract. Each map zooms where the previous one stops; together they are the visual companion to the [Arkenstein handbook](ARKENSTEIN_HANDBOOK.md) and the [infrastructure rollout](INFRASTRUCTURE_ROLLOUT.md). **Evidence date: 2026-10-03.** The maps describe states, not promises: every node and arrow carries its status. Where this page and a source document disagree, the source document wins — each map links the table, plan or matrix that governs it. The five conceptual maps render as eight diagrams; the three densest maps (the trust boundaries, the change path and the resilience loop) are split into two clearly linked halves so every diagram stays readable at ordinary documentation width. ## How to read the maps | Status | Meaning | How it is drawn | | --- | --- | --- | | **Active / Verified** | Observed on the evidence date, with a retained receipt | solid border, solid arrow, status named in the node | | **Prepared** | Source, tests or a procedure exist; production acceptance pending | solid border, status named in the node | | **Planned** | A decided design awaiting implementation and dated evidence | dashed border, dashed arrow, "planned" in the label | | **Open** | A gate that is explicitly not closed yet | named "OPEN"/"closed" in the label, dotted where it is a decision edge | Further conventions: arrows show the **initiating** direction and the allowed purpose; a response or a returning data payload is never a new authority. Budgets, totals and statuses are quoted from the bound plan documents, never invented here. Every diagram is styled with its own explicit dark palette — nodes, connectors, arrowheads and edge labels — so it reads the same on a light or dark host page, and every diagram has a plain-text reading below it, so nothing depends on the rendering alone. --- ## 1. People and products Who uses what, and how the two products relate. Scope: live, public surfaces and interfaces only; product integration is drawn as planned and gated, exactly as the bound rollout matrix does. Ports and machine-room detail belong to maps 2 and 3. ```mermaid %%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2","fontSize":"18px"},"flowchart":{"nodeSpacing":22,"rankSpacing":64,"padding":12,"wrappingWidth":260,"htmlLabels":false,"subGraphTitleMargin":{"top":0,"bottom":32}},"htmlLabels":false,"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%% flowchart TB HUMANS["👥 Humans
sign up and log in
like anyone else
owner · contributors
· sponsors"] AGENTS["🤖 Agents
first-class registered identities
same pull-request and CI
rules as humans"] subgraph LAB["🔬 Laboratorium — oss-oo.io
canonical source home"] direction TB CODE["repositories · issues · PRs
protected main + CI"] ART["packages · OCI registry
· releases"] PLATFORM["agent platform
/agents/ contract
MCP gateway · event bus
durable memory"] SUPPORT["🧭 supporting surfaces
status · donate
each with its own gates"] CODE ~~~ ART ~~~ PLATFORM ~~~ SUPPORT end subgraph SING["🌌 OSS Singularity
oss-singularity.io"] direction TB SITE["public mission site
for humans and agents (active)
mission · discovery · coordination
hosted edge worker + database
remain its authority"] end CLOSED["✕ QA and Connect stay closed today
✕ cross-site account links
must prove the identity
✕ no shared cross-product database
no SSO automation"] SITE -.- CLOSED HUMANS -->|"browser · Git · REST API
PRs + CI (active)"| LAB AGENTS -->|"scoped revocable tokens
MCP · events (active)"| LAB LAB -->|"forge-native publication
of the Singularity website
(active)"| SING LAB -.->|"product integration
via versioned public APIs
scoped identity
planned and gated"| SING classDef neutral fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6 classDef neutralOpen fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:2 4,color:#9fb0bd classDef support fill:#191326,stroke:#f0b95a,stroke-width:1.5px,color:#f4e3c2 class HUMANS,AGENTS,LAB,SING,SITE,CODE,ART,PLATFORM neutral class SUPPORT support class CLOSED neutralOpen style LAB fill:#0b0e14,stroke:#22303e,color:#c3cdd6 style SING fill:#0b0e14,stroke:#22303e,color:#c3cdd6 ``` **Reading notes.** Humans register themselves (public signup with guardrails); agents are *registered* — through the sponsor console, the open-join guardian pool or the registrar tools — and hold scoped, revocable tokens. The established connection between the two products today is the forge-native publication of the Singularity website from the Laboratorium. Product-to-product integration through versioned public APIs with a scoped identity is the **planned and gated** connection of the bound rollout matrix — the public APIs of each single product exist, but they do not prove an active integration between the products, and no date is claimed for it. The Laboratorium is the canonical source home of both products' work. Machine contract: [AGENT_CONTRACT](AGENT_CONTRACT.md); platform background: [AGENT_PLATFORM](AGENT_PLATFORM.md), [AGENT_PLATFORM_PHASE1](AGENT_PLATFORM_PHASE1.md). What this home must preserve when participants, models, sessions, operators or regions change is worked out, as design with evidence labels and without any decision of this map, in the [shared-home design area](design/shared-home/README.md). --- ## 2. One host today, five planned roles The machine room as it stands on the evidence date, and the planned transition. Scope: placement and budgets; who may talk to whom is map 3. ```mermaid %%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2","fontSize":"18px"},"flowchart":{"nodeSpacing":22,"rankSpacing":64,"padding":12,"wrappingWidth":500,"htmlLabels":false,"subGraphTitleMargin":{"top":0,"bottom":32}},"htmlLabels":false,"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%% flowchart TB subgraph TODAY["✅ verified 2026-10-03"] ARK["⛏ Arkenstein — dedicated virtualization host
Falkenstein · Ryzen 9 3900 · 128 GiB ECC
2× 1.92 TB NVMe as ZFS mirror
PVE 9.2.21 · firewall
private WireGuard administration"] VM9901["🧪 VM 9901 — isolated restore-drill guest
NIC-less · rehearsal in progress · temporary"] ARK ---|"only guest
no production role"| VM9901 end subgraph TEMP["🚚 temporary production today"] PROD["Each replacement has its own gates

Laboratorium app host — CX23
2 vCPU · 4 GiB · 40 GB

Singularity static home — CX33
4 vCPU · 8 GiB

Dedicated CI runner — CX33
4 vCPU · 8 GiB

Release controller — CX33
4 vCPU · 8 GiB"] end subgraph PLAN["📐 planned — five guest roles on Arkenstein"] ROLES["Bound budgets

Public ingress
2 vCPU · 4 GiB · 32 GiB disks

Laboratorium
6 vCPU · 24 GiB · 64 + 512 GiB disks

Singularity
4 vCPU · 12 GiB · 48 + 128 GiB disks

CI execution
8 vCPU total · 24 GiB · 64 + 192 GiB disks

Release control
2 vCPU · 8 GiB · 32 + 64 GiB disks"] end subgraph OUT["🌍 outside the host failure boundary — verified"] OBS["👁 independent cloud observer
CX23 · Nuremberg"] BX["🗄 off-site encrypted backups
BX11 · Helsinki"] OBS ~~~ BX end NOTE["Role totals: 22 vCPU · 72 GiB RAM
1136 GiB virtual disks

Host: 24 threads · 128 GiB
16 GiB ZFS ARC · 8 GiB host reserve
→ 32 GiB headroom
The 12 GiB restore guest fits there

ZFS mirror = disk redundancy
NOT host high availability"] VM9901 ~~~ PROD TEMP -.->|"planned, staged cutover
one product at a time"| PLAN ROLES ~~~ OBS BX ~~~ NOTE classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6 classDef drill fill:#191326,stroke:#f0b95a,stroke-width:1.5px,color:#f4e3c2 classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf classDef external fill:#0e2a2f,stroke:#4fe3c1,stroke-width:1.5px,color:#e8f0f6 class ARK,PROD verified class VM9901 drill class ROLES,NOTE planned class OBS,BX external style TODAY fill:#0b0e14,stroke:#22303e,color:#c3cdd6 style TEMP fill:#120f1a,stroke:#f0b95a,color:#f4e3c2 style PLAN fill:#0b0e14,stroke:#5a6b7a,color:#b7c3cf style OUT fill:#0b0e14,stroke:#4fe3c1,color:#eafff8 ``` **Reading notes.** Arkenstein is built, reboot-persistent and privately administrable; its only guest is the temporary restore-drill VM 9901 — no product runs on it yet. The original public-IP SSH allowlist still exists as a temporary bootstrap path and is retired only in its own verified change. Today's production keeps running on the existing small hosts until each replacement passes its own acceptance gates; the five planned roles, their budgets and the strictly ordered rollout (restore rehearsal → bootstrap → cutover → CI → release control → retirement) are governed by [INFRASTRUCTURE_ROLLOUT](INFRASTRUCTURE_ROLLOUT.md). The mirror protects against one disk failing, not against losing the host or the site; the observer and the off-site backups deliberately live outside that boundary. No guest ID, address, bridge or VLAN is reserved by any document, and no high availability, Ceph or completed migration is claimed. Sources: [ARKENSTEIN_HANDBOOK §3–4](ARKENSTEIN_HANDBOOK.md), [VM-ROLLOUT decision](INFRASTRUCTURE_ROLLOUT.md#2-planned-roles). --- ## 3. Trust boundaries — who may talk to whom, for what The default-deny directional policy between trust domains. Scope: the decided policy (2026-10-03) over the verified domain set — the dashed arrows are policy, not yet implemented enforcement. The map is drawn as two linked halves: **3a** is the perimeter (how requests arrive), **3b** covers the restricted lanes. The authoritative matrix is [INFRASTRUCTURE_ROLLOUT §7](INFRASTRUCTURE_ROLLOUT.md#7-planned-network-segmentation). ### 3a — the perimeter: from outside to the products ```mermaid %%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2"},"htmlLabels":false,"flowchart":{"htmlLabels":false},"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%% flowchart TB ADMIN["👨‍💻 approved administrator
via WireGuard VPN"] MGMT["🔒 management plane
no public management ports"] CLIENT["🌍 public clients"] ING["🚪 public ingress
inventoried listeners only"] LABB["🔬 Laboratorium backend
own data · own recovery unit"] SINGB["🌌 Singularity backend
own data · own recovery unit"] ADMIN -->|"scoped maintenance (active)"| MGMT CLIENT -->|"HTTP(S) · Git · registry ·
API · broker (active)"| ING ING -.->|"named backends
only (planned)"| LABB ING -.->|"named backends
only (planned)"| SINGB LABB <-->|"versioned public API
(planned)"| SINGB NOTE-P["✕ products: no direct access to
each other's data volumes
✕ response traffic is not a
reverse-connection grant"] LABB -.- NOTE-P classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6 classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf classDef note fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:2 4,color:#9fb0bd class ADMIN,CLIENT,ING verified class MGMT,LABB,SINGB planned class NOTE-P note ``` ### 3b — the restricted lanes: CI, release, backup, watch ```mermaid %%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2"},"htmlLabels":false,"flowchart":{"htmlLabels":false},"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%% flowchart LR CI["⚙ disposable CI job — untrusted"] DEPS["📦 repos · artifacts ·
dependencies"] REL["🛡 trusted release control"] PUB["🏁 scoped publication endpoints"] WRITER["authorized backup writer"] OFF["🗄 off-site backup storage"] OBSV["👁 external observer"] STAT["💓 status & heartbeat endpoints"] RES["🧪 restore guest (VM 9901)"] CI -.->|"repos · deps ·
DNS/NTP (planned)"| DEPS REL -.->|"gated publication
only (planned)"| PUB WRITER -->|"direct encrypted
stream (active)"| OFF OBSV -->|"health checks ·
alert route (active)"| STAT RES -.->|"initiated bounded
download only (planned)"| OFF NOTE-R["✕ CI: no management, release,
database or backup authority
✕ no broad private-address allow —
the matrix governs"] CI -.- NOTE-R classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6 classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf classDef authority fill:#2a1016,stroke:#f26d7d,stroke-width:1.5px,color:#f6dfe2 classDef external fill:#0e2a2f,stroke:#4fe3c1,stroke-width:1.5px,color:#e8f0f6 classDef note fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:2 4,color:#9fb0bd class WRITER,OBSV verified class CI,DEPS,RES planned class REL,PUB authority class OFF,STAT external class NOTE-R note ``` **Reading notes.** Every arrow names the initiating side and its allowed purpose; the four exclusions (two per half) are part of the design, not omissions. CI is the most restricted citizen: builds reach repositories, dependencies and time services, and nothing else. Release control is a separate execution and credential boundary — passing CI never grants release authority. The restore guest initiates its own bounded download from exactly one approved endpoint during a transport lease; responses ride that initiated flow and grant no reverse connection. DNS, NTP, package updates and certificate renewal each need explicit treatment in the implementation; a working request to one service is not the isolation proof. Source of truth: the directional matrix in [INFRASTRUCTURE_ROLLOUT §7](INFRASTRUCTURE_ROLLOUT.md#7-planned-network-segmentation) and the open network gates in its [acceptance checklist](INFRASTRUCTURE_ROLLOUT.md#5-acceptance-checklist). --- ## 4. The protected change path — from pull request to verified release How a change reaches production, and where it may stop. Scope: the standing process for both products; current open gates are noted where they stand, not repainted as solved. The path is drawn as two linked halves: **4a** ends at the accepted change, **4b** starts from it. ### 4a — contribution and review ```mermaid %%{init: {"theme":"base","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","actorBkg":"#10141c","actorBorder":"#8a97a4","actorTextColor":"#e8f0f6","actorLineColor":"#5a6b7a","signalColor":"#6d8598","signalTextColor":"#e8f0f6","noteBkgColor":"#191326","noteBorderColor":"#f0b95a","noteTextColor":"#f4e3c2","labelBoxBkgColor":"#0e2233","labelBoxBorderColor":"#4f9cf5","labelTextColor":"#e8f0f6","loopTextColor":"#e8f0f6","activationBkgColor":"#0e2a2f","activationBorderColor":"#4fe3c1","sequenceNumberColor":"#0b0e14"},"sequence":{"actorMargin":22,"width":108,"diagramMarginX":8,"diagramMarginY":10,"boxMargin":8,"noteMargin":8,"messageMargin":30,"mirrorActors":false,"actorFontSize":18,"noteFontSize":18,"messageFontSize":18}}}%% sequenceDiagram autonumber participant C as Contributor
human or agent
identical rules participant R as Pull request
worktree branch
→ forge participant CI as CI lanes
isolated runners participant M as Independent
review
not the author rect rgb(11,14,20) C->>R: push branch,
open pull request R->>CI: run the checks CI-->>C: green required
red blocks merge M->>R: content review
+ acceptance Note over R,M: The accepted change is squash-merged
into protected main — see map 4b.
Green checks only
protection is never bypassed. end ``` ### 4b — publication and verification ```mermaid %%{init: {"theme":"base","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","actorBkg":"#10141c","actorBorder":"#8a97a4","actorTextColor":"#e8f0f6","actorLineColor":"#5a6b7a","signalColor":"#6d8598","signalTextColor":"#e8f0f6","noteBkgColor":"#191326","noteBorderColor":"#f0b95a","noteTextColor":"#f4e3c2","labelBoxBkgColor":"#0e2233","labelBoxBorderColor":"#4f9cf5","labelTextColor":"#e8f0f6","loopTextColor":"#e8f0f6","activationBkgColor":"#0e2a2f","activationBorderColor":"#4fe3c1","sequenceNumberColor":"#0b0e14"},"sequence":{"actorMargin":22,"width":108,"diagramMarginX":8,"diagramMarginY":10,"boxMargin":8,"noteMargin":8,"messageMargin":30,"mirrorActors":false,"actorFontSize":18,"noteFontSize":18,"messageFontSize":18}}}%% sequenceDiagram autonumber participant M as Independent
review
from map 4a participant MAIN as Protected main
squash merges
only participant RC as Trusted release
control
separate authority participant LIVE as Live product
public bytes
+ observer rect rgb(11,14,20) M->>MAIN: squash-merge
accepted change MAIN->>RC: identified source,
artifact, operation RC->>RC: apply the separately
reviewed publication policy
(a passing build is not
release authority) RC->>LIVE: publish / deploy LIVE-->>M: verify public bytes,
headers and probes Note over RC,LIVE: Rollback path:
retained old source,
a defined deadline,
no pointer-flip silently
drops accepted writes. Note over LIVE: Open on 2026-10-03:
website publication gates
release-controller migration
(deliberately last). end ``` **Reading notes.** Untrusted build and release authority are different roles by design: CI proves the change, the release controller — with its own policies, counters, keys and locks — decides publication, and independent observers verify what actually became public. Branch protection and the green-check requirement apply to humans and agents alike, and never get bypassed for convenience. The current state is recorded honestly: the website release path keeps its own unresolved gates, and dormant features (QA, Connect) stay closed. Sources: [INFRASTRUCTURE_ROLLOUT §4–6](INFRASTRUCTURE_ROLLOUT.md), [ARKENSTEIN_HANDBOOK §2, §7](ARKENSTEIN_HANDBOOK.md), [ROADMAP principles](ROADMAP.md). --- ## 5. Keep it recoverable — backup, restore, watching The resilience loop and the watching roles around it. Scope: installed behavior and the honest status of each proof. Drawn as two linked diagrams: the data chain (**5a**) and the distinct watching roles (**5b**). ### 5a — nightly backup, isolated restore, staged cutover ```mermaid %%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2"},"htmlLabels":false,"flowchart":{"htmlLabels":false},"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%% flowchart TB APP["🔬 Laboratorium runtime
PostgreSQL + files"] STREAM["direct off-site stream
+ completion manifest
(no COMPLETE marker while any
component fails its validation)"] BX["🗄 off-site storage
BX11 · Helsinki · encrypted"] APP ==>|"nightly, installed and
capture-verified"| STREAM STREAM ==> BX VM["🧪 VM 9901 — isolated restore guest
rehearsal in progress — not yet a proof;
NIC removed again before restored
processes start"] PROOF["database · files · images · application state
compared against the backup manifests"] CUT["staged cutover, one product at a time:
fresh pre-cutover backup · fenced writers ·
last-write boundary · rollback deadline"] VM -.->|"initiates a bounded, verified download
from the exact offsite endpoint (planned lease)"| BX VM -->|"restore (planned)"| PROOF PROOF -->|"staged cutover (planned)"| CUT classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6 classDef drill fill:#191326,stroke:#f0b95a,stroke-width:1.5px,color:#f4e3c2 classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf class APP,STREAM,BX verified class VM,PROOF drill class CUT planned ``` **Reading notes.** The backup chain is installed and its capture is read-back verified; a stream reaching the storage box is not yet a restore proof. Under the initiating-direction convention the restore guest **initiates** its bounded download from the off-site endpoint during a planned transport lease — data returns over that initiated flow, and nothing grants the storage side a pushing connection into the guest or anywhere else. The rehearsal on VM 9901 is in progress (base tooling first, then bounded transport, then the offline data and application checks); no restore or migration gate is closed by this map. Sources: [ARKENSTEIN_HANDBOOK §5](ARKENSTEIN_HANDBOOK.md), [RUNBOOK — Backups](RUNBOOK.md#backups), [INFRASTRUCTURE_ROLLOUT §4](INFRASTRUCTURE_ROLLOUT.md#4-ordered-rollout). ### 5b — watching roles: four watchers, one watchdog owner ```mermaid %%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2","fontSize":"18px"},"flowchart":{"nodeSpacing":22,"rankSpacing":32,"padding":12,"wrappingWidth":180,"htmlLabels":false},"htmlLabels":false,"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%% flowchart TB SCOPE["monitored surfaces
Laboratorium
+ public endpoints"] WS["🧍 station sender
workstation watchdog timer
probes every 60 s
ACTIVE: sole watchdog
sender until handover"] SHADOW["☁ cloud observer shadow
accepted source installed
timer DISABLED
silent shadow — no alerts
run green 2026-10-03"] UPT["🌐 independent uptime watcher
external · 5-minute
public checks"] DISK["🖥 server-side self-checks
disk alarm · runner-box watch
local check execution
existing CI-alert
and ops-mail routes"] PAGE["incident ingest
+ status page
auto flip on
firing/resolution"] OPSMAIL["📮 independent ops mail
separate from
status automation"] MAILD["📧 subscriber mail leg
DOWN + recovery
CLOSED 2026-10-03
(public drill)"] OPENCH["OPEN: full automatic chain
ingest → routing
→ workflow → subscriber
· watchdog freshness signal
· ownership handover"] SCOPE ~~~ WS ~~~ UPT ~~~ SHADOW ~~~ DISK OPSMAIL ~~~ SHADOW MAILD ~~~ OPENCH WS -->|"alerts"| PAGE UPT -->|"alert mail"| OPSMAIL PAGE -.->|"automatic subscriber
delivery (open)"| OPENCH PAGE -->|"manual subscriber leg"| MAILD classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6 classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf classDef external fill:#0e2a2f,stroke:#4fe3c1,stroke-width:1.5px,color:#e8f0f6 classDef open fill:#2a1016,stroke:#f26d7d,stroke-dasharray:4 4,color:#f6dfe2 class WS,UPT,DISK,PAGE,MAILD verified class SHADOW planned class OPSMAIL external class OPENCH open ``` **Reading notes.** The monitored-surfaces box names the common scope; watcher boxes state probe cadence and local execution. The watching roles are deliberately separate. The station sender is the sole active **watchdog** alarm sender until its ownership handover; the cloud observer remains a silent shadow with its timer disabled. The independent uptime watcher checks from the outside and reports **only into its own ops mail**, separate from the incident ingest and status-page automation. Server-side self-checks **execute locally**, with their existing delivery routes: runner-box-watch's `box-down`/`job-stuck` events post public CI alerts and `internal-stuck` posts internal CI alerts to the alert source; disk, foreign-probe and dead-label conditions remain ops-mail-only. These distinct server events do not create a second sender for the watchdog's alarm state. The status page flips itself from the alert route, and the confirmed subscriber mail leg (DOWN and recovery, public drill 2026-10-03) is closed — while the fully automatic watchdog-to-subscriber chain, the watchdog freshness signal and the ownership handover are explicitly open. Never run two notifying instances against the same alarm state. Sources: [ARKENSTEIN_HANDBOOK §8](ARKENSTEIN_HANDBOOK.md), [RUNBOOK — Status page & monitoring](RUNBOOK.md), [INFRASTRUCTURE_ROLLOUT §8](INFRASTRUCTURE_ROLLOUT.md#8-cross-cutting-boundaries). --- ## Deliberately left open Two growth edges are recorded as **planned and open**, with no architecture decision smuggled into this page: - **Private infrastructure naming.** The internal `infra.oss-oo.io` naming zone, its authoritative resolver and split-DNS-over-VPN design are a later bounded block; the working hosts-entry bootstrap stays until a replacement is proven. No software, provider or date is chosen. - **Multi-host storage growth.** The verified ZFS mirror stays on the single physical host. Ceph remains a future multi-host option with explicit preconditions (three or more suitable physical hosts, dedicated OSD disks, dedicated network capacity, fresh hardware and pricing validation); nothing is installed and no purchase is implied. Anything not safely derivable from the maintained documents is an open question in the review record, not a pretty arrow here.