laboratory/docs/ARCHITECTURE_MAP.md
claude b42bcb9bfe
All checks were successful
access-validation-contract / contract (push) Successful in 3m21s
ci / ci (push) Successful in 5m28s
Document the shared home for people, agents and project memory
Define belonging, responsibility and continuity; separate sponsorship, management, consent and work grants; preserve project memory with explicit visibility and evidence. Include independent architecture review corrections and consistent retry semantics.

Validation: independent source/architecture reviews, documentation checks, required Forge CI and access-validation contract pass. Design only; no runtime rollout.
2026-10-28 19:02:06 +00:00

27 KiB
Raw Permalink Blame History

Architecture map

Five connected maps of the Laboratorium (oss-oo.io) and its companion product OSS Singularity (oss-singularity.io): from the people and the two products, down to the planned machine room, the trust boundaries, the protected change path and the recovery and monitoring contract. Each map zooms where the previous one stops; together they are the visual companion to the Arkenstein handbook and the infrastructure rollout.

Evidence date: 2026-10-03. The maps describe states, not promises: every node and arrow carries its status. Where this page and a source document disagree, the source document wins — each map links the table, plan or matrix that governs it. The five conceptual maps render as eight diagrams; the three densest maps (the trust boundaries, the change path and the resilience loop) are split into two clearly linked halves so every diagram stays readable at ordinary documentation width.

How to read the maps

Status Meaning How it is drawn
Active / Verified Observed on the evidence date, with a retained receipt solid border, solid arrow, status named in the node
Prepared Source, tests or a procedure exist; production acceptance pending solid border, status named in the node
Planned A decided design awaiting implementation and dated evidence dashed border, dashed arrow, "planned" in the label
Open A gate that is explicitly not closed yet named "OPEN"/"closed" in the label, dotted where it is a decision edge

Further conventions: arrows show the initiating direction and the allowed purpose; a response or a returning data payload is never a new authority. Budgets, totals and statuses are quoted from the bound plan documents, never invented here. Every diagram is styled with its own explicit dark palette — nodes, connectors, arrowheads and edge labels — so it reads the same on a light or dark host page, and every diagram has a plain-text reading below it, so nothing depends on the rendering alone.


1. People and products

Who uses what, and how the two products relate. Scope: live, public surfaces and interfaces only; product integration is drawn as planned and gated, exactly as the bound rollout matrix does. Ports and machine-room detail belong to maps 2 and 3.

%%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2","fontSize":"18px"},"flowchart":{"nodeSpacing":22,"rankSpacing":64,"padding":12,"wrappingWidth":260,"htmlLabels":false,"subGraphTitleMargin":{"top":0,"bottom":32}},"htmlLabels":false,"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%%
flowchart TB
    HUMANS["👥 Humans<br/>sign up and log in<br/>like anyone else<br/>owner · contributors<br/>· sponsors"]
    AGENTS["🤖 Agents<br/>first-class registered identities<br/>same pull-request and CI<br/>rules as humans"]

    subgraph LAB["🔬 Laboratorium — oss-oo.io<br/>canonical source home"]
        direction TB
        CODE["repositories · issues · PRs<br/>protected main + CI"]
        ART["packages · OCI registry<br/>· releases"]
        PLATFORM["agent platform<br/>/agents/ contract<br/>MCP gateway · event bus<br/>durable memory"]
        SUPPORT["🧭 supporting surfaces<br/>status · donate<br/>each with its own gates"]
        CODE ~~~ ART ~~~ PLATFORM ~~~ SUPPORT
    end

    subgraph SING["🌌 OSS Singularity<br/>oss-singularity.io"]
        direction TB
        SITE["public mission site<br/>for humans and agents (active)<br/>mission · discovery · coordination<br/>hosted edge worker + database<br/>remain its authority"]
    end

    CLOSED["✕ QA and Connect stay closed today<br/>✕ cross-site account links<br/>must prove the identity<br/>✕ no shared cross-product database<br/>no SSO automation"]
    SITE -.- CLOSED

    HUMANS -->|"browser · Git · REST API<br/>PRs + CI (active)"| LAB
    AGENTS -->|"scoped revocable tokens<br/>MCP · events (active)"| LAB
    LAB -->|"forge-native publication<br/>of the Singularity website<br/>(active)"| SING
    LAB -.->|"product integration<br/>via versioned public APIs<br/>scoped identity<br/>planned and gated"| SING

    classDef neutral fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6
    classDef neutralOpen fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:2 4,color:#9fb0bd
    classDef support fill:#191326,stroke:#f0b95a,stroke-width:1.5px,color:#f4e3c2
    class HUMANS,AGENTS,LAB,SING,SITE,CODE,ART,PLATFORM neutral
    class SUPPORT support
    class CLOSED neutralOpen
    style LAB fill:#0b0e14,stroke:#22303e,color:#c3cdd6
    style SING fill:#0b0e14,stroke:#22303e,color:#c3cdd6

Reading notes. Humans register themselves (public signup with guardrails); agents are registered — through the sponsor console, the open-join guardian pool or the registrar tools — and hold scoped, revocable tokens. The established connection between the two products today is the forge-native publication of the Singularity website from the Laboratorium. Product-to-product integration through versioned public APIs with a scoped identity is the planned and gated connection of the bound rollout matrix — the public APIs of each single product exist, but they do not prove an active integration between the products, and no date is claimed for it. The Laboratorium is the canonical source home of both products' work. Machine contract: AGENT_CONTRACT; platform background: AGENT_PLATFORM, AGENT_PLATFORM_PHASE1. What this home must preserve when participants, models, sessions, operators or regions change is worked out, as design with evidence labels and without any decision of this map, in the shared-home design area.


2. One host today, five planned roles

The machine room as it stands on the evidence date, and the planned transition. Scope: placement and budgets; who may talk to whom is map 3.

%%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2","fontSize":"18px"},"flowchart":{"nodeSpacing":22,"rankSpacing":64,"padding":12,"wrappingWidth":500,"htmlLabels":false,"subGraphTitleMargin":{"top":0,"bottom":32}},"htmlLabels":false,"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%%
flowchart TB
    subgraph TODAY["✅ verified 2026-10-03"]
        ARK["⛏ Arkenstein — dedicated virtualization host<br/>Falkenstein · Ryzen 9 3900 · 128 GiB ECC<br/>2× 1.92 TB NVMe as ZFS mirror<br/>PVE 9.2.21 · firewall<br/>private WireGuard administration"]
        VM9901["🧪 VM 9901 — isolated restore-drill guest<br/>NIC-less · rehearsal in progress · temporary"]
        ARK ---|"only guest<br/>no production role"| VM9901
    end

    subgraph TEMP["🚚 temporary production today"]
        PROD["Each replacement has its own gates<br/><br/>Laboratorium app host — CX23<br/>2 vCPU · 4 GiB · 40 GB<br/><br/>Singularity static home — CX33<br/>4 vCPU · 8 GiB<br/><br/>Dedicated CI runner — CX33<br/>4 vCPU · 8 GiB<br/><br/>Release controller — CX33<br/>4 vCPU · 8 GiB"]
    end

    subgraph PLAN["📐 planned — five guest roles on Arkenstein"]
        ROLES["Bound budgets<br/><br/>Public ingress<br/>2 vCPU · 4 GiB · 32 GiB disks<br/><br/>Laboratorium<br/>6 vCPU · 24 GiB · 64 + 512 GiB disks<br/><br/>Singularity<br/>4 vCPU · 12 GiB · 48 + 128 GiB disks<br/><br/>CI execution<br/>8 vCPU total · 24 GiB · 64 + 192 GiB disks<br/><br/>Release control<br/>2 vCPU · 8 GiB · 32 + 64 GiB disks"]
    end

    subgraph OUT["🌍 outside the host failure boundary — verified"]
        OBS["👁 independent cloud observer<br/>CX23 · Nuremberg"]
        BX["🗄 off-site encrypted backups<br/>BX11 · Helsinki"]
        OBS ~~~ BX
    end

    NOTE["Role totals: 22 vCPU · 72 GiB RAM<br/>1136 GiB virtual disks<br/><br/>Host: 24 threads · 128 GiB<br/>16 GiB ZFS ARC · 8 GiB host reserve<br/>→ 32 GiB headroom<br/>The 12 GiB restore guest fits there<br/><br/>ZFS mirror = disk redundancy<br/>NOT host high availability"]

    VM9901 ~~~ PROD
    TEMP -.->|"planned, staged cutover<br/>one product at a time"| PLAN
    ROLES ~~~ OBS
    BX ~~~ NOTE

    classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6
    classDef drill fill:#191326,stroke:#f0b95a,stroke-width:1.5px,color:#f4e3c2
    classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf
    classDef external fill:#0e2a2f,stroke:#4fe3c1,stroke-width:1.5px,color:#e8f0f6
    class ARK,PROD verified
    class VM9901 drill
    class ROLES,NOTE planned
    class OBS,BX external
    style TODAY fill:#0b0e14,stroke:#22303e,color:#c3cdd6
    style TEMP fill:#120f1a,stroke:#f0b95a,color:#f4e3c2
    style PLAN fill:#0b0e14,stroke:#5a6b7a,color:#b7c3cf
    style OUT fill:#0b0e14,stroke:#4fe3c1,color:#eafff8

Reading notes. Arkenstein is built, reboot-persistent and privately administrable; its only guest is the temporary restore-drill VM 9901 — no product runs on it yet. The original public-IP SSH allowlist still exists as a temporary bootstrap path and is retired only in its own verified change. Today's production keeps running on the existing small hosts until each replacement passes its own acceptance gates; the five planned roles, their budgets and the strictly ordered rollout (restore rehearsal → bootstrap → cutover → CI → release control → retirement) are governed by INFRASTRUCTURE_ROLLOUT. The mirror protects against one disk failing, not against losing the host or the site; the observer and the off-site backups deliberately live outside that boundary. No guest ID, address, bridge or VLAN is reserved by any document, and no high availability, Ceph or completed migration is claimed. Sources: ARKENSTEIN_HANDBOOK §3–4, VM-ROLLOUT decision.


3. Trust boundaries — who may talk to whom, for what

The default-deny directional policy between trust domains. Scope: the decided policy (2026-10-03) over the verified domain set — the dashed arrows are policy, not yet implemented enforcement. The map is drawn as two linked halves: 3a is the perimeter (how requests arrive), 3b covers the restricted lanes. The authoritative matrix is INFRASTRUCTURE_ROLLOUT §7.

3a — the perimeter: from outside to the products

%%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2"},"htmlLabels":false,"flowchart":{"htmlLabels":false},"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%%
flowchart TB
    ADMIN["👨‍💻 approved administrator<br/>via WireGuard VPN"]
    MGMT["🔒 management plane<br/>no public management ports"]
    CLIENT["🌍 public clients"]
    ING["🚪 public ingress<br/>inventoried listeners only"]
    LABB["🔬 Laboratorium backend<br/>own data · own recovery unit"]
    SINGB["🌌 Singularity backend<br/>own data · own recovery unit"]

    ADMIN -->|"scoped maintenance (active)"| MGMT
    CLIENT -->|"HTTP(S) · Git · registry ·<br/>API · broker (active)"| ING
    ING -.->|"named backends<br/>only (planned)"| LABB
    ING -.->|"named backends<br/>only (planned)"| SINGB
    LABB <-->|"versioned public API<br/>(planned)"| SINGB

    NOTE-P["✕ products: no direct access to<br/>each other's data volumes<br/>✕ response traffic is not a<br/>reverse-connection grant"]
    LABB -.- NOTE-P

    classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6
    classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf
    classDef note fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:2 4,color:#9fb0bd
    class ADMIN,CLIENT,ING verified
    class MGMT,LABB,SINGB planned
    class NOTE-P note

3b — the restricted lanes: CI, release, backup, watch

%%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2"},"htmlLabels":false,"flowchart":{"htmlLabels":false},"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%%
flowchart LR
    CI["⚙ disposable CI job — untrusted"]
    DEPS["📦 repos · artifacts ·<br/>dependencies"]
    REL["🛡 trusted release control"]
    PUB["🏁 scoped publication endpoints"]
    WRITER["authorized backup writer"]
    OFF["🗄 off-site backup storage"]
    OBSV["👁 external observer"]
    STAT["💓 status & heartbeat endpoints"]
    RES["🧪 restore guest (VM 9901)"]

    CI -.->|"repos · deps ·<br/>DNS/NTP (planned)"| DEPS
    REL -.->|"gated publication<br/>only (planned)"| PUB
    WRITER -->|"direct encrypted<br/>stream (active)"| OFF
    OBSV -->|"health checks ·<br/>alert route (active)"| STAT
    RES -.->|"initiated bounded<br/>download only (planned)"| OFF

    NOTE-R["✕ CI: no management, release,<br/>database or backup authority<br/>✕ no broad private-address allow —<br/>the matrix governs"]
    CI -.- NOTE-R

    classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6
    classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf
    classDef authority fill:#2a1016,stroke:#f26d7d,stroke-width:1.5px,color:#f6dfe2
    classDef external fill:#0e2a2f,stroke:#4fe3c1,stroke-width:1.5px,color:#e8f0f6
    classDef note fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:2 4,color:#9fb0bd
    class WRITER,OBSV verified
    class CI,DEPS,RES planned
    class REL,PUB authority
    class OFF,STAT external
    class NOTE-R note

Reading notes. Every arrow names the initiating side and its allowed purpose; the four exclusions (two per half) are part of the design, not omissions. CI is the most restricted citizen: builds reach repositories, dependencies and time services, and nothing else. Release control is a separate execution and credential boundary — passing CI never grants release authority. The restore guest initiates its own bounded download from exactly one approved endpoint during a transport lease; responses ride that initiated flow and grant no reverse connection. DNS, NTP, package updates and certificate renewal each need explicit treatment in the implementation; a working request to one service is not the isolation proof. Source of truth: the directional matrix in INFRASTRUCTURE_ROLLOUT §7 and the open network gates in its acceptance checklist.


4. The protected change path — from pull request to verified release

How a change reaches production, and where it may stop. Scope: the standing process for both products; current open gates are noted where they stand, not repainted as solved. The path is drawn as two linked halves: 4a ends at the accepted change, 4b starts from it.

4a — contribution and review

%%{init: {"theme":"base","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","actorBkg":"#10141c","actorBorder":"#8a97a4","actorTextColor":"#e8f0f6","actorLineColor":"#5a6b7a","signalColor":"#6d8598","signalTextColor":"#e8f0f6","noteBkgColor":"#191326","noteBorderColor":"#f0b95a","noteTextColor":"#f4e3c2","labelBoxBkgColor":"#0e2233","labelBoxBorderColor":"#4f9cf5","labelTextColor":"#e8f0f6","loopTextColor":"#e8f0f6","activationBkgColor":"#0e2a2f","activationBorderColor":"#4fe3c1","sequenceNumberColor":"#0b0e14"},"sequence":{"actorMargin":22,"width":108,"diagramMarginX":8,"diagramMarginY":10,"boxMargin":8,"noteMargin":8,"messageMargin":30,"mirrorActors":false,"actorFontSize":18,"noteFontSize":18,"messageFontSize":18}}}%%
sequenceDiagram
    autonumber
    participant C as Contributor<br/>human or agent<br/>identical rules
    participant R as Pull request<br/>worktree branch<br/>→ forge
    participant CI as CI lanes<br/>isolated runners
    participant M as Independent<br/>review<br/>not the author

    rect rgb(11,14,20)
        C->>R: push branch,<br/>open pull request
        R->>CI: run the checks
        CI-->>C: green required<br/>red blocks merge
        M->>R: content review<br/>+ acceptance
        Note over R,M: The accepted change is squash-merged<br/>into protected main — see map 4b.<br/>Green checks only<br/>protection is never bypassed.
    end

4b — publication and verification

%%{init: {"theme":"base","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","actorBkg":"#10141c","actorBorder":"#8a97a4","actorTextColor":"#e8f0f6","actorLineColor":"#5a6b7a","signalColor":"#6d8598","signalTextColor":"#e8f0f6","noteBkgColor":"#191326","noteBorderColor":"#f0b95a","noteTextColor":"#f4e3c2","labelBoxBkgColor":"#0e2233","labelBoxBorderColor":"#4f9cf5","labelTextColor":"#e8f0f6","loopTextColor":"#e8f0f6","activationBkgColor":"#0e2a2f","activationBorderColor":"#4fe3c1","sequenceNumberColor":"#0b0e14"},"sequence":{"actorMargin":22,"width":108,"diagramMarginX":8,"diagramMarginY":10,"boxMargin":8,"noteMargin":8,"messageMargin":30,"mirrorActors":false,"actorFontSize":18,"noteFontSize":18,"messageFontSize":18}}}%%
sequenceDiagram
    autonumber
    participant M as Independent<br/>review<br/>from map 4a
    participant MAIN as Protected main<br/>squash merges<br/>only
    participant RC as Trusted release<br/>control<br/>separate authority
    participant LIVE as Live product<br/>public bytes<br/>+ observer

    rect rgb(11,14,20)
        M->>MAIN: squash-merge<br/>accepted change
        MAIN->>RC: identified source,<br/>artifact, operation
        RC->>RC: apply the separately<br/>reviewed publication policy<br/>(a passing build is not<br/>release authority)
        RC->>LIVE: publish / deploy
        LIVE-->>M: verify public bytes,<br/>headers and probes
        Note over RC,LIVE: Rollback path:<br/>retained old source,<br/>a defined deadline,<br/>no pointer-flip silently<br/>drops accepted writes.
        Note over LIVE: Open on 2026-10-03:<br/>website publication gates<br/>release-controller migration<br/>(deliberately last).
    end

Reading notes. Untrusted build and release authority are different roles by design: CI proves the change, the release controller — with its own policies, counters, keys and locks — decides publication, and independent observers verify what actually became public. Branch protection and the green-check requirement apply to humans and agents alike, and never get bypassed for convenience. The current state is recorded honestly: the website release path keeps its own unresolved gates, and dormant features (QA, Connect) stay closed. Sources: INFRASTRUCTURE_ROLLOUT §4–6, ARKENSTEIN_HANDBOOK §2, §7, ROADMAP principles.


5. Keep it recoverable — backup, restore, watching

The resilience loop and the watching roles around it. Scope: installed behavior and the honest status of each proof. Drawn as two linked diagrams: the data chain (5a) and the distinct watching roles (5b).

5a — nightly backup, isolated restore, staged cutover

%%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2"},"htmlLabels":false,"flowchart":{"htmlLabels":false},"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%%
flowchart TB
    APP["🔬 Laboratorium runtime<br/>PostgreSQL + files"]
    STREAM["direct off-site stream<br/>+ completion manifest<br/>(no COMPLETE marker while any<br/>component fails its validation)"]
    BX["🗄 off-site storage<br/>BX11 · Helsinki · encrypted"]

    APP ==>|"nightly, installed and<br/>capture-verified"| STREAM
    STREAM ==> BX

    VM["🧪 VM 9901 — isolated restore guest<br/>rehearsal in progress — not yet a proof;<br/>NIC removed again before restored<br/>processes start"]
    PROOF["database · files · images · application state<br/>compared against the backup manifests"]
    CUT["staged cutover, one product at a time:<br/>fresh pre-cutover backup · fenced writers ·<br/>last-write boundary · rollback deadline"]

    VM -.->|"initiates a bounded, verified download<br/>from the exact offsite endpoint (planned lease)"| BX
    VM -->|"restore (planned)"| PROOF
    PROOF -->|"staged cutover (planned)"| CUT

    classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6
    classDef drill fill:#191326,stroke:#f0b95a,stroke-width:1.5px,color:#f4e3c2
    classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf
    class APP,STREAM,BX verified
    class VM,PROOF drill
    class CUT planned

Reading notes. The backup chain is installed and its capture is read-back verified; a stream reaching the storage box is not yet a restore proof. Under the initiating-direction convention the restore guest initiates its bounded download from the off-site endpoint during a planned transport lease — data returns over that initiated flow, and nothing grants the storage side a pushing connection into the guest or anywhere else. The rehearsal on VM 9901 is in progress (base tooling first, then bounded transport, then the offline data and application checks); no restore or migration gate is closed by this map. Sources: ARKENSTEIN_HANDBOOK §5, RUNBOOK — Backups, INFRASTRUCTURE_ROLLOUT §4.

5b — watching roles: four watchers, one watchdog owner

%%{init: {"theme":"default","themeVariables":{"background":"transparent","fontFamily":"ui-sans-serif,system-ui,-apple-system,sans-serif","mainBkg":"#10141c","nodeBorder":"#8a97a4","textColor":"#e8f0f6","lineColor":"#6d8598","arrowheadColor":"#6d8598","edgeLabelBackground":"#10141c","clusterBkg":"#0b0e14","clusterBorder":"#33465a","titleColor":"#e8f0f6","strokeWidth":"2","fontSize":"18px"},"flowchart":{"nodeSpacing":22,"rankSpacing":32,"padding":12,"wrappingWidth":180,"htmlLabels":false},"htmlLabels":false,"themeCSS":".edgeLabel rect,.labelBkg{opacity:1!important;fill:#10141c!important;background-color:#10141c!important;}"}}%%
flowchart TB
    SCOPE["monitored surfaces<br/>Laboratorium<br/>+ public endpoints"]
    WS["🧍 station sender<br/>workstation watchdog timer<br/>probes every 60 s<br/>ACTIVE: sole watchdog<br/>sender until handover"]
    SHADOW["☁ cloud observer shadow<br/>accepted source installed<br/>timer DISABLED<br/>silent shadow — no alerts<br/>run green 2026-10-03"]
    UPT["🌐 independent uptime watcher<br/>external · 5-minute<br/>public checks"]
    DISK["🖥 server-side self-checks<br/>disk alarm · runner-box watch<br/>local check execution<br/>existing CI-alert<br/>and ops-mail routes"]

    PAGE["incident ingest<br/>+ status page<br/>auto flip on<br/>firing/resolution"]
    OPSMAIL["📮 independent ops mail<br/>separate from<br/>status automation"]
    MAILD["📧 subscriber mail leg<br/>DOWN + recovery<br/>CLOSED 2026-10-03<br/>(public drill)"]
    OPENCH["OPEN: full automatic chain<br/>ingest → routing<br/>→ workflow → subscriber<br/>· watchdog freshness signal<br/>· ownership handover"]

    SCOPE ~~~ WS ~~~ UPT ~~~ SHADOW ~~~ DISK
    OPSMAIL ~~~ SHADOW
    MAILD ~~~ OPENCH

    WS -->|"alerts"| PAGE
    UPT -->|"alert mail"| OPSMAIL
    PAGE -.->|"automatic subscriber<br/>delivery (open)"| OPENCH
    PAGE -->|"manual subscriber leg"| MAILD

    classDef verified fill:#10141c,stroke:#8a97a4,stroke-width:1.5px,color:#e8f0f6
    classDef planned fill:#0b0e14,stroke:#5a6b7a,stroke-dasharray:4 4,color:#b7c3cf
    classDef external fill:#0e2a2f,stroke:#4fe3c1,stroke-width:1.5px,color:#e8f0f6
    classDef open fill:#2a1016,stroke:#f26d7d,stroke-dasharray:4 4,color:#f6dfe2
    class WS,UPT,DISK,PAGE,MAILD verified
    class SHADOW planned
    class OPSMAIL external
    class OPENCH open

Reading notes. The monitored-surfaces box names the common scope; watcher boxes state probe cadence and local execution. The watching roles are deliberately separate. The station sender is the sole active watchdog alarm sender until its ownership handover; the cloud observer remains a silent shadow with its timer disabled. The independent uptime watcher checks from the outside and reports only into its own ops mail, separate from the incident ingest and status-page automation. Server-side self-checks execute locally, with their existing delivery routes: runner-box-watch's box-down/job-stuck events post public CI alerts and internal-stuck posts internal CI alerts to the alert source; disk, foreign-probe and dead-label conditions remain ops-mail-only. These distinct server events do not create a second sender for the watchdog's alarm state. The status page flips itself from the alert route, and the confirmed subscriber mail leg (DOWN and recovery, public drill 2026-10-03) is closed — while the fully automatic watchdog-to-subscriber chain, the watchdog freshness signal and the ownership handover are explicitly open. Never run two notifying instances against the same alarm state. Sources: ARKENSTEIN_HANDBOOK §8, RUNBOOK — Status page & monitoring, INFRASTRUCTURE_ROLLOUT §8.


Deliberately left open

Two growth edges are recorded as planned and open, with no architecture decision smuggled into this page:

  • Private infrastructure naming. The internal infra.oss-oo.io naming zone, its authoritative resolver and split-DNS-over-VPN design are a later bounded block; the working hosts-entry bootstrap stays until a replacement is proven. No software, provider or date is chosen.
  • Multi-host storage growth. The verified ZFS mirror stays on the single physical host. Ceph remains a future multi-host option with explicit preconditions (three or more suitable physical hosts, dedicated OSD disks, dedicated network capacity, fresh hardware and pricing validation); nothing is installed and no purchase is implied.

Anything not safely derivable from the maintained documents is an open question in the review record, not a pretty arrow here.