# IGNITION — from zero to https://oss-oo.io

> Historical direct-edge VPS bootstrap, retained as source history. This is
> **not** the current PVE installation, update or rollback procedure. Do not
> run its bootstrap account creation, `.env` replacement, incomplete Compose
> startup, old DNS rollback or server-destruction steps on the restored system.
> Use [RUNBOOK: Current deployment](RUNBOOK.md#current-deployment--2026-10-13)
> and [PVE HTTP backend](RUNBOOK.md#pve-http-backend-behind-ingress) instead.
> Current production writes require explicit reconciliation before reverting
> to old application data; the historical redirect host is not a data rollback.

For the current cross-product architecture, verified infrastructure and migration
gates, see the [Arkenstein handbook](ARKENSTEIN_HANDBOOK.md).

Two humans-actions are required (server + nothing else; DNS is already
automated via the Cloudflare token in the station keyring). Everything else
is scripted. Total time on a fresh server: ~10 minutes.

## 1. Buy the server (operator)

Hetzner Cloud → new server:

| Setting    | Value                                              |
|------------|----------------------------------------------------|
| Type       | **cx23** (x86, 2 vCPU / 4 GB / 40 GB, 5.49 €/mo)   |
| Image      | Ubuntu 24.04                                       |
| Location   | Falkenstein (FSN1) or Nuremberg (NBG1)             |
| SSH key    | `~/.ssh/oss_oo_forge_ed25519` on the station       |

API note (2026): the Cloud API base is `api.hetzner.cloud` — the old
`api.hetzner.com` paths 404. Full token: `~/.api-tokens/hetzner_api.token`.
The agent can create the server itself with that token (done for
oss-oo-forge-1 on 2026-09-22).

Alternative ARM: cax21 (4 vCPU / 8 GB / 80 GB, ~10.49 €/mo) — all pinned
images are multi-arch; x86 keeps third-party CI images friction-free.

Orgs on the forge are created by the OWNER's personal account (not the
bootstrap admin); `scripts/bootstrap-forge.sh` creates only admin + CI
runner unless org names are passed explicitly.

Hand the IP (and root SSH, if not key-injected) to the agent. Fully automated
alternative: create a Hetzner Cloud **API token** (read/write) — then the
agent provisions the server itself.

## 2. Provision (agent, ~2 min)

```bash
ssh root@SERVER_IP 'PROVISION_SSH_KEY="$(cat ~/.ssh/oss_oo_forge_ed25519.pub)" bash -s' \
  < provision/provision.sh
```

Installs docker + compose, ufw (22/80/443/2222), unattended-upgrades,
`forge` ops user, key-only sshd, `/opt/oss-oo`.

## 3. Ship the stack (agent)

```bash
rsync -a --delete --exclude .git --exclude .env --exclude backups/ \
  /home/claudiu/git/oss-oo/ forge@SERVER_IP:/opt/oss-oo/

ssh forge@SERVER_IP
cd /opt/oss-oo
cp .env.example .env
# set POSTGRES_PASSWORD, FORGEJO_ADMIN_PASSWORD (openssl rand -hex 24),
# and ACME_EMAIL (runner image is pinned already)
docker compose -f compose.yml -f compose.prod.yml up -d
exit
./scripts/bootstrap-forge.sh          # admin, orgs, CI runner
```

## 4. DNS flip (agent, automated)

oss-oo.io lives on Cloudflare DNS; the station keyring token has DNS edit on
the zone. Currently the A record points at 188.68.47.93 (netcup redirect to
oss-singularity.io) — that becomes the rollback target.

- Set `A oss-oo.io → SERVER_IP` (DNS-only/grey cloud, no proxy: git-over-ssh
  on 2222 and webhooks do not tolerate CDN proxying)
- Set `A www.oss-oo.io → SERVER_IP` (Caddy 301s www → apex)
- Wait for propagation, then Caddy obtains the Let's Encrypt certificate on
  first HTTPS hit.

## 5. Verify (agent)

```bash
./scripts/verify-live.sh oss-singularity/website SERVER_IP
```

Checks DNS, HTTPS edge, forge API, www redirect, `git ls-remote`, TLS cert.

## 6. Operate

- Backups: install the cron line from `scripts/backup.sh` header (root).
- First repos: push mirrors from `/home/claudiu/git/mirrors/` (see
  `scripts/mirror-github.sh`) into the forge orgs, then retarget checkouts.
  Done 2026-09-22: all 16 oss-singularity repos live in the `OSS-Singularity`
  org (heads + tags; forgejo rejects GitHub `refs/pull/*` as hidden refs, so
  `push-mirrors.sh` pushes explicit refspecs).
- This repository is self-hosted at `https://oss-oo.io/OSS-OO/laboratory` —
  `main` is protected there (PR + green CI required), making oss-oo.io
  self-hosting from day one.
- Independent watchdog: follow [the observer contract](WATCHDOG.md). Use shadow
  until the separate notification evidence, stopped/finished/reconciled prior
  owner and fresh eight-key baseline are verified. Source tests or an ingest
  ACK alone do not authorize cutover or prove DOWN/recovery email delivery.
  Before an observer source upgrade, verify the restricted systemd credential
  ACL/mount context with a disposable credential and the exact accepted reader;
  preserve that no-network proof before a silent shadow run. Credential
  compatibility alone does not authorize a timer, notification or owner change.
  Direct HTTP workers in a declared service context must also refuse an
  inadmissible required MQTT credential before any outgoing observation.

## Rollback

1. DNS: restore `A oss-oo.io → 188.68.47.93` (netcup redirect). Propagation
   minutes-to-hours depending on TTL.
2. Server: `docker compose -f compose.yml -f compose.prod.yml down` — data
   stays in volumes; server can be destroyed any time after backups exist.