feat(guix): carry prebuilt runtime and recover preserved state

1 files changed, 207 insertions, 0 deletions

diff --git a/docs/GUIX.txt b/docs/GUIX.txt
new file mode 100644
index 0000000..05aa572
--- /dev/null
+++ b/docs/GUIX.txt
@@ -0,0 +1,207 @@
+Copyright (C) 2026 Doan Luu Duc Tai <doan@luuduct.ai>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as
+published by the Free Software Foundation, either version 3 of the
+License, or (at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+GUIX RECOVERY
+
+    Goal: one .tar.gz, unpack/bootstrap, one command restores and starts services.
+    Private state is GPG-encrypted inside the tarball. Bootstrap, binaries and
+    public release/source are plaintext. Outer tarball is not encrypted or signed.
+    Trust its origin; SHA256SUMS detects corruption, not malicious replacement.
+    Keep decryption key/passphrase separately; never send them through chat.
+
+    Guix supplies Podman/tools. Existing Caddy/cgit/SearXNG/chat OCI applications,
+    static musl/OpenSSL builds, domains, HTTPS/WKD and crypto remain.
+    No new Rust/npm dependency, Guix System install, or native service rewrite.
+    One root .env and public .env.example; databases/TLS/owner/history: var/data.
+
+BUILD LOCAL
+
+        sh infra/guix/build.sh
+        cargo run --release --locked -- release
+
+    Needs working Guix daemon, x86_64 Linux and rootless Podman prerequisites.
+    Channel commits: bin/guix-channels.scm. time-machine builds the manifest's
+    bin/guix-runtime.tar.gz. Release runs full checks and packages matching source,
+    binaries, browser assets and saved images. No compilation on recovery host.
+    Podman info smoke test is not clean-host/container lifecycle proof.
+
+    Ordinary pack uses canonical /gnu/store. Former -RR wrapper creates one-ID
+    user namespace; privileged helpers cannot map host subordinate IDs there.
+    Builder's canonical store could bypass relocation and hide this defect.
+    https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-pack.html
+    https://guix.gnu.org/manual/devel/en/html_node/Replicating-Guix.html
+    https://raw.githubusercontent.com/containers/podman/main/docs/tutorials/rootless_tutorial.md
+
+MAKE CARRY FILE
+
+    Production: create consistent encrypted export; copy and verify it locally:
+
+        ./bin/stack backup --include-images
+
+    Bundle a trusted freshly checked release with that export:
+
+        sh infra/guix/bundle.sh /path/backup.tar.gz.gpg /path/release.tar.gz /path/recovery.tar.gz
+
+    Bundler executes the trusted release's CLI to validate it with restore
+    --release. Backup-kind archives, private .env/var/data, missing runtime/chat
+    artifacts and missing saved images are refused before publication.
+    CLI, scripts, config, runtime and provenance all come from the same release.
+    Original release archive is unchanged and includes corresponding source and
+    LICENSE. Original encrypted backup retains prior source/work and rollback.
+    Runtime is duplicated outside release for bootstrap simplicity; extra disk
+    space avoids another installer/format. One carry file remains sufficient.
+
+    Old backups remain supported; release must implement this migration's
+    --release validation and runtime export. Blindly reusing old management binary
+    loses Guix files on the next backup. Restore now activates matching new source
+    and binary, including canonical infra/guix/run.sh needed by later backups.
+
+BOOTSTRAP HOST; RESTORE
+
+    Fresh x86_64 Linux: trusted shell, GNU coreutils/tar/gzip/findutils/diffutils,
+    awk, util-linux flock, sudo/root access. Bootstrap refuses /var/guix and foreign
+    /gnu trees. Installs only /gnu/store entries; never host /bin or /etc links.
+
+        tar -xzf recovery.tar.gz
+        sudo ./stack-recovery/bootstrap
+
+    Root-owned non-writable closure entries are staged on /gnu's filesystem,
+    published without overwriting, then recorded by pack digest. Retry reconciles
+    unfinished installs; later packs retain old entries. Root install records and
+    lock: /gnu/.stack-runtimes. Target needs no Guix daemon/downloads. Existing
+    managed Guix hosts need separate planning; do not merge unmanaged store items.
+
+    Administrator also supplies privileged newuidmap/newgidmap, >=65536 subordinate
+    UIDs/GIDs, user namespaces, writable login XDG_RUNTIME_DIR, low-port sysctl
+    net.ipv4.ip_unprivileged_port_start <=80. DNS/firewall/ports remain host duties.
+    OPERATIONS.txt Debian setup installs distro Podman too: existing fallback,
+    not evidence of a Guix-only target. Service user imports GPG key locally.
+
+        ./stack-recovery/restore /home/ops/stack
+
+    New destination required. Capsule checks hashes/bootstrap/prerequisites,
+    decrypts through Rust manifest/path validation, requires saved backup images,
+    then activates its paired release under the capsule's Guix runtime/config.
+    Activation installs new public source/binaries/images, preserves .git, .env,
+    var/data, and starts four services. Source worktree becomes new release;
+    original source remains in encrypted backup/history. Only tested version
+    pairs establish application/data compatibility; format-2 integrity cannot.
+
+    Failed activation retains data plus private var/guix-recovery intent marker.
+    Fix cause and rerun the same restore command with the same capsule/destination.
+    Matching backup/release digests permit resume without decrypting/restoring
+    twice; unrelated destinations are refused. Recovery lock excludes concurrent
+    activation and is closed in child processes. Success removes pending marker.
+    Do not replace an interrupted activation with plain up; it cannot finish
+    copying a release. Completed recoveries use normal management commands:
+
+        /home/ops/stack/bin/run up --production
+
+    Launcher supplies --root and Guix-only CONTAINERS_CONF; do not pass --root.
+    Runtime expands privately into content-addressed var/guix; old packs remain.
+    Activation atomically replaces the archive/provenance files, including existing
+    read-only packs; regression test covers replacement followed by export/restore.
+    Direct bin/stack and releases without pack use host tools. Guix symlinks remain
+    inside opaque runtime pack; Rust application extractor still rejects links.
+    Boot setup remains explicit: systemd user service plus linger, OPERATIONS.txt.
+    Unit uses bin/run at ~/stack; adapt paths for another destination.
+
+CHECKS; DEPLOYMENT GATE
+
+        sh infra/guix/test.sh
+        cargo stack check
+
+    Shell tests need bubblewrap. Synthetic packs test actual bootstrap/filesystem
+    behavior. Synthetic recovery tests actual orchestration, failed activation,
+    resume, intent/lock refusal, private sentinels, and child FD closure. Fake tools
+    do not prove real encryption, image import, services or kernel UID mapping.
+    Rust tests prove archive-kind refusal before publication and runtime roundtrip.
+    Actual static CLI also checked synthetic bundle validation/publication/privacy.
+    No strict Rust/Clippy/Rustdoc, typed ESLint, LibreJS or crypto gate weakened.
+    Existing documented lint/dependency exceptions remain unchanged.
+
+    Clean-host test exposed missing distro container signature policy. Image load
+    now supplies a temporary policy: reject by default; accept local Docker/OCI
+    archives only. Export supplies a separate temporary policy accepting only
+    containers-storage; registry transport remains rejected. Verified archive
+    manifests and architecture/layer/image-ID locks
+    still gate startup. Registry transport is not accepted by this load policy;
+    global host policy stays unchanged. Policies are generated by AGPL Rust code;
+    the foreign schema rejects comment/legal metadata, so no headerless source file
+    is added. The initial failed VM retained decrypted data and pending intent.
+
+    A second clean-host check found legacy backups had collapsed Caddy's /data
+    and /config into one caddy/ tree. After manifest verification, backup restore
+    moves credentials into data/caddy and autosave into config/caddy; bytes and
+    permissions are retained. Conflicting old/new layouts are refused before
+    publishing a destination. New exports copy /data/. and /config/. separately.
+
+    Before production, in order:
+    1. Full checked release with real Guix runtime and source/binary pair.
+    2. Fresh host without Guix/distro Podman/Git/GPG: offline bootstrap/restore;
+       four services, exact images, detached UID/network helpers.
+    3. .env count, domains, HTTPS/WKD, original owner fingerprint; unchanged chat
+       rows and history decryption with both original participant identities.
+    4. Backup recovered stack, second recovery, reboot startup.
+    5. First host-Podman activation and Guix runtime upgrades. Ordinary direct
+       activate retains its process PATH; deploy through matching release runtime.
+    6. Fresh verified independent production backup; deploy only prebuilt release;
+       verify public routes/state, retain prior release/runtime and encrypted export.
+
+    Rollback: stop new containers before old runtime binds production ports.
+    Keep .env/var/data; never delete closure entries used by running processes.
+
+VERIFIED — 2026-11-01
+
+    Offline KVM guest: official Debian 12 x86_64 image, no initial /gnu, Guix,
+    Podman, Git or GPG. Host supplies only documented kernel/account prerequisites.
+    Network restricted before bootstrap; readonly carry file plus separately
+    forwarded GPG agent; no private key copied. Bootstrap and one restore command
+    loaded four saved images and started all services without source builds.
+
+    Strict HTTPS verified all four domains using original certificates; both WKD
+    routes matched the original key. All 651 immutable snapshot files matched
+    after legacy Caddy path normalization: 18 active TLS keys, 16 certificates.
+    All 126 release source/binary files matched. Original .env, owner fingerprint
+    and every canonical SQLite table matched. Production snapshot contained zero
+    conversations/messages; original real-message decryption therefore untestable.
+    Disposable Chromium/Firefox tests verified both participant identities,
+    54-message history, encrypted snapshots/recovery and GNU LibreJS 7.21.1.
+
+    Full local release gates passed live npm/Rust advisory checks, strict fmt,
+    Clippy, Rustdoc, typed ESLint and both TypeScript compilers; no suppressions.
+    Required OpenSSL 3.6.5 hybrid suites, protected OpenPGP v6 identity, wrong-pass
+    and tamper controls passed. Synthetic shell tests cover bootstrap collisions,
+    interrupted recovery/retry, lock exclusion and child descriptor closure.
+
+    Runtime: 500687877 bytes, SHA256:
+        c5c7b849da152ee7a98b01602a1e91285ce4e3a80021a9cecb5f1e594dfc7810
+    Guix channel 230aa373f315f247852ee07dff34146e9b480aec; 113 store entries.
+    Git 2.52.0, GnuPG 2.4.7, Podman 5.7.0, conmon 2.1.13, crun 1.25.1.
+    Empty helper lists force PATH lookup, selecting packed conmon/crun. Ordinary
+    host config stays unchanged. Rootless cleanup handles immutable user-owned
+    temporary trees without changing symlink targets or the published pack.
+    https://github.com/containers/podman/blob/v5.7.0/vendor/go.podman.io/common/pkg/config/config.go#L849-L875
+    https://github.com/containers/podman/blob/v5.7.0/cmd/podman/images/save.go#L91-L94
+
+    Deployment gate above still applies to each carried artifact. Retain complete
+    original worktree, binary/launcher, installed user unit and engine storage
+    independently before activation. Reload the installed unit after switching to
+    bin/run. For rollback stop Guix containers first, retain their engine tree,
+    restore prior engine path, binary/launcher and unit; invoke prior activation
+    with host PATH and Guix CONTAINERS_CONF overrides unset. Keep current .env and
+    var/data. An old release alone does not remove a newer Guix launcher/runtime.
+    Private evidence: var/guix-*.log and .tools/guix-production/. These are ignored
+    and include private state hashes; do not publish them with public source.