feat(guix): carry prebuilt runtime and recover preserved state

5 files changed, 240 insertions, 11 deletions

diff --git a/docs/CHAT-PRODUCTION.txt b/docs/CHAT-PRODUCTION.txt
index fdb48a6..2507e07 100644
--- a/docs/CHAT-PRODUCTION.txt
+++ b/docs/CHAT-PRODUCTION.txt
@@ -106,8 +106,8 @@ DEPLOY AND CREATE THE REAL OWNER FILE
     firewall rules. Caddy obtains and renews public certificates automatically.
     Public certificate validation requires standard reachable ACME ports.
 
-    The systemd user service runs ./bin/stack up --production at boot and
-    ./bin/stack down on stop. Podman restarts
+    The supplied systemd user service uses ./bin/run for up --production and down;
+    the launcher selects an included Guix runtime when present. Podman restarts
     failed containers. Complete matching-architecture releases need no compilation
     or registry pull. Check all four services after activation:
 
diff --git a/docs/GUIX.txt b/docs/GUIX.txt
new file mode 100644
index 0000000..05aa572
--- /dev/null
+++ b/docs/GUIX.txt
@@ -0,0 +1,207 @@
+Copyright (C) 2026 Doan Luu Duc Tai <doan@luuduct.ai>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as
+published by the Free Software Foundation, either version 3 of the
+License, or (at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+GUIX RECOVERY
+
+    Goal: one .tar.gz, unpack/bootstrap, one command restores and starts services.
+    Private state is GPG-encrypted inside the tarball. Bootstrap, binaries and
+    public release/source are plaintext. Outer tarball is not encrypted or signed.
+    Trust its origin; SHA256SUMS detects corruption, not malicious replacement.
+    Keep decryption key/passphrase separately; never send them through chat.
+
+    Guix supplies Podman/tools. Existing Caddy/cgit/SearXNG/chat OCI applications,
+    static musl/OpenSSL builds, domains, HTTPS/WKD and crypto remain.
+    No new Rust/npm dependency, Guix System install, or native service rewrite.
+    One root .env and public .env.example; databases/TLS/owner/history: var/data.
+
+BUILD LOCAL
+
+        sh infra/guix/build.sh
+        cargo run --release --locked -- release
+
+    Needs working Guix daemon, x86_64 Linux and rootless Podman prerequisites.
+    Channel commits: bin/guix-channels.scm. time-machine builds the manifest's
+    bin/guix-runtime.tar.gz. Release runs full checks and packages matching source,
+    binaries, browser assets and saved images. No compilation on recovery host.
+    Podman info smoke test is not clean-host/container lifecycle proof.
+
+    Ordinary pack uses canonical /gnu/store. Former -RR wrapper creates one-ID
+    user namespace; privileged helpers cannot map host subordinate IDs there.
+    Builder's canonical store could bypass relocation and hide this defect.
+    https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-pack.html
+    https://guix.gnu.org/manual/devel/en/html_node/Replicating-Guix.html
+    https://raw.githubusercontent.com/containers/podman/main/docs/tutorials/rootless_tutorial.md
+
+MAKE CARRY FILE
+
+    Production: create consistent encrypted export; copy and verify it locally:
+
+        ./bin/stack backup --include-images
+
+    Bundle a trusted freshly checked release with that export:
+
+        sh infra/guix/bundle.sh /path/backup.tar.gz.gpg /path/release.tar.gz /path/recovery.tar.gz
+
+    Bundler executes the trusted release's CLI to validate it with restore
+    --release. Backup-kind archives, private .env/var/data, missing runtime/chat
+    artifacts and missing saved images are refused before publication.
+    CLI, scripts, config, runtime and provenance all come from the same release.
+    Original release archive is unchanged and includes corresponding source and
+    LICENSE. Original encrypted backup retains prior source/work and rollback.
+    Runtime is duplicated outside release for bootstrap simplicity; extra disk
+    space avoids another installer/format. One carry file remains sufficient.
+
+    Old backups remain supported; release must implement this migration's
+    --release validation and runtime export. Blindly reusing old management binary
+    loses Guix files on the next backup. Restore now activates matching new source
+    and binary, including canonical infra/guix/run.sh needed by later backups.
+
+BOOTSTRAP HOST; RESTORE
+
+    Fresh x86_64 Linux: trusted shell, GNU coreutils/tar/gzip/findutils/diffutils,
+    awk, util-linux flock, sudo/root access. Bootstrap refuses /var/guix and foreign
+    /gnu trees. Installs only /gnu/store entries; never host /bin or /etc links.
+
+        tar -xzf recovery.tar.gz
+        sudo ./stack-recovery/bootstrap
+
+    Root-owned non-writable closure entries are staged on /gnu's filesystem,
+    published without overwriting, then recorded by pack digest. Retry reconciles
+    unfinished installs; later packs retain old entries. Root install records and
+    lock: /gnu/.stack-runtimes. Target needs no Guix daemon/downloads. Existing
+    managed Guix hosts need separate planning; do not merge unmanaged store items.
+
+    Administrator also supplies privileged newuidmap/newgidmap, >=65536 subordinate
+    UIDs/GIDs, user namespaces, writable login XDG_RUNTIME_DIR, low-port sysctl
+    net.ipv4.ip_unprivileged_port_start <=80. DNS/firewall/ports remain host duties.
+    OPERATIONS.txt Debian setup installs distro Podman too: existing fallback,
+    not evidence of a Guix-only target. Service user imports GPG key locally.
+
+        ./stack-recovery/restore /home/ops/stack
+
+    New destination required. Capsule checks hashes/bootstrap/prerequisites,
+    decrypts through Rust manifest/path validation, requires saved backup images,
+    then activates its paired release under the capsule's Guix runtime/config.
+    Activation installs new public source/binaries/images, preserves .git, .env,
+    var/data, and starts four services. Source worktree becomes new release;
+    original source remains in encrypted backup/history. Only tested version
+    pairs establish application/data compatibility; format-2 integrity cannot.
+
+    Failed activation retains data plus private var/guix-recovery intent marker.
+    Fix cause and rerun the same restore command with the same capsule/destination.
+    Matching backup/release digests permit resume without decrypting/restoring
+    twice; unrelated destinations are refused. Recovery lock excludes concurrent
+    activation and is closed in child processes. Success removes pending marker.
+    Do not replace an interrupted activation with plain up; it cannot finish
+    copying a release. Completed recoveries use normal management commands:
+
+        /home/ops/stack/bin/run up --production
+
+    Launcher supplies --root and Guix-only CONTAINERS_CONF; do not pass --root.
+    Runtime expands privately into content-addressed var/guix; old packs remain.
+    Activation atomically replaces the archive/provenance files, including existing
+    read-only packs; regression test covers replacement followed by export/restore.
+    Direct bin/stack and releases without pack use host tools. Guix symlinks remain
+    inside opaque runtime pack; Rust application extractor still rejects links.
+    Boot setup remains explicit: systemd user service plus linger, OPERATIONS.txt.
+    Unit uses bin/run at ~/stack; adapt paths for another destination.
+
+CHECKS; DEPLOYMENT GATE
+
+        sh infra/guix/test.sh
+        cargo stack check
+
+    Shell tests need bubblewrap. Synthetic packs test actual bootstrap/filesystem
+    behavior. Synthetic recovery tests actual orchestration, failed activation,
+    resume, intent/lock refusal, private sentinels, and child FD closure. Fake tools
+    do not prove real encryption, image import, services or kernel UID mapping.
+    Rust tests prove archive-kind refusal before publication and runtime roundtrip.
+    Actual static CLI also checked synthetic bundle validation/publication/privacy.
+    No strict Rust/Clippy/Rustdoc, typed ESLint, LibreJS or crypto gate weakened.
+    Existing documented lint/dependency exceptions remain unchanged.
+
+    Clean-host test exposed missing distro container signature policy. Image load
+    now supplies a temporary policy: reject by default; accept local Docker/OCI
+    archives only. Export supplies a separate temporary policy accepting only
+    containers-storage; registry transport remains rejected. Verified archive
+    manifests and architecture/layer/image-ID locks
+    still gate startup. Registry transport is not accepted by this load policy;
+    global host policy stays unchanged. Policies are generated by AGPL Rust code;
+    the foreign schema rejects comment/legal metadata, so no headerless source file
+    is added. The initial failed VM retained decrypted data and pending intent.
+
+    A second clean-host check found legacy backups had collapsed Caddy's /data
+    and /config into one caddy/ tree. After manifest verification, backup restore
+    moves credentials into data/caddy and autosave into config/caddy; bytes and
+    permissions are retained. Conflicting old/new layouts are refused before
+    publishing a destination. New exports copy /data/. and /config/. separately.
+
+    Before production, in order:
+    1. Full checked release with real Guix runtime and source/binary pair.
+    2. Fresh host without Guix/distro Podman/Git/GPG: offline bootstrap/restore;
+       four services, exact images, detached UID/network helpers.
+    3. .env count, domains, HTTPS/WKD, original owner fingerprint; unchanged chat
+       rows and history decryption with both original participant identities.
+    4. Backup recovered stack, second recovery, reboot startup.
+    5. First host-Podman activation and Guix runtime upgrades. Ordinary direct
+       activate retains its process PATH; deploy through matching release runtime.
+    6. Fresh verified independent production backup; deploy only prebuilt release;
+       verify public routes/state, retain prior release/runtime and encrypted export.
+
+    Rollback: stop new containers before old runtime binds production ports.
+    Keep .env/var/data; never delete closure entries used by running processes.
+
+VERIFIED — 2026-11-01
+
+    Offline KVM guest: official Debian 12 x86_64 image, no initial /gnu, Guix,
+    Podman, Git or GPG. Host supplies only documented kernel/account prerequisites.
+    Network restricted before bootstrap; readonly carry file plus separately
+    forwarded GPG agent; no private key copied. Bootstrap and one restore command
+    loaded four saved images and started all services without source builds.
+
+    Strict HTTPS verified all four domains using original certificates; both WKD
+    routes matched the original key. All 651 immutable snapshot files matched
+    after legacy Caddy path normalization: 18 active TLS keys, 16 certificates.
+    All 126 release source/binary files matched. Original .env, owner fingerprint
+    and every canonical SQLite table matched. Production snapshot contained zero
+    conversations/messages; original real-message decryption therefore untestable.
+    Disposable Chromium/Firefox tests verified both participant identities,
+    54-message history, encrypted snapshots/recovery and GNU LibreJS 7.21.1.
+
+    Full local release gates passed live npm/Rust advisory checks, strict fmt,
+    Clippy, Rustdoc, typed ESLint and both TypeScript compilers; no suppressions.
+    Required OpenSSL 3.6.5 hybrid suites, protected OpenPGP v6 identity, wrong-pass
+    and tamper controls passed. Synthetic shell tests cover bootstrap collisions,
+    interrupted recovery/retry, lock exclusion and child descriptor closure.
+
+    Runtime: 500687877 bytes, SHA256:
+        c5c7b849da152ee7a98b01602a1e91285ce4e3a80021a9cecb5f1e594dfc7810
+    Guix channel 230aa373f315f247852ee07dff34146e9b480aec; 113 store entries.
+    Git 2.52.0, GnuPG 2.4.7, Podman 5.7.0, conmon 2.1.13, crun 1.25.1.
+    Empty helper lists force PATH lookup, selecting packed conmon/crun. Ordinary
+    host config stays unchanged. Rootless cleanup handles immutable user-owned
+    temporary trees without changing symlink targets or the published pack.
+    https://github.com/containers/podman/blob/v5.7.0/vendor/go.podman.io/common/pkg/config/config.go#L849-L875
+    https://github.com/containers/podman/blob/v5.7.0/cmd/podman/images/save.go#L91-L94
+
+    Deployment gate above still applies to each carried artifact. Retain complete
+    original worktree, binary/launcher, installed user unit and engine storage
+    independently before activation. Reload the installed unit after switching to
+    bin/run. For rollback stop Guix containers first, retain their engine tree,
+    restore prior engine path, binary/launcher and unit; invoke prior activation
+    with host PATH and Guix CONTAINERS_CONF overrides unset. Keep current .env and
+    var/data. An old release alone does not remove a newer Guix launcher/runtime.
+    Private evidence: var/guix-*.log and .tools/guix-production/. These are ignored
+    and include private state hashes; do not publish them with public source.
diff --git a/docs/MAP.txt b/docs/MAP.txt
index 6f5e13d..de546b8 100644
--- a/docs/MAP.txt
+++ b/docs/MAP.txt
@@ -15,7 +15,7 @@ along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 FILE MAP
 
-    Public tree: every tracked file, including this map.
+    Public tree: every tracked file plus pending Guix source/docs and this map.
     Static inventory; update when public paths change. Root: stack/.
     Private/generated trees below list roles only; contents stay unlisted.
 
@@ -67,6 +67,7 @@ stack/  Personal service source, build recipes and recovery tooling.
 |   |-- CHAT-OPENSSL-MIGRATION.txt  OpenSSL crypto migration evidence, dependencies, reviewed exceptions.
 |   |-- CHAT-PRODUCTION.txt  Chat release packaging, HTTPS activation, owner setup, production recovery.
 |   |-- CONFIGURATION.txt  Supported root settings, validation, public/private state boundaries.
+|   |-- GUIX.txt  Experimental Guix capsule, canonical-store bootstrap, gates and blockers.
 |   |-- LINTING.txt  Strict Rust/browser checks, generated-code limits, justified exceptions.
 |   |-- MAP.txt  This public-file tree and private/runtime directory roles.
 |   |-- OPERATIONS.txt  Stack development, encrypted backup/restore, host setup, deployment.
@@ -90,6 +91,17 @@ stack/  Personal service source, build recipes and recovery tooling.
 |   |   |   `-- supervisord.conf  Run nginx and fcgiwrap inside the cgit container.
 |   |   `-- chat/  Prebuilt chat image recipe.
 |   |       `-- Containerfile  Package prebuilt static chat server and web/crypto assets.
+|   |-- guix/  Experimental Guix runtime and recovery capsule tooling.
+|   |   |-- bootstrap.sh  Root-owned canonical closure installation; refuse managed/foreign stores.
+|   |   |-- build.sh  Capture Guix channels; build canonical runtime pack; probe Podman info.
+|   |   |-- bundle.sh  Validate paired release; wrap encrypted backup, release and bootstrap.
+|   |   |-- containers.conf  Guix-only PATH-selected conmon/crun and rootless networking.
+|   |   |-- manifest.scm  Guix packages for the portable container/runtime tools.
+|   |   |-- restore.sh  Check capsule/host, decrypt safe restore, require images, start production.
+|   |   |-- run.sh  Select content-addressed Guix runtime; launch stack with explicit root.
+|   |   |-- test-bootstrap.sh  Real bootstrap in isolated filesystem using synthetic closure packs.
+|   |   |-- test-recovery.sh  Synthetic release activation, resume, intent/locks and private-state preservation.
+|   |   `-- test.sh  Bootstrap, immutable cleanup, integrity, overwrite refusal and launcher fixtures.
 |   |-- searxng/  Search service configuration.
 |   |   `-- settings.yml  SearXNG defaults: private instance, rate limiting, HTML search.
 |   |-- Caddyfile  Production domain routing, HTTPS, WKD and service reverse proxies.
@@ -100,11 +112,11 @@ stack/  Personal service source, build recipes and recovery tooling.
 |   `-- stack.service  Systemd user boot unit for ~/stack start/stop.
 |-- src/  Management executable source.
 |   |-- app/  Management command implementation modules.
-|   |   |-- archive.rs  Consistent snapshots, encrypted exports and validated directory restores.
+|   |   |-- archive.rs  Consistent snapshots, encrypted exports, legacy Caddy layout and validated restores.
 |   |   |-- chat.rs  Chat build/package artifacts, container settings and owner administration.
 |   |   |-- cli.rs  Management subcommands, flags and argument validation.
-|   |   |-- files.rs  Safe filesystem/archive operations, manifests and hashes.
-|   |   |-- images.rs  Export/import exact container images; enforce platform and content locks.
+|   |   |-- files.rs  Safe filesystem/archive operations, manifests, hashes and runtime copies.
+|   |   |-- images.rs  Export/import saved images with archive-only policy, platform and content locks.
 |   |   |-- process.rs  Run child processes; capture output and report failures/status.
 |   |   |-- project.rs  Discover/lock project; settings, Podman commands, service lifecycle.
 |   |   |-- release.rs  Build checked release archives; validate/install production artifacts.
@@ -152,7 +164,7 @@ PRIVATE AND GENERATED PATHS — NOT PUBLIC SOURCE
     .env                 Sole private root configuration; never commit.
     .git/                Local Git objects, refs, hooks and repository settings.
     .tools/              Downloaded toolchains, build tools, sources and caches.
-    bin/                 Prebuilt management/chat commands and packaged chat assets.
+    bin/                 Prebuilt management/chat commands, launcher and Guix pack.
     dist/                Locally built release tarballs and checksum files.
     node_modules/        Locked npm development dependencies; recreated by npm ci.
     target/              Cargo build output, WASM targets and generated rustdoc.
@@ -166,10 +178,15 @@ PRIVATE AND GENERATED PATHS — NOT PUBLIC SOURCE
         repos/           Served bare Git repositories; separate from root .git/.
         searxng/         Runtime search configuration and persistent service files.
       exports/           Encrypted application backups and checksum files.
+      guix/              Unpacked content-addressed runtime packs; old packs retained.
       incoming/          Copied release archives and temporary deployment bootstrap.
       podman/            Recreated container storage/run state; never transfer as backup.
       graphify/          Public-source graph index, visual map and extraction artifacts.
 
     Other var/ paths may hold generated fixtures or temporary work. This map does
     not enumerate private files. Backups must keep .env and var/data; public Git
-    alone cannot recover services. See OPERATIONS.txt.
+    alone cannot recover services. See OPERATIONS.txt and GUIX.txt.
+
+    Experimental host integration: /gnu/store holds canonical Guix packages;
+    /gnu/.stack-runtimes holds root-owned package install records and lock.
+    Neither holds application configuration, databases or TLS keys.
diff --git a/docs/OPERATIONS.txt b/docs/OPERATIONS.txt
index 55dc81c..91ab10b 100644
--- a/docs/OPERATIONS.txt
+++ b/docs/OPERATIONS.txt
@@ -18,6 +18,7 @@ STACK OPERATIONS
     Root: ~/stack. Develop on local Linux; production: ops@23.155.44.67.
     Public source: https://git.doan.luuduct.ai/stack.
     GNU Affero General Public License, version 3 or later.
+    Experimental Guix capsule: GUIX.txt. Commands below use host Podman.
 
 KEEP
 
@@ -68,7 +69,7 @@ CHECK LOCAL
 
         cargo stack check
 
-    Checks formatting, all workspace Clippy targets/features, tests, private
+    Checks Guix shell/bootstrap fixtures (bubblewrap), formatting, workspace Clippy, tests, private
     rustdoc, advisories, duplicate versions, sources, licenses. cargo-about
     regenerates THIRD-PARTY-LICENSES.html. Format with cargo fmt --all.
     Dependency changes: update Cargo.lock deliberately; run full check; commit
@@ -184,8 +185,8 @@ NEW PRODUCTION HOST
         systemctl --user enable --now stack.service
         systemctl --user status stack.service
 
-    Unit assumes ~/stack; edit paths for other location. Uses bin/stack.
-    Linger keeps user service alive
+    Unit assumes ~/stack; edit paths for other location. Current unit uses bin/run:
+    confirm launcher exists before enabling. Linger keeps user service alive
     without login. Containers use unless-stopped restart policy.
 
     Optional former VPS SSH Git path; administrator only:
diff --git a/docs/POLICY.txt b/docs/POLICY.txt
index dea595b..747b774 100644
--- a/docs/POLICY.txt
+++ b/docs/POLICY.txt
@@ -32,6 +32,8 @@ PROJECT CONVENTIONS
     Required legal headers are the exception. Prose uses .txt; generated HTML
     license page is an explicit exception. Rust doc attributes supply crate
     metadata without comments.
+    Temporary container import/export policies are generated protocol JSON: its strict
+    foreign schema rejects legal metadata. AGPL notice stays in the Rust generator.
 
     One root .env holds deployment overrides/configured secrets. Never commit
     .env or var/. Publish only root .env.example; ignore variants/nested examples.
@@ -76,6 +78,8 @@ PORTABILITY
     packages, subordinate UID/GID allocations, one low-port sysctl, user linger,
     systemd user-service link. Host SSH keys, firewall policy, accounts, and disks
     are host administration, excluded from application exports.
+    Experimental Guix capsule also installs canonical /gnu/store packages and
+    /gnu/.stack-runtimes install records. Private application state stays in stack/.
 
     Releases include static x86_64 Linux management/chat commands, WASM browser
     crypto, exact corresponding sources, native linux/amd64 container images.