libglacier-ng/GLACIER_BOOTSTRAP.txt

+--------------------------------------------+
| glacier-bootstrap                          |
+--------------------------------------------+

STATUS: PLAN. Nothing here is implemented. Section 9 lists the
decisions still open; everything else is a recommendation to argue
with, not a settled design.

glacier-bootstrap takes a template index and builds a chrootable
Everest system in a target directory, from either a local directory of
packages or a remote repository. It is the "glacier-bootstrap
TEMPLATE_INDEX TARGET" tool named in SECURITY_DESIGN.txt section 10, and
the `syspkg --root` step of IMAGE_DESIGN.txt's pipeline.

Companion documents: IMAGE_DESIGN.txt (why images are bootstrap OUTPUT,
the sysroot library change), SECURITY_DESIGN.txt (refusal matrix,
syspkg's privilege model), PACKAGE_SCOPES.txt (link routing).


+--------------------------------------------+
| 1. What it does                            |
+--------------------------------------------+

    glacier-bootstrap [-n] [-t] (-d PKGDIR | -r URL) TEMPLATE TARGET

    -d PKGDIR   packages come from a local directory (gbuild/gworld
                output: NAME-VER.gpkg plus NAME-VER.gpkg.sha256)
    -r URL      packages come from a remote repository (section 5)
    -n          dry run: parse, resolve, fetch, verify, print the plan;
                never touch TARGET
    -t          after installing, smoke-test: chroot TARGET /bin/sh -c true

Sequence:

    1. parse TEMPLATE                                       (section 3)
    2. resolve: check the package list is complete          (section 4)
    3. fetch into a private cache; verify every hash        (section 5)
    4. doas syspkg --root TARGET -n                         create the tree
    5. doas syspkg --root TARGET --strict -l PKG...         ONE transaction
    6. optional smoke test (-t)

Step 5 is one transaction on purpose: TARGET ends up either the whole
system or an empty tree, never half a system.


+--------------------------------------------+
| 2. Trust and privilege                     |
+--------------------------------------------+

glacier-bootstrap NEVER WRITES INTO TARGET ITSELF. Every byte under
TARGET goes through `doas syspkg --root TARGET`.

  - SECURITY_DESIGN.txt section 5: syspkg owns ALL system-scope
    mutation, "one tool to secure, one tool to audit." A bootstrapper
    that extracted packages itself would be the second, unaudited entry
    point that section warns about.
  - glacier-bootstrap runs UNPRIVILEGED. Parsing, resolving, fetching and
    verifying need no root, so it needs no third copy of the hardened
    root-loading preamble (syspkg and gstore already carry it; CLAUDE.md
    flags the duplication). Root is borrowed only for the two syspkg
    calls.
  - Its own verification is early-fail, for good error messages. syspkg's
    is authoritative: syspkg re-hashes every file at the moment it
    reads it (refusal matrix, SECURITY_DESIGN section 4). So a file
    swapped in the cache between bootstrap's check and syspkg's read is
    caught. The cache is the invoking user's (0700). The only party who
    could swap files there is the person who can already run doas
    syspkg.

Where trust comes from, per source:

  local (-d)       The sidecar next to each .gpkg, same as syspkg today.
                   INTEGRITY ONLY, which is all the current tree offers
                   anywhere.

  remote, pinned   The TEMPLATE pins each package's sha256 (section 3).
                   glacier-bootstrap writes the sidecar FROM THE PIN, not
                   from anything the server sent. So syspkg's existing
                   "sidecar matches file" check enforces the template's
                   hash, with no syspkg change needed. The server is an
                   untrusted blob store. Authenticity is exactly as good
                   as the template file's, which you hold locally and can
                   review or diff.

  remote, signed   Later: a signed repo index (the agreed next major item
                   after the checkpoints). Only then may a template name a
                   remote package without a pin.

  remote, unpinned and unsigned: REFUSED. A .gpkg and its .sha256
  fetched from the same server prove nothing (SECURITY_DESIGN section 1:
  the hash travelling with the file).


+--------------------------------------------+
| 3. Template index format                   |
+--------------------------------------------+

Extends IMAGE_DESIGN.txt section 5's KEY=value proposal with one line
per package, so packages can carry pins:

    # templates/minimal.template
    NAME=minimal
    SYS_PROFILE=x86_64-musl
    PKG=filesystem 1.0.0
    PKG=musl 1.2.5 sha256:3a5f...
    PKG=busybox 1.37.0 sha256:91c0...

  - Exact versions only, no ranges. A template describes ONE system,
    reproducibly.
  - A pin is optional for -d (if present, it must ALSO match the
    sidecar) and required for -r until signed repos exist.
  - Strict parsing: unknown keys, a duplicate PKG name, a malformed pin
    and an empty package list are all hard errors. Remember gworld bug
    10: a field silently never parsing, with a default that looks right.
  - A `glacier-bootstrap --pin PKGDIR TEMPLATE` helper that fills pins in
    from local sidecars is the obvious way to produce a pinned template
    from a gworld build. Pinning then becomes a review step, not
    hand-copying hex.


+--------------------------------------------+
| 4. Resolution                              |
+--------------------------------------------+

Manifests already carry PKG_DEPS and PKG_CONFS (name + version), and
libglacier parses both into gl_pkg_t. Nothing consumes them today, and
every existing package leaves them empty. (IMAGE_DESIGN.txt section 5
says glacier has no runtime deps at all. The format exists; the use
doesn't.)

Recommendation: CHECK, don't auto-resolve.

  - Every PKG_DEPS entry of every listed package must itself be listed,
    with a matching version. Otherwise error: "busybox needs musl 1.2.5,
    which the template does not list."
  - No two listed packages may conflict (PKG_CONFS).
  - Each manifest's PKG_NAME/PKG_VER must match its template line. This
    catches a mis-named file in -d mode, where there may be no pin.

Why not pull dependencies in automatically: anything auto-added would be
an unpinned package the template's author never reviewed, which defeats
the trust model in section 2. Closure resolution can come with signed
repos.

SYS_PROFILE can't be checked against packages yet: manifests don't
record which profile they were built under (open item in CLAUDE.md).
Once they do, a mismatched package is a resolution error.


+--------------------------------------------+
| 5. Package sources                         |
+--------------------------------------------+

LOCAL (-d PKGDIR): a flat directory, exactly what gbuild emits —
NAME-VER.gpkg, NAME-VER.gpkg.sha256. No index file. The template's
package list says what to take from the directory.

REMOTE (-r URL): the same flat layout over HTTPS:

    URL/NAME-VER.gpkg

  - Fetch by shelling out to curl (`--proto =https --fail`), the same
    way gbuild shells out to git and tar, into
    $XDG_CACHE_HOME/glacier-bootstrap/ (0700) under a temporary name.
    Hash it, and only then rename it into place.
  - Server-provided .sha256 files are never consulted (section 2).
  - Cache hits are re-hashed against the pin on every run. The cache is
    a download optimization, not a trusted store.
  - Later, with signing: URL/repo.idx plus its signature, listing name,
    version, repo and hash. The fetch path stays the same.


+--------------------------------------------+
| 6. What "chrootable" requires              |
+--------------------------------------------+

Found while planning, by reading the code and the packages in pkgs/.
Each one is a hard blocker for `chroot TARGET /bin/sh` working at all,
independent of glacier-bootstrap itself:

  a. [FIXED 2026-10-04 — extraction + linking, with both defenses
     below; tests/test_package_symlinks.lua] SYMLINKS IN PACKAGES ARE
     DROPPED. The extraction loop in
     gl_install_pkg handles only AE_IFDIR and AE_IFREG (istoreutils.c);
     symlink entries never reach the store. gl_link_pkg likewise links
     only regular files. Real packages need symlinks:
       - musl ships lib/ld-musl-x86_64.so.1 -> /usr/lib/libc.so. That is
         the dynamic loader every musl binary asks the kernel for. It is
         currently never installed; Alpine-based tests passed only
         because Alpine provides its own.
       - shared-library soname links (libz.so.1 -> libz.so.1.3.1);
       - busybox applets.
     Fix: extract symlink entries verbatim into the store, and link
     them like regular files. SECURITY TRAP to design around: an archive
     shipping `x -> /etc` followed by `x/passwd` writes through the
     symlink, as root, into the real /etc (classic tar-slip via
     symlink). Extraction must never write through a symlink: create
     symlinks only after every regular file, and/or refuse any entry
     whose parent path is a symlink in the store.

  b. [DONE 2026-10-04 — gl_create_fs_tree, run by syspkg -n; takes a
     root prefix ready for --root; tests/test_fs_tree.lua] NO TOP-LEVEL
     TREE. Base packages link only under /usr
     (PACKAGE_SCOPES section 3), and a leading "usr/" is the only prefix
     stripped. So musl's lib/ld-musl... would land at /usr/lib, while
     PT_INTERP says /lib/ld-musl-x86_64.so.1. TARGET needs merged /usr:
         /bin -> usr/bin   /lib -> usr/lib   /sbin -> usr/bin
     plus the mountpoints and dirs nothing ships: /dev /proc /sys /tmp
     (1777) /run /var /root /home /etc. No package can provide these
     under the current link rules (everything base goes under /usr).
     Recommendation: `syspkg -n` creates this skeleton as part of
     system init (under --root, and on a real system). Creating the
     system's skeleton is system-scope mutation, so it belongs to
     syspkg, not to the bootstrapper.

  c. [DONE 2026-10-04 — decision 9c, option 1; tests/test_config_files.lua]
     /etc HAS NO STORY. A base package's etc/passwd would link to
     /usr/etc/passwd. And config files can't simply be symlinks into
     the store anyway: editing them would modify the store and break
     its hashes. This needs its own decision (section 9c). For the first
     chroot milestone, the skeleton step can write a minimal
     /etc/passwd and /etc/group (root only), and /etc/profile.

  d. NO SHELL. The only packages that exist are musl, kilo and sbase,
     and sbase is coreutils with no sh. A busybox (or mksh/dash) recipe
     in base is needed.

  e. REPO TIERS. sbase's manifest says PKG_REPO = "core", which is not
     a tier (base / extra / community). Under system scope it would link
     into /glacier/sys/links, not /usr/bin. Rebuild it as base.

  f. [sidecars generated for the existing archives 2026-10-04] NO
     SIDECARS. None of pkgs/*.gpkg has a .sha256. They were built
     before checkpoint 2. Rebuild them with the current gbuild.


+--------------------------------------------+
| 7. Prerequisites, in order                 |
+--------------------------------------------+

  1. [DONE] Hash verification at install time (SECURITY_DESIGN
     section 4). Pins mean nothing until syspkg actually checks sidecars.
  2. [DONE] Symlink extraction + linking, with the write-through-symlink
     defense (6a). Needed for anything to run.
  3. sysroot in the library (IMAGE_DESIGN section 3), plus the "host
     /usr and /glacier untouched" test.
  4. syspkg --root DIR (IMAGE_DESIGN section 4), --strict (any link
     refusal fails the transaction), and the skeleton on -n (6b) [the
     skeleton is DONE; --root and --strict are not].
  5. Packages: busybox recipe (base), sbase rebuilt as base, every
     package rebuilt with sidecars (6d-f).
  6. glacier-bootstrap, local mode.
     MILESTONE: `glacier-bootstrap -t -d pkgs templates/minimal.template
     /mnt/everest` succeeds, `chroot /mnt/everest /bin/sh` gives a
     working shell, and the host is byte-identical before and after.
  7. Remote mode with pinned templates, plus the --pin helper.
  8. Signed repo index (separate design pass), then unpinned remote.

1 and 2 are independent and can go in either order. 3 and 4 are the
bulk of the library/syspkg work. 6 itself is small: it's mostly
orchestration over things that have to exist anyway.


+--------------------------------------------+
| 8. Implementation shape                    |
+--------------------------------------------+

  - LuaJIT, in glacier-ng, alongside gworld. It shares gworld's
    KEY=value parsing (factor that into lib/ rather than copy it), and
    reads manifests through libglacier's existing parser via FFI, so
    PKG_DEPS etc. parse exactly the way syspkg will see them.
  - Unprivileged, so the CWD-relative require convention is fine here.
  - Shells out to: curl (remote), doas syspkg, chroot (for -t, via doas).
  - Exit codes distinguish parse / resolve / fetch / verify / install
    failure, so an image build script can tell them apart.


+--------------------------------------------+
| 9. Open decisions                          |
+--------------------------------------------+

  a. IMAGE_DESIGN.txt section 9a still stands: images are bootstrap
     OUTPUT, never install INPUT. Everything here assumes yes.
  b. Template format (section 3): one PKG= line per package with an
     optional pin, or something else.
  c. DECIDED 2026-10-04: option 1, copy on install, .new on conflict.
     Implemented in istoreutils.c (install_config / prune_config): a
     system-scope package's top-level etc/ is copied into /etc, never
     linked. An admin-modified, admin-deleted or not-glacier-installed
     file is never overwritten; package changes to it are offered once
     as FILE.new. Unmodified files are updated in place and removed with
     their package. User scope still links etc/ as before (open).
  d. DECIDED 2026-10-04: syspkg -n creates the top-level tree itself
     (6b). Implemented as gl_create_fs_tree(root, verbose), called by
     gl_init_sys before the index is written.
  e. Dependency handling (section 4): check-only, as recommended, or
     automatic closure.
  f. Pinning helper: part of glacier-bootstrap (--pin) or of gworld
     (which already knows what it just built).