+--------------------------------------------+ | Building Everest system images | +--------------------------------------------+ STATUS: PROPOSAL. Nothing here is implemented, and the items in section 9 are decisions still to be made, not settled ones. This is the "dedicated design pass" SECURITY_DESIGN.txt section 10 says the installer needs, started from the build side rather than the install side. Companion documents: SECURITY_DESIGN.txt (refusal matrix, syspkg privilege model, section 10's installer reference material), PACKAGE_SCOPES.txt (symlink routing, why /usr is shared territory). +--------------------------------------------+ | 0. Why now | +--------------------------------------------+ The first real `syspkg -l musl` (Alpine container, 2026-10-04) worked exactly as designed, and showed why testing on a foreign distro has run out of road: Alpine's own musl-dev already owns 231 of the paths musl links into /usr, so every one was refused, and the -O/--overwrite flag had to be added just to make musl's files the live ones. In a system glacier builds itself, /usr starts empty. No conflicts, no need for -O, and every file in /usr is a glacier symlink by construction. That is the actual target. Every test so far has run against a host distro's /usr instead. +--------------------------------------------+ | 1. What "system image" means here | +--------------------------------------------+ SECURITY_DESIGN.txt section 10 records that the old "system image tarball" idea was DISCARDED in favor of package-list-driven bootstrapping (glacier-bootstrap TEMPLATE_INDEX TARGET). This proposal does not bring it back. The distinction: REJECTED an image as the INPUT to an install: a prebuilt rootfs tarball unpacked onto the target. Files arrive without passing through syspkg, so nothing in it is under the integrity guarantee. PROPOSED an image as the OUTPUT of a bootstrap: a target root built entirely by `syspkg` installing a defined package list, every package through the normal verified path. Packing that directory (tarball, OCI, disk image) is just shipping a result, not a second way of producing one. So "build a system image" = "run the bootstrap against a directory, then optionally pack it". The bootstrap is the thing being designed; the packing formats are thin (section 6). DECISION NEEDED (9a): confirm this reading of section 10. +--------------------------------------------+ | 2. Pipeline | +--------------------------------------------+ image definition (section 5: profile + package list) | gworld -P PROFILE build every recipe the image needs | -> pkgs/*.gpkg + *.gpkg.sha256 syspkg --root T -n create T/glacier/sys/... | syspkg --root T -l PKG... install the package list into T | (base -> T/usr, others -> | T/glacier/sys/links) finalize T /etc skeleton, doas.conf, etc. | pack T dir | rootfs tarball | OCI | disk Everything above already exists except `--root` (section 3/4), the image definition (section 5), finalize, and pack. The bootstrapping-the-bootstrapper problem (PACKAGE_SCOPES.txt section 6) mostly dissolves: syspkg, libglacier-ng, luajit and the rest of the glacier tooling are just base-repo packages in the list. The build host's syspkg installs them into T through the same verified path as everything else. The trust edge moves from "whatever lays syspkg down" to "the build host's syspkg". That's an honest edge, but a much smaller and more auditable one than an unverified bootstrap step. +--------------------------------------------+ | 3. Library prerequisite: install root | +--------------------------------------------+ Every path in the library is a hardcoded absolute path today: pkg.c's gl_root/gl_sys*/gl_usr*, transaction.c's GL_*_BASE and GL_*_INDEX/STORE/LINKS macros, derive_base_paths, the stage and old paths in init/commit, the fsync_dir list, GL_BASE_LINKS_DEST ("/usr"). ctx->root_path exists but is always "/glacier" and nothing reads it as a prefix. Building into T needs TWO distinct roots, and conflating them is the dangerous mistake: install root where bytes are written: T (e.g. /mnt/everest) runtime root what the paths mean once the / image is booted / chrooted Symlink TARGETS must use runtime paths (/glacier/sys/store/...), or the image breaks the moment it's booted. Every FILESYSTEM OPERATION must use install-root paths (T/glacier/sys/store/..., T/usr/...). The specific trap: prune_stale_links decides a glacier symlink is stale by stat()ing through it. For an absolute target, that resolves against the HOST's filesystem, not T. That would prune live links because the host doesn't have the package, or worse, keep stale ones because it does. is_glacier_symlink is fine: it only compares the readlink() string against the runtime store prefix. But every "does the target exist" check has to become "does T + target exist". The same goes for restore_displaced and for anything else that follows a link. Proposed shape: - gl_context_t gains `char sysroot[PATH_MAX]` ("" == "/"; appended at the end, matched in lib/glacier_cdef.lua and tests/glacier_ffi.lua). - New gl_init_live_context_at / gl_init_stage_context_at / gl_init_sys_at taking a sysroot. The existing functions become wrappers passing "/", so every current caller is unchanged. - One helper, gl_at(ctx, runtime_path, out), used for EVERY filesystem operation. Runtime paths stay what's stored and compared. - GL_BASE_LINKS_DEST resolves to sysroot + "/usr". Never the host's /usr when a sysroot is set. The non-negotiable test: an image build must leave the host's /usr and /glacier byte-for-byte untouched. That is a test assertion, not a code review item. Same class of bug as the old gl_relink_store wipe: a path assumption quietly stops holding. +--------------------------------------------+ | 4. syspkg --root | +--------------------------------------------+ syspkg gains `-r/--root DIR`, which selects the *_at init variants. gpkg/gstore do NOT get it. Per-user scope has no business in an image build, and section 5 of SECURITY_DESIGN.txt keeps those tools from ever reaching system scope. Privilege: still root via doas, still no setuid. New trust check, same reasoning as syspkg's existing self-check: DIR and every ancestor must be root-owned and not group/world-writable. A root process writing into a user-controlled tree is a symlink-swap away from writing somewhere else entirely (replace T/usr with a symlink to the host's /usr between two syscalls). Same refusal style, same message format. The hash refusal matrix applies unchanged. Everything in an image is either base (always refuse unverified) or system scope (always refuse unverified), so an image build can only ever contain packages with valid sidecars. That's the property we want; it means section 2's gworld step must produce sidecars for everything (gbuild already does). +--------------------------------------------+ | 5. Image definition = template index | +--------------------------------------------+ SECURITY_DESIGN.txt section 10 noted that the old guide's "template index" questions (32-bit libs? which libc? SELinux? init system?) are the same axes SYS_PROFILE already encodes. Proposal: a template index IS a profile plus an explicit package list, in the same KEY=value format the recipe parser already reads: # images/minimal.image NAME=minimal SYS_PROFILE=x86_64-musl PKGS=musl busybox luajit libarchive zlib libconfig libglacier-ng glacier-ng doas RECIPES=recipes/ No runtime dependency resolution yet. Manifests DO carry PKG_DEPS and PKG_CONFS, and libglacier parses them, but nothing consumes them and every package leaves them empty. (DEPS in recipes is build ordering.) So the list is explicit and complete. GLACIER_BOOTSTRAP.txt section 4 proposes checking it against PKG_DEPS rather than auto-resolving. Remember gworld bug 10: a field that silently never parses and falls back to a plausible default. Unknown keys and an empty PKGS must be hard errors. +--------------------------------------------+ | 6. Output formats | +--------------------------------------------+ In order of usefulness right now: a. Directory. What the pipeline produces anyway. chroot-able immediately, which is the first milestone (section 8). b. Rootfs tarball -> `docker import`. Replaces the Alpine-based test Dockerfiles with a real Everest userland. Ends the glibc-vs-musl .so mismatch and the missing-linux-headers class of problem, because the test environment is Everest. Every syspkg/gpkg test then runs against a /usr glacier built. c. Bootable disk image (kernel, bootloader, partition layout per the old guide's /boot + optional separate /glacier partition). Later, and blocked on there being kernel/bootloader recipes at all. Reproducibility, so two builds of the same definition can be compared: index files are named by wall-clock time (make_index_name), and tar records mtimes and ownership. Honour SOURCE_DATE_EPOCH in make_index_name, and pack with sorted entries, clamped mtimes and numeric root:root ownership. +--------------------------------------------+ | 7. Building without a host /usr | +--------------------------------------------+ Consequences of /usr starting empty that the design has to own: - Strict linking. In an image build, any "Refusing to overwrite non-glacier path" means two packages in the list conflict, or finalize wrote somewhere it shouldn't. It should FAIL the build, not warn (an image-build flag on the library, or syspkg treating the warning count as an error under --root). -O must never be used in an image build; with an empty /usr it has nothing legitimate to do. - Paths the system needs that no package ships: /etc (passwd, group, doas.conf), /dev /proc /sys mountpoints, /tmp, /var. Either a `base` package (e.g. `filesystem`, Arch's approach) or the finalize step. Recommendation: a package, so it's verified and owned like everything else. Finalize then shrinks to things that genuinely vary per image (hostname, users). - /bin, /lib, /sbin. PACKAGE_SCOPES routes base to /usr only. Either the filesystem package ships /bin -> usr/bin style merged-usr symlinks, or every #! /bin/sh breaks. Recommendation: merged /usr, shipped by the filesystem package. +--------------------------------------------+ | 8. Milestones | +--------------------------------------------+ 1. sysroot in the library (section 3), plus the host-untouched test. No CLI changes, testable through FFI alone. 2. syspkg --root (section 4), with the root-owned-target check. 3. A `filesystem` base package (section 7). 4. images/minimal.image + a driver script (gworld -> syspkg --root -> finalize). Milestone: `chroot T /bin/sh` works, and `chroot T syspkg -l` installs a package from inside. 5. Tarball + `docker import`; switch the test suite to run on it. 6. Disk images: blocked on kernel/bootloader recipes and on the installer side of section 10 (partitioning, the separate /glacier partition question). Hash verification at install time (SECURITY_DESIGN.txt section 4) should land before or alongside milestone 2. An image build is "base-repo packages, the strictest tier" by definition, so it's the first real consumer of the refusal matrix. +--------------------------------------------+ | 9. Open decisions | +--------------------------------------------+ a. Section 1's reading of SECURITY_DESIGN section 10: images as bootstrap OUTPUT are fine, images as install INPUT stay rejected. b. Template index format: the KEY=value file in section 5, or something else. c. `filesystem` package vs finalize script for /etc and friends, and merged /usr (section 7). d. Build privilege: building as real root inside a throwaway container (works today), or rootless via user namespaces (unshare -r) later. Rootless changes nothing in section 4's checks, since inside the namespace the tree IS root-owned, but it does need the packing step to write numeric root ownership regardless of who built it. e. Whether the image's own index/store should be built with the image's final paths only, or also carry provenance: which SYS_PROFILE and which image definition produced it. CLAUDE.md already lists "manifests don't record SYS_PROFILE" as open. An image is where that gap starts to hurt.