libglacier-ng/CLAUDE.md

CLAUDE.md

Context for Claude Code sessions on libglacier-ng: the package manager for Everest Linux. Kept as a handoff, so a fresh session starts with full continuity instead of re-deriving everything. Last full rewrite: 2026-10-04, at the end of a long session. Update it as you go.

Start here

1. ROADMAP.txt: what's left, in priority order, plus the decisions that are still open (section 6). Pick work from it.
2. This file: how things work, what the user wants, how to test.
3. The design docs, which are authoritative for their topics and not repeated here:
   • SECURITY_DESIGN.txt: integrity verification, scope separation, refusal matrix, syspkg's privilege model.
   • PACKAGE_SCOPES.txt: layout, link routing, /etc config handling.
   • IMAGE_DESIGN.txt: PROPOSAL for system images (bootstrap OUTPUT, the sysroot library change). Open decisions are in section 9.
   • GLACIER_BOOTSTRAP.txt: PLAN for glacier-bootstrap. Prerequisites are in section 7, decisions in section 9 (9c and 9d are decided).
   • SECURITY_REVIEW_2026-07-16.md: older review findings.

Nothing from the 2026-10-04 session is committed. libglacier-ng is on branch dev with a large uncommitted diff (src/, the docs). Commit only when the user asks.

The three projects

Path	What	Git?
~/Projects/libglacier-ng	C library, design docs, this file	yes (dev)
~/Projects/glacier-ng	Frontend: gpkg, gstore, syspkg, gbuild, gworld, lib/*.lua, tests/, Dockerfiles, pkgs/	no: back files up before editing (file.old; if that exists, .old2, .old3, ...)
~/Projects/everest	The distro build: everest.sh build|initramfs|boot, packages, bootable QEMU image	no

glacier-ng/lib/glacier/ holds .sos built on the host (glibc). They won't load on musl. Every Alpine image and test builds libglacier-ng from source instead.

What this project is

A from-scratch, security-focused Linux distribution (Everest) with its own package manager (libglacier-ng + gpkg/gstore/syspkg) and build tooling (gbuild/gworld). It's a C library, LuaJIT+FFI CLI tools and bash build scripts. The user writes it; Claude implements and designs alongside them.

How the user wants things done

These are preferences stated in sessions. Follow them.

• Output format: every message uses glacier's lg_printf levels: [x] is error (2), [!] warning (1), [i] info (0), [✓] success (3). Messages are capitalized. Exact strings the user chose include Cannot open system index: Permission denied (no period) and Nothing to do. (success level).
  • The pre-trust-check die() in syspkg/gstore reproduces the [x] format byte for byte, because it can't load libglacier_log.so before that file has been verified.
• No hint messages. Don't print lines that point users to other flags or commands (for example "use -lO to ..."). The user called these clutter. A refusal states the fact, in one line. The user also removed a Run doas gstore -n ... follow-up hint from gstore themselves.
• Ask before settling a design choice that is the user's to make, and give a recommendation. They have answered every such question promptly. Don't silently pick a layout or semantics.
• Verify by running, in Docker or QEMU, never by inspection alone. For any security fix, also prove the test catches the bug: remove the defence in a throwaway copy and confirm the test fails ("mutation check"). Several defences here are verified that way.
• Use doas, not sudo. syspkg is never setuid.

Decisions the user has made (don't relitigate)

• Per-uid user layout (/glacier/usr/<uid>/..., root-owned parent). Both root and the user can create it: root runs gstore -n UID; a user runs doas gstore -n through a doas rule pinned to args -n (option "B").
• syspkg -n creates the top-level tree itself (GLACIER_BOOTSTRAP 9d).
• /etc config files: option 1. Copy on install, .new on conflict (GLACIER_BOOTSTRAP 9c).
• -O with -l = replace. -O displaces non-glacier files, and -lO also replaces an installed different version (upgrade or downgrade from a local file). Plain -l with another version installed refuses in one line.
• KEEP_OLD in /etc/glacier/glacier.conf: the number of backups of replaced versions to keep per package (0 = none).
  • It has no default. It ships as KEEP_OLD = 1.
  • If it's missing or invalid, an operation that would replace a version errors. Only then: other operations don't need it.
  • "Backed up" means the old package's files are compressed into a .gpkg. A restore interface comes later.
• gpkg -u / syspkg -u upgrade from a REMOTE repository, and gpkg -f is -l but fetched by name from a remote repository. Both wait for remote repos + signing (ROADMAP 3.1). Don't implement them against local files.
• everest-bootscripts is the name of the base /etc package. Its source lives in ~/Projects/everest/pkgsrc/.

Core architecture

gl_context_t (transaction.h) is the central abstraction. Fields: uid, root_path, index_path, store_path, links_path, mode (GL_CTX_LIVE/GL_CTX_STAGE), scope (GL_SCOPE_USR/GL_SCOPE_SYS), lock_fd, and overwrite (last field, zeroed by the init functions).

Layout. Each scope owns ONE base directory holding {index,store,links,stage,old,lock,backup}. Every create, rename and lock a transaction does stays inside it.

Scope	Base dir	Ownership
GL_SCOPE_USR	/glacier/usr/<uid>/	uid, mode 0700, inside a root-owned 0755 /glacier/usr
GL_SCOPE_SYS	/glacier/sys/	root

• The helpers gl_usr_base, gl_usr_base_check and the layout comment are in common.h.
• gl_init_stage_context refuses a user base that isn't a real directory (lstat) owned by its uid with no group/other bits, or whose /glacier/usr is loosened. gl_usr_base_status exports that check, so gpkg can say why before any prompt.

Transaction lifecycle. gl_init_live_context / gl_init_stage_context → gl_commit_transaction or gl_abort_transaction.

• Stage init takes the lock FIRST, then clears any leftover stage.
• Stage contexts hardlink-seed index/store from live, so removing a live package inside a transaction needs no special casing. Seeding copies dotfiles: only . and .. are skipped.
• A stage on another filesystem fails at seeding with GL_TXN_ERR_XDEV.
• Commit: rename-based promotion (stage → live, previous live → old/), then fsync the base. Then gl_relink_store + gl_rebuild_index regenerate links and index from a fresh live context and rescan the WHOLE store (every commit relinks every package, so link-time logic must be idempotent). old/ is then deleted.

Link routing (per package, in gl_link_pkg):

Scope	Repo	Destination
USR	any	<base>/links
SYS	base	/usr
SYS	anything else	/glacier/sys/links

• A package's leading usr/ is stripped.
• In system scope, a package's top-level etc/ is never linked: it's COPIED into /etc (see /etc config handling below).

Repos. base (the minimal system; boot-critical by definition, a repo-level rule), extra, community. extra and community behave identically.

Hidden files in the store (dot-prefixed, so linking, index rebuild and the store scanners skip them, while seeding carries them through transactions):

File	Purpose
<ver>/.glacier-manifest	the package's manifest, saved at install, used by gl_backup_pkg
<ver>/.glacier-fresh	one-shot "installed in this transaction"; link conflicts are only reported when present
<ver>/.glacier-overwrite	one-shot -O permission for this package's next link pass
<store>/.displaced/<abs path>	originals displaced by -O, restored when the link that replaced them is pruned
<store>/.etc-state/<path>	the last version of each /etc file a package offered

Configuration. /etc/glacier/glacier.conf (libconfig syntax).

• It's read with gl_conf_get_int(path, key, &out) in pkg.c. Missing file, missing key and wrong type are distinct statuses, and there are no defaults.
• The shipped copy is glacier-ng/etc/glacier/glacier.conf. Every Dockerfile installs it.

Safety invariants: read before touching links/symlink code

/usr is NOT glacier-exclusive territory. Treat every link destination as possibly shared:

• Ownership test. is_glacier_symlink(path, store_prefix) is the only test for "did glacier create this". Never touch anything it returns false for, except through -O's displace/restore.
• Overwriting. gl_link_pkg won't overwrite a non-glacier path without the -O marker. It never descends through another package's directory symlink into that package's store directory.
• Pruning. gl_relink_store PRUNES stale links; it never wipes a links destination. A link is stale only if its own store target is gone (checked with lstat, not by following it): a package symlink may legitimately dangle.
• Stage links. The stage links tree is never seeded.
• Extraction never writes through a symlink.
  • Symlink entries are created only after all regular content.
  • Any entry whose parent is a symlink is refused (crosses_symlink, plus O_NOFOLLOW).
  • Any entry with .. or an absolute path refuses the whole package.
  • A failed extraction removes its half-made version directory.
• Recursive deletes. gl_rmdir_recursive skips only ./... Every caller passes a glacier-only path; keep it that way. gl_delete_user lstat-checks its target first, because a symlinked base would otherwise have its target emptied.
• Wide wrappers. Don't add convenience wrappers that silently touch a broader filesystem scope than asked. This codebase has been bitten twice by exactly that.

Feature reference (current behaviour)

Integrity verification (gpkg_common.lua: verify_and_copy, verify_all):

• Each package is copied into a private mktemp -d directory while being hashed with libglacier_verify.so. The result is checked against FILE.sha256 (sha256:<hex>).
• Everything after that (summary, manifest read, gl_install_pkg) reads ONLY the copy.
• A missing or malformed sidecar counts as a mismatch and is refused before the prompt; one bad package refuses the whole batch. No bypass exists.
• This lives in the CLIs' shared module, not the library (as SECURITY_DESIGN section 4 specifies), so a third-party caller of gl_install_pkg is not verified (ROADMAP 3.2).
• The base→USR refusal (GL_INSTALL_ERR_SCOPE = 7), by contrast, is in the library.

-O / displace:

• A displaced foreign file or symlink moves to <store>/.displaced/: renamed, or on EXDEV copied (keeping mode, owner and mtime) and then unlinked.
• Directories, special files and paths that already have a backup are still refused.
• In user scope -O is effectively a no-op, because the links tree is rebuilt from scratch on every commit.

Version replacement (-lO, M.install in gpkg_common.lua):

1. Installed entries are read from the live index's current file (repo::name::version lines).
2. A same name+repo with a different version is a replacement. Without -O it's refused; with -O, keep_old() reads KEEP_OLD, and the summary shows old -> new.
3. In the transaction: gl_backup_pkg(live, repo, name, X.gpkg.part), then gl_remove_pkg from the stage, then install.
4. After the commit: rename .part → <scope>/backup/<repo>/<name>/<name>-<ver>.gpkg, write its sidecar, and run gl_prune_backups(dir, KEEP_OLD) (newest by nanosecond mtime).

• On any abort, finish() deletes the .part.
• A backup is a complete installable .gpkg, so -lO backup.gpkg restores it.

/etc config handling (install_config / prune_config in istoreutils.c), system scope only:

Situation	Result
/etc/F missing, no record	install
missing, record exists	the admin deleted it: stays deleted
unchanged since glacier wrote it	updated in place
admin-modified, or never glacier's	never touched; a package change is offered ONCE as F.new
no package ships it any more	removed if unchanged, kept otherwise

User-scope etc/ is still linked (ROADMAP 2.9).

Top-level tree. gl_create_fs_tree(root, verbose) (exported) creates the merged-/usr tree: /bin and /sbin → usr/bin, /lib → usr/lib, /usr/sbin → bin, all RELATIVE links. It also creates /usr/{bin,lib}, /etc, /dev, /proc, /sys, /run, /tmp and /var/tmp (1777), /var, /root (0700) and /home.

• It creates only what's missing, of any type.
• gl_init_sys runs it before writing the index. The index marks -n as complete, so a second syspkg -n is refused: Cannot create /glacier/sys: already exists.
• It takes a root prefix, ready for syspkg --root.

Package symlinks are extracted verbatim and linked like files. Hardlink, device and fifo entries are skipped. add_to_archive (used by backups and gstore -b) archives symlinks.

CLIs (glacier-ng)

Tool	Flags	Runs as
gpkg	-l FILE... -x NAME... -O -S -V -h -v; -f/-u are stubs	the user; per-user scope only, NEVER constructs GL_SCOPE_SYS
syspkg	-n -l FILE... -x NAME... -O -S -V -h -v	root via doas; -h/-v without root
gstore	-n [UID] -b UID OUTFILE -x UID -s (stub) -V -h -v	-n needs root (direct, or doas gstore -n for yourself)

• gpkg: modifiers -O/-S/-V are applied in a pre-pass, so their position doesn't matter (-lO == -Ol). It finds its user via /proc/self/loginuid, falling back to getuid().
• syspkg, -n: creates the tree plus /glacier/sys.
• syspkg as non-root: -n/-l/-x are refused after argument parsing with Cannot open system index: Permission denied.
• gstore, -n without a UID, under doas: acts for the invoking user (loginuid, then DOAS_USER, then getuid). The doas rule permit nopass :wheel as root cmd /path/gstore args -n is the boundary that stops a user naming another UID.
• Shared code: lib/gpkg_common.lua (M.install, M.remove, verification, replacement, backups).
• The -x summary comes from the live index; a name that isn't installed fails before the prompt.
• Hardened loading (syspkg and gstore): #!/usr/bin/luajit -E, package.path and every ffi.load pinned to their own realpath, and when euid is 0, every ancestor directory and every loaded .lua/.so must be root-owned and not group/world-writable. The preamble is DUPLICATED in both files on purpose: change one, change both.
• gpkg uses the older CWD-relative convention (require("lib/..."), ffi.load("./lib/glacier/...")): run it from its own directory.
• Declarations: lib/glacier_cdef.lua is the single source of truth for every ffi.cdef. tests/glacier_ffi.lua is only a wrapper around it. Declaration ORDER matters: a declaration using gl_context_t must come after its typedef.

Build tooling

• gbuild (bash, the only copy is glacier-ng/gbuild):
  • -n -V -g -c -d -m -D -C -b -i -o -p -j -F -t -x -u -r -s; -v/--version prints the version, --verbose turns on verbose output.
  • Autodetects the build system, with autotools checked BEFORE a bare Makefile.
  • Writes NAME-VER.gpkg plus a .sha256 sidecar, using gsha256sum from $GSHA256SUM_BIN, else <script dir>/build/bin, else PATH.
  • There is NO -P system-profile support, and gworld has no SYS_PROFILE field (ROADMAP 4.1). The target libc is decided by where gbuild runs: Alpine for musl.
• gworld (Lua): batch orchestrator, reads recipes/*.recipe, sorts topologically by DEPS, runs gbuild per package.
• ~/Projects/everest: the gbuild command for each package lives in scripts/container.sh; everest.sh runs it inside an Alpine image (Dockerfile.build).
• Kernel version: linux 6.12.112; plain v6.12 doesn't build with GCC 15.

libglacier-ng build

• Separate .so per module (libglacier_log, _pkg, _istoreutils, _transaction, _dag, _verify). This split is deliberate: a third-party program links only what it needs.
• The combined libglacier-ng.so (make libglacier-ng) exists only for the FFI tests.
• libglacier_verify.so is private: no public header.
• Warnings: config.mk builds with -Werror. -Wno-format-truncation is set because that warning is noise; -Wstringop-truncation stays fatal.
• Makefile quirk: make log builds only log.o; the shared object comes from make log_test (or plain make).

Testing

Run everything in throwaway containers. Several suites write to /usr, /etc and /glacier.

FFI suite (glacier-ng/tests/*.lua; 8 files, all passing):

docker run --rm --cap-add SYS_ADMIN -v ~/Projects/glacier-ng:/src:ro \
  -v ~/Projects/libglacier-ng:/libsrc:ro alpine:latest sh -c '
  apk add --no-cache build-base luajit luajit-dev libarchive-dev libconfig-dev linux-headers tar >/dev/null
  cp -a /libsrc /build && cd /build && make clean && make && make libglacier-ng
  cp -a /src /g && cd /g/tests && GLACIER_TEST_CONFIRM=yes \
    LIBGLACIER_SO=/build/build/lib/shared/libglacier-ng.so luajit run_all.lua'

GNU tar is required: the hostile-archive fixtures use tar --append.

Shell suites (glacier-ng/tests/*.sh):

Suite	Image
syspkg_alpine.sh, overwrite_alpine.sh	alpine:latest with /src, /libsrc and /t.sh mounted (see each header); overwrite_alpine.sh runs both as-is and with --tmpfs /glacier:exec
verify_alpine.sh, user_layout_alpine.sh, upgrade_alpine.sh	glacier-alpine_root

Build the glacier-alpine_root image with:

cd ~/Projects/glacier-ng && docker build -f Dockerfile.alpine_root \
  --build-context libglacier=../libglacier-ng -t glacier-alpine_root .
docker run --rm -v "$PWD/tests/NAME.sh:/t.sh:ro" glacier-alpine_root sh /t.sh

Interactive images: ./run_docker.sh Dockerfile.{alpine,alpine_root,arch,debian}. run_docker.sh passes the libglacier build context. alpine, arch and debian default to user glacier (uid 1000, in wheel); alpine_root defaults to root. All of them have:

• root-owned /app;
• no /glacier until gstore -n / syspkg -n;
• the doas rule plus permit nopass glacier;
• glacier.conf installed.

Boot test: cd ~/Projects/everest && ./everest.sh boot boots QEMU (KVM). Press Enter for a root shell; poweroff -f to leave.

Bug history: so these don't get reintroduced

All fixed in the current tree.

1. gl_abort_transaction never released the lock (wrong variable). Now open(O_CREAT) + flock(), and lock files are never unlinked.
2. No EXDEV check before renaming. Commit stat()s both sides and returns GL_TXN_ERR_XDEV.
3. Segfault on install: the transaction target didn't link istoreutils/log.
4. Stale links post-commit: gl_link_pkg ran during staged installs, embedding stage paths. It now runs only for LIVE contexts, and links regenerate post-commit.
5. Dangling links were never cleaned on removal → prune_stale_links.
6. GL_TXN_ERR_RENAME after the first commit: a leftover old/. It's now wiped first.
7. -Wstringop-truncation: strncpy(..., PATH_MAX - 1) could leave strings unterminated. Use snprintf(dst, sizeof dst, "%s", ...).
8. gbuild detected a bare Makefile before configure. The fix had been lost from the tree and was re-applied 2026-10-04.
9. The index had no version field; it's now repo::pkgname::version.
10. [The code this was about is NOT in the tree: ROADMAP 4.1.] A recipe field silently never parsed and fell back to a plausible default. Remember the class.
11. gsha256sum.c included sha256.c instead of .h.
12. gbuild told users to run gpkg -f (a stub) instead of gpkg -l.
13. gpkg -f's stub referenced an undefined uid.
14. Path traversal from manifests/archives: PKG_NAME/PKG_REPO must pass gl_valid_pkg_ident, and ../absolute entries refuse the package. Finding 1 (self-declared base) stays open until signing.
15. Stage seeding and gl_rmdir_recursive skipped ALL dot-entries: dotfiles vanished, and old/ couldn't be removed (bug 6 again).
16. Per-user transactions could never work for a real user (shared writable parents needed). Fixed by the per-uid layout. Found alongside:
    • gstore -n never created parent directories;
    • the stage was wiped before the lock was taken;
    • cross-filesystem seeding was reported as MKDIR;
    • gl_restore_istore doubled the uid and accepted .. entries;
    • gstore loaded the combined .so via LIBGLACIER_SO from the environment.
17. Package symlinks were silently dropped at extraction, and only regular files were linked, so musl's loader was never installed. Alpine-based tests passed only because Alpine provides its own loader.
18. add_to_archive (gstore -b, backups) silently dropped symlinks.
19. gpkg acted on -l as soon as it parsed it, so -lO ignored its O.
20. tests/glacier_ffi.lua duplicated the cdefs and had drifted (it was missing GL_INSTALL_ERR_SCOPE). It's now a wrapper.
21. Link conflicts were re-reported on every system commit, for every package (it now uses the .glacier-fresh marker).
22. A failed extraction left a half-made version directory, so in a live context every retry reported ALREADY_INSTALLED.
23. Backups written in the same second were pruned in the wrong order (whole-second mtimes).

Conventions

• C style: tabs, but the codebase is inconsistent; match the surrounding function.
• Paths: PATH_MAX stack buffers plus snprintf for path construction.
• Lua requires: gpkg uses bare CWD-relative requires. syspkg and gstore pin their paths (see CLIs).
• Lua numbers in C varargs: a Lua number reaches C varargs as a double, so never pass one to lg_printf("%d"). Use %s with tostring(), or ffi.new("int", n).
• Shell commands from Lua: always shell_quote interpolated paths (syspkg runs as root).
• Verify by running. Several real bugs here were found only by executing something.