libglacier-ng/CLAUDE.md

# CLAUDE.md

Context for Claude Code sessions on libglacier-ng: the package manager
for Everest Linux. Kept as a handoff, so a fresh session starts with full
continuity instead of re-deriving everything. Last full rewrite:
2026-10-04, at the end of a long session. **Update it as you go.**

## Start here

1. `ROADMAP.txt`: what's left, in priority order, plus the decisions that
   are still open (section 6). Pick work from it.
2. This file: how things work, what the user wants, how to test.
3. The design docs, which are authoritative for their topics and not
   repeated here:
   - `SECURITY_DESIGN.txt`: integrity verification, scope separation,
     refusal matrix, syspkg's privilege model.
   - `PACKAGE_SCOPES.txt`: layout, link routing, `/etc` config handling.
   - `IMAGE_DESIGN.txt`: PROPOSAL for system images (bootstrap OUTPUT,
     the sysroot library change). Open decisions are in section 9.
   - `GLACIER_BOOTSTRAP.txt`: PLAN for `glacier-bootstrap`. Prerequisites
     are in section 7, decisions in section 9 (9c and 9d are decided).
   - `SECURITY_REVIEW_2026-07-16.md`: older review findings.

Nothing from the 2026-10-04 session is committed. libglacier-ng is on
branch `dev` with a large uncommitted diff (`src/`, the docs). Commit only
when the user asks.

## The three projects

| Path | What | Git? |
|---|---|---|
| `~/Projects/libglacier-ng` | C library, design docs, this file | yes (`dev`) |
| `~/Projects/glacier-ng` | Frontend: `gpkg`, `gstore`, `syspkg`, `gbuild`, `gworld`, `lib/*.lua`, `tests/`, Dockerfiles, `pkgs/` | **no**: back files up before editing (`file.old`; if that exists, `.old2`, `.old3`, ...) |
| `~/Projects/everest` | The distro build: `everest.sh build\|initramfs\|boot`, packages, bootable QEMU image | no |

`glacier-ng/lib/glacier/` holds `.so`s built on the host (glibc). They
won't load on musl. Every Alpine image and test builds libglacier-ng from
source instead.

## What this project is

A from-scratch, security-focused Linux distribution (Everest) with its own
package manager (libglacier-ng + `gpkg`/`gstore`/`syspkg`) and build tooling
(`gbuild`/`gworld`). It's a C library, LuaJIT+FFI CLI tools and bash build
scripts. The user writes it; Claude implements and designs alongside them.

## How the user wants things done

These are preferences stated in sessions. Follow them.

- **Output format:** every message uses glacier's `lg_printf` levels:
  `[x]` is error (2), `[!]` warning (1), `[i]` info (0), `[✓]` success (3).
  Messages are capitalized. Exact strings the user chose include
  `Cannot open system index: Permission denied` (no period) and
  `Nothing to do.` (success level).
  - The pre-trust-check `die()` in syspkg/gstore reproduces the `[x]`
    format byte for byte, because it can't load `libglacier_log.so`
    before that file has been verified.
- **No hint messages.** Don't print lines that point users to other flags
  or commands (for example "use -lO to ..."). The user called these
  clutter. A refusal states the fact, in one line. The user also removed a
  `Run doas gstore -n ...` follow-up hint from gstore themselves.
- **Ask before settling a design choice that is the user's to make**, and
  give a recommendation. They have answered every such question promptly.
  Don't silently pick a layout or semantics.
- **Verify by running, in Docker or QEMU, never by inspection alone.** For
  any security fix, also prove the test catches the bug: remove the
  defence in a throwaway copy and confirm the test fails ("mutation
  check"). Several defences here are verified that way.
- Use doas, not sudo. syspkg is never setuid.

## Decisions the user has made (don't relitigate)

- **Per-uid user layout** (`/glacier/usr/<uid>/...`, root-owned parent).
  Both root and the user can create it: root runs `gstore -n UID`; a user
  runs `doas gstore -n` through a doas rule pinned to `args -n`
  (option "B").
- **`syspkg -n` creates the top-level tree itself** (GLACIER_BOOTSTRAP 9d).
- **`/etc` config files: option 1.** Copy on install, `.new` on conflict
  (GLACIER_BOOTSTRAP 9c).
- **`-O` with `-l` = replace.** `-O` displaces non-glacier files, and
  `-lO` also replaces an installed *different version* (upgrade or
  downgrade from a local file). Plain `-l` with another version installed
  refuses in one line.
- **`KEEP_OLD`** in `/etc/glacier/glacier.conf`: the number of backups of
  replaced versions to keep per package (0 = none).
  - It has **no default.** It ships as `KEEP_OLD = 1`.
  - If it's missing or invalid, an operation that would replace a version
    errors. Only then: other operations don't need it.
  - "Backed up" means the old package's files are compressed into a
    `.gpkg`. A restore interface comes later.
- **`gpkg -u` / `syspkg -u` upgrade from a REMOTE repository**, and
  `gpkg -f` is `-l` but fetched by name from a remote repository. Both wait
  for remote repos + signing (ROADMAP 3.1). Don't implement them against
  local files.
- `everest-bootscripts` is the name of the base `/etc` package. Its source
  lives in `~/Projects/everest/pkgsrc/`.

## Core architecture

**`gl_context_t`** (transaction.h) is the central abstraction. Fields:
`uid`, `root_path`, `index_path`, `store_path`, `links_path`, `mode`
(`GL_CTX_LIVE`/`GL_CTX_STAGE`), `scope` (`GL_SCOPE_USR`/`GL_SCOPE_SYS`),
`lock_fd`, and `overwrite` (last field, zeroed by the init functions).

**Layout.** Each scope owns ONE base directory holding
`{index,store,links,stage,old,lock,backup}`. Every create, rename and lock
a transaction does stays inside it.

| Scope | Base dir | Ownership |
|---|---|---|
| `GL_SCOPE_USR` | `/glacier/usr/<uid>/` | uid, mode 0700, inside a root-owned 0755 `/glacier/usr` |
| `GL_SCOPE_SYS` | `/glacier/sys/` | root |

- The helpers `gl_usr_base`, `gl_usr_base_check` and the layout comment
  are in `common.h`.
- `gl_init_stage_context` refuses a user base that isn't a real directory
  (lstat) owned by its uid with no group/other bits, or whose
  `/glacier/usr` is loosened. `gl_usr_base_status` exports that check, so
  gpkg can say why before any prompt.

**Transaction lifecycle.** `gl_init_live_context` / `gl_init_stage_context`
→ `gl_commit_transaction` or `gl_abort_transaction`.
- Stage init takes the lock FIRST, then clears any leftover stage.
- Stage contexts hardlink-seed `index`/`store` from live, so removing a
  live package inside a transaction needs no special casing. Seeding
  copies dotfiles: only `.` and `..` are skipped.
- A stage on another filesystem fails at seeding with `GL_TXN_ERR_XDEV`.
- **Commit:** rename-based promotion (stage → live, previous live → `old/`),
  then fsync the base. Then `gl_relink_store` + `gl_rebuild_index`
  regenerate links and index from a *fresh live context* and rescan the
  WHOLE store (every commit relinks every package, so link-time logic must
  be idempotent). `old/` is then deleted.

**Link routing** (per package, in `gl_link_pkg`):

| Scope | Repo | Destination |
|---|---|---|
| USR | any | `<base>/links` |
| SYS | `base` | `/usr` |
| SYS | anything else | `/glacier/sys/links` |

- A package's leading `usr/` is stripped.
- In system scope, a package's top-level `etc/` is never linked: it's
  COPIED into `/etc` (see `/etc` config handling below).

**Repos.** `base` (the minimal system; boot-critical by definition, a
repo-level rule), `extra`, `community`. `extra` and `community` behave
identically.

**Hidden files in the store** (dot-prefixed, so linking, index rebuild and
the store scanners skip them, while seeding carries them through
transactions):

| File | Purpose |
|---|---|
| `<ver>/.glacier-manifest` | the package's manifest, saved at install, used by `gl_backup_pkg` |
| `<ver>/.glacier-fresh` | one-shot "installed in this transaction"; link conflicts are only reported when present |
| `<ver>/.glacier-overwrite` | one-shot `-O` permission for this package's next link pass |
| `<store>/.displaced/<abs path>` | originals displaced by `-O`, restored when the link that replaced them is pruned |
| `<store>/.etc-state/<path>` | the last version of each `/etc` file a package offered |

**Configuration.** `/etc/glacier/glacier.conf` (libconfig syntax).
- It's read with `gl_conf_get_int(path, key, &out)` in `pkg.c`. Missing
  file, missing key and wrong type are distinct statuses, and there are no
  defaults.
- The shipped copy is `glacier-ng/etc/glacier/glacier.conf`. Every
  Dockerfile installs it.

## Safety invariants: read before touching links/symlink code

`/usr` is NOT glacier-exclusive territory. Treat every link destination as
possibly shared:

- **Ownership test.** `is_glacier_symlink(path, store_prefix)` is the only
  test for "did glacier create this". Never touch anything it returns
  false for, except through `-O`'s displace/restore.
- **Overwriting.** `gl_link_pkg` won't overwrite a non-glacier path without
  the `-O` marker. It never descends through *another package's* directory
  symlink into that package's store directory.
- **Pruning.** `gl_relink_store` PRUNES stale links; it never wipes a links
  destination. A link is stale only if its own store target is gone
  (checked with `lstat`, not by following it): a package symlink may
  legitimately dangle.
- **Stage links.** The stage links tree is never seeded.
- **Extraction never writes through a symlink.**
  - Symlink entries are created only after all regular content.
  - Any entry whose parent is a symlink is refused (`crosses_symlink`,
    plus `O_NOFOLLOW`).
  - Any entry with `..` or an absolute path refuses the whole package.
  - A failed extraction removes its half-made version directory.
- **Recursive deletes.** `gl_rmdir_recursive` skips only `.`/`..`. Every
  caller passes a glacier-only path; keep it that way. `gl_delete_user`
  lstat-checks its target first, because a symlinked base would otherwise
  have its *target* emptied.
- **Wide wrappers.** Don't add convenience wrappers that silently touch a
  broader filesystem scope than asked. This codebase has been bitten twice
  by exactly that.

## Feature reference (current behaviour)

**Integrity verification** (`gpkg_common.lua`: `verify_and_copy`,
`verify_all`):
- Each package is copied into a private `mktemp -d` directory while being
  hashed with `libglacier_verify.so`. The result is checked against
  `FILE.sha256` (`sha256:<hex>`).
- Everything after that (summary, manifest read, `gl_install_pkg`) reads
  ONLY the copy.
- A missing or malformed sidecar counts as a mismatch and is refused before
  the prompt; one bad package refuses the whole batch. No bypass exists.
- This lives in the CLIs' shared module, not the library (as
  SECURITY_DESIGN section 4 specifies), so a third-party caller of
  `gl_install_pkg` is not verified (ROADMAP 3.2).
- The base→USR refusal (`GL_INSTALL_ERR_SCOPE = 7`), by contrast, *is* in
  the library.

**`-O` / displace:**
- A displaced foreign file or symlink moves to `<store>/.displaced/`:
  renamed, or on `EXDEV` copied (keeping mode, owner and mtime) and then
  unlinked.
- Directories, special files and paths that already have a backup are
  still refused.
- In user scope `-O` is effectively a no-op, because the links tree is
  rebuilt from scratch on every commit.

**Version replacement** (`-lO`, `M.install` in `gpkg_common.lua`):
1. Installed entries are read from the live index's `current` file
   (`repo::name::version` lines).
2. A same name+repo with a different version is a replacement. Without
   `-O` it's refused; with `-O`, `keep_old()` reads `KEEP_OLD`, and the
   summary shows `old -> new`.
3. In the transaction: `gl_backup_pkg(live, repo, name, X.gpkg.part)`, then
   `gl_remove_pkg` from the stage, then install.
4. After the commit: rename `.part` → `<scope>/backup/<repo>/<name>/<name>-<ver>.gpkg`,
   write its sidecar, and run `gl_prune_backups(dir, KEEP_OLD)` (newest by
   nanosecond mtime).
- On any abort, `finish()` deletes the `.part`.
- A backup is a complete installable `.gpkg`, so `-lO backup.gpkg`
  restores it.

**`/etc` config handling** (`install_config` / `prune_config` in
istoreutils.c), system scope only:

| Situation | Result |
|---|---|
| `/etc/F` missing, no record | install |
| missing, record exists | the admin deleted it: stays deleted |
| unchanged since glacier wrote it | updated in place |
| admin-modified, or never glacier's | never touched; a package change is offered ONCE as `F.new` |
| no package ships it any more | removed if unchanged, kept otherwise |

User-scope `etc/` is still linked (ROADMAP 2.9).

**Top-level tree.** `gl_create_fs_tree(root, verbose)` (exported) creates
the merged-/usr tree: `/bin` and `/sbin` → `usr/bin`, `/lib` → `usr/lib`,
`/usr/sbin` → `bin`, all RELATIVE links. It also creates `/usr/{bin,lib}`,
`/etc`, `/dev`, `/proc`, `/sys`, `/run`, `/tmp` and `/var/tmp` (1777),
`/var`, `/root` (0700) and `/home`.
- It creates only what's missing, of any type.
- `gl_init_sys` runs it before writing the index. The index marks `-n` as
  complete, so a second `syspkg -n` is refused: `Cannot create
  /glacier/sys: already exists`.
- It takes a root prefix, ready for `syspkg --root`.

**Package symlinks** are extracted verbatim and linked like files.
Hardlink, device and fifo entries are skipped. `add_to_archive` (used by
backups and `gstore -b`) archives symlinks.

## CLIs (glacier-ng)

| Tool | Flags | Runs as |
|---|---|---|
| `gpkg` | `-l FILE...` `-x NAME...` `-O` `-S` `-V` `-h` `-v`; `-f`/`-u` are stubs | the user; per-user scope only, NEVER constructs `GL_SCOPE_SYS` |
| `syspkg` | `-n` `-l FILE...` `-x NAME...` `-O` `-S` `-V` `-h` `-v` | root via doas; `-h`/`-v` without root |
| `gstore` | `-n [UID]` `-b UID OUTFILE` `-x UID` `-s` (stub) `-V` `-h` `-v` | `-n` needs root (direct, or `doas gstore -n` for yourself) |

- **gpkg:** modifiers `-O`/`-S`/`-V` are applied in a pre-pass, so their
  position doesn't matter (`-lO` == `-Ol`). It finds its user via
  `/proc/self/loginuid`, falling back to `getuid()`.
- **syspkg, `-n`:** creates the tree plus `/glacier/sys`.
- **syspkg as non-root:** `-n`/`-l`/`-x` are refused after argument
  parsing with `Cannot open system index: Permission denied`.
- **gstore, `-n` without a UID, under doas:** acts for the invoking user
  (loginuid, then `DOAS_USER`, then getuid). The doas rule
  `permit nopass :wheel as root cmd /path/gstore args -n` is the
  boundary that stops a user naming another UID.
- **Shared code:** `lib/gpkg_common.lua` (`M.install`, `M.remove`,
  verification, replacement, backups).
- **The `-x` summary** comes from the live index; a name that isn't
  installed fails before the prompt.
- **Hardened loading** (syspkg and gstore): `#!/usr/bin/luajit -E`,
  `package.path` and every `ffi.load` pinned to their own `realpath`, and
  when euid is 0, every ancestor directory and every loaded `.lua`/`.so`
  must be root-owned and not group/world-writable. The preamble is
  DUPLICATED in both files on purpose: change one, change both.
- **gpkg** uses the older CWD-relative convention (`require("lib/...")`,
  `ffi.load("./lib/glacier/...")`): run it from its own directory.
- **Declarations:** `lib/glacier_cdef.lua` is the single source of truth
  for every `ffi.cdef`. `tests/glacier_ffi.lua` is only a wrapper around
  it. Declaration ORDER matters: a declaration using `gl_context_t` must
  come after its typedef.

## Build tooling

- **gbuild** (bash, the only copy is `glacier-ng/gbuild`):
  - `-n -V -g -c -d -m -D -C -b -i -o -p -j -F -t -x -u -r -s`;
    `-v`/`--version` prints the version, `--verbose` turns on verbose
    output.
  - Autodetects the build system, with autotools checked BEFORE a bare
    Makefile.
  - Writes `NAME-VER.gpkg` plus a `.sha256` sidecar, using `gsha256sum`
    from `$GSHA256SUM_BIN`, else `<script dir>/build/bin`, else PATH.
  - **There is NO `-P` system-profile support, and gworld has no
    `SYS_PROFILE` field** (ROADMAP 4.1). The target libc is decided by
    where gbuild runs: Alpine for musl.
- **gworld** (Lua): batch orchestrator, reads `recipes/*.recipe`, sorts
  topologically by `DEPS`, runs gbuild per package.
- **`~/Projects/everest`:** the gbuild command for each package lives in
  `scripts/container.sh`; `everest.sh` runs it inside an Alpine image
  (`Dockerfile.build`).
- **Kernel version:** linux `6.12.112`; plain `v6.12` doesn't build with
  GCC 15.

## libglacier-ng build

- **Separate `.so` per module** (`libglacier_log`, `_pkg`, `_istoreutils`,
  `_transaction`, `_dag`, `_verify`). This split is deliberate: a
  third-party program links only what it needs.
- **The combined `libglacier-ng.so`** (`make libglacier-ng`) exists only
  for the FFI tests.
- **`libglacier_verify.so`** is private: no public header.
- **Warnings:** `config.mk` builds with `-Werror`. `-Wno-format-truncation`
  is set because that warning is noise; `-Wstringop-truncation` stays
  fatal.
- **Makefile quirk:** `make log` builds only `log.o`; the shared object
  comes from `make log_test` (or plain `make`).

## Testing

Run everything in throwaway containers. Several suites write to `/usr`,
`/etc` and `/glacier`.

**FFI suite** (`glacier-ng/tests/*.lua`; 8 files, all passing):
```sh
docker run --rm --cap-add SYS_ADMIN -v ~/Projects/glacier-ng:/src:ro \
  -v ~/Projects/libglacier-ng:/libsrc:ro alpine:latest sh -c '
  apk add --no-cache build-base luajit luajit-dev libarchive-dev libconfig-dev linux-headers tar >/dev/null
  cp -a /libsrc /build && cd /build && make clean && make && make libglacier-ng
  cp -a /src /g && cd /g/tests && GLACIER_TEST_CONFIRM=yes \
    LIBGLACIER_SO=/build/build/lib/shared/libglacier-ng.so luajit run_all.lua'
```
GNU `tar` is required: the hostile-archive fixtures use `tar --append`.

**Shell suites** (`glacier-ng/tests/*.sh`):

| Suite | Image |
|---|---|
| `syspkg_alpine.sh`, `overwrite_alpine.sh` | `alpine:latest` with `/src`, `/libsrc` and `/t.sh` mounted (see each header); `overwrite_alpine.sh` runs both as-is and with `--tmpfs /glacier:exec` |
| `verify_alpine.sh`, `user_layout_alpine.sh`, `upgrade_alpine.sh` | `glacier-alpine_root` |

Build the `glacier-alpine_root` image with:
```sh
cd ~/Projects/glacier-ng && docker build -f Dockerfile.alpine_root \
  --build-context libglacier=../libglacier-ng -t glacier-alpine_root .
docker run --rm -v "$PWD/tests/NAME.sh:/t.sh:ro" glacier-alpine_root sh /t.sh
```

**Interactive images:** `./run_docker.sh Dockerfile.{alpine,alpine_root,arch,debian}`.
`run_docker.sh` passes the `libglacier` build context. `alpine`, `arch`
and `debian` default to user `glacier` (uid 1000, in wheel); `alpine_root`
defaults to root. All of them have:
- root-owned `/app`;
- no `/glacier` until `gstore -n` / `syspkg -n`;
- the doas rule plus `permit nopass glacier`;
- `glacier.conf` installed.

**Boot test:** `cd ~/Projects/everest && ./everest.sh boot` boots QEMU
(KVM). Press Enter for a root shell; `poweroff -f` to leave.

## Bug history: so these don't get reintroduced

All fixed in the current tree.

1. `gl_abort_transaction` never released the lock (wrong variable). Now
   `open(O_CREAT)` + `flock()`, and lock files are never unlinked.
2. No `EXDEV` check before renaming. Commit `stat()`s both sides and
   returns `GL_TXN_ERR_XDEV`.
3. Segfault on install: the `transaction` target didn't link
   `istoreutils`/`log`.
4. Stale links post-commit: `gl_link_pkg` ran during staged installs,
   embedding stage paths. It now runs only for LIVE contexts, and links
   regenerate post-commit.
5. Dangling links were never cleaned on removal → `prune_stale_links`.
6. `GL_TXN_ERR_RENAME` after the first commit: a leftover `old/`. It's now
   wiped first.
7. `-Wstringop-truncation`: `strncpy(..., PATH_MAX - 1)` could leave
   strings unterminated. Use `snprintf(dst, sizeof dst, "%s", ...)`.
8. gbuild detected a bare `Makefile` before `configure`. The fix had been
   lost from the tree and was re-applied 2026-10-04.
9. The index had no version field; it's now `repo::pkgname::version`.
10. [The code this was about is NOT in the tree: ROADMAP 4.1.] A recipe
    field silently never parsed and fell back to a plausible default.
    Remember the class.
11. `gsha256sum.c` included `sha256.c` instead of `.h`.
12. gbuild told users to run `gpkg -f` (a stub) instead of `gpkg -l`.
13. `gpkg -f`'s stub referenced an undefined `uid`.
14. Path traversal from manifests/archives: `PKG_NAME`/`PKG_REPO` must
    pass `gl_valid_pkg_ident`, and `..`/absolute entries refuse the
    package. Finding 1 (self-declared `base`) stays open until signing.
15. Stage seeding and `gl_rmdir_recursive` skipped ALL dot-entries:
    dotfiles vanished, and `old/` couldn't be removed (bug 6 again).
16. Per-user transactions could never work for a real user (shared
    writable parents needed). Fixed by the per-uid layout. Found alongside:
    - `gstore -n` never created parent directories;
    - the stage was wiped before the lock was taken;
    - cross-filesystem seeding was reported as `MKDIR`;
    - `gl_restore_istore` doubled the uid and accepted `..` entries;
    - gstore loaded the combined `.so` via `LIBGLACIER_SO` from the
      environment.
17. Package symlinks were silently dropped at extraction, and only regular
    files were linked, so musl's loader was never installed. Alpine-based
    tests passed only because Alpine provides its own loader.
18. `add_to_archive` (`gstore -b`, backups) silently dropped symlinks.
19. gpkg acted on `-l` as soon as it parsed it, so `-lO` ignored its `O`.
20. `tests/glacier_ffi.lua` duplicated the cdefs and had drifted (it was
    missing `GL_INSTALL_ERR_SCOPE`). It's now a wrapper.
21. Link conflicts were re-reported on every system commit, for every
    package (it now uses the `.glacier-fresh` marker).
22. A failed extraction left a half-made version directory, so in a live
    context every retry reported `ALREADY_INSTALLED`.
23. Backups written in the same second were pruned in the wrong order
    (whole-second mtimes).

## Conventions

- **C style:** tabs, but the codebase is inconsistent; match the
  surrounding function.
- **Paths:** `PATH_MAX` stack buffers plus `snprintf` for path
  construction.
- **Lua requires:** gpkg uses bare CWD-relative requires.
  syspkg and gstore pin their paths (see CLIs).
- **Lua numbers in C varargs:** a Lua number reaches C varargs as a
  double, so never pass one to `lg_printf("%d")`. Use `%s` with
  `tostring()`, or `ffi.new("int", n)`.
- **Shell commands from Lua:** always `shell_quote` interpolated paths
  (syspkg runs as root).
- **Verify by running.** Several real bugs here were found only by
  executing something.