Files
libglacier-ng/CLAUDE.md
T
2026-10-14 00:50:56 -04:00

1048 lines
58 KiB
Markdown

# CLAUDE.md
Context for Claude Code sessions on libglacier-ng: the package manager
for Everest Linux. Kept as a handoff, so a fresh session starts with full
continuity instead of re-deriving everything. Last full rewrite:
2026-10-04; brought fully up to date at the end of the 2026-10-07
session. **Update it as you go.**
## Start here
1. `ROADMAP.txt`: what's left, in priority order, plus the decisions that
are still open (section 6). Pick work from it.
2. This file: how things work, what the user wants, how to test.
3. The design docs, which are authoritative for their topics and not
repeated here:
- `SECURITY_DESIGN.txt`: integrity verification, scope separation,
refusal matrix, syspkg's privilege model.
- `PACKAGE_SCOPES.txt`: layout, link routing, `/etc` config handling.
- `IMAGE_DESIGN.txt`: system images (bootstrap OUTPUT). Sections 3
(sysroot) and 4 (`syspkg --root`) are IMPLEMENTED (milestones 1-2);
the rest is still a proposal. Open decisions are in section 9.
- `GLACIER_BOOTSTRAP.txt`: `glacier-bootstrap`. Local mode is
IMPLEMENTED (ROADMAP 1.3; section 10 says how it differs from the
plan); remote mode and `--pin` are still a plan. Open: 9a, 9f.
- `REPO_DESIGN.txt`: remote repos + signing (ROADMAP 3.1). Sections
1 and 10 = the user's decisions (TLS library still open); steps
1-4 of section 9 DONE (see "3.1 status"); sections 7, 5 and 6
describe steps 2, 3 and 4 as implemented.
- `SECURITY_REVIEW_2026-10-06.md`: the full audit (findings, fixes,
what was removed, what's still open).
- `SECURITY_REVIEW_2026-07-16.md`: older review findings.
Nothing from the 2026-10-04 through 2026-10-07 sessions is
committed. The user said "do not commit yet" (2026-10-05), and on
2026-10-06 that their git server is down: they will commit themselves
once it's back. libglacier-ng is on branch
`dev` with a large uncommitted diff (`src/`, the docs, `build/`). Commit
only when the user asks; when they do, propose a few logical commits.
2026-10-05 session: system-scope permissions fix (bug 24), ROADMAP 2.3
dependencies (`src/deps.c`), index corruption on abort (bug 25), ROADMAP
1.1 sysroot, live removal deleting foreign files (bug 26), ROADMAP 1.2
(`syspkg -R`, `--strict`) + 2.6 (file collisions), gpkg ignoring unknown
flags (bug 27).
2026-10-06 session: ROADMAP 1.3 `glacier-bootstrap` local mode, and
Everest's initramfs switched to it (boots in QEMU). Tidy-up (gbuild's
hint removed). ROADMAP 1.4 DONE (packages for glacier itself; details in
ROADMAP 1.4 and section 7): full regression green, new
`tests/installed_alpine.sh` for the installed layout, which found bug 29
(fixed, mutation-checked, Everest rebuilt and booted with the fix).
The installer (1.5) is DEFERRED by the user: manual installation is
one of Everest's main features. Then ROADMAP 3.1 was designed with the
user (REPO_DESIGN.txt, every decision taken; only the TLS library for
curl in base is open) and steps 1, 2 and (that evening, after the
laptop died mid-rebuild) 3 of its section 9 were done.
2026-10-07 session: ROADMAP 3.1 step 4 (`-f`, `-u`, dependency
closure, the cache; see "3.1 status"). Help text for -f/-u/-O
shortened at the user's request. Refresh output (user's wording): `[i] Syncing
all repositories...`, then `[✓] All repositories synced successfully.`
if every repo succeeded; the per-repo success/up-to-date lines only
with `-V`; errors always (repo_alpine.sh checks both ways).
Then (same day) glacier-bootstrap configures repositories (user: it must
leave a semi-functional system, pacstrap-like): see "Bootstrap
repositories" below.
2026-10-09: help text. `gsign`, `grepo`, `gsha256sum` (C, `getopt_long`,
static `help()` in each tool's own file), `glacier-bootstrap` and `gworld`
now have `-h/--help` and `-v/--version` (`v4.0.0`) in gpkg's format
(`%-25s` `{-x --long} ARG` lines, GPL footer), with long forms for every
flag. `-h` goes to stdout, exit 0; usage errors stay one usage block on
stderr, exit 1. repo_alpine.sh checks the help of grepo/gsign/gsha256sum.
Backups: `glacier-bootstrap.old2`, `gworld.old`, `tests/repo_alpine.sh.old5`.
### Bootstrap repositories (2026-10-07)
The user chose: the config comes from a NEW base package
(`everest-repos` 1.0.0: `etc/glacier/repos.conf` + `keys/everest-2026.pub`,
source `~/Projects/everest/pkgsrc/everest-repos`; everest-bootscripts
1.3.0 no longer ships `etc/glacier`, its old copy is
`pkgsrc/everest-bootscripts/glacier.old`); bootstrap runs `syspkg -R T
-y` after installing and a failed fetch is an error (exit 7, no
"Bootstrapped"); under `-R`, `-y`/`-f`/`-u` use T's OWN keyring
(`keyring(sysroot)` in gpkg_common.lua; `require_trusted` also refuses a
file whose realpath leaves T). `-R -l` signatures stay the host's
(`HOST_KEYRING`). No url in T: `[!] No remote repositories are
configured in T.`, exit 0 (Everest today: base isn't published).
Copying the host's repos.conf (pacstrap's mirrorlist) left out for now.
The user said everything a template lists for base also goes in the
base repo (the published repo = the template's base packages); no url
yet. Everest rebuilt (bootscripts 1.3.0, everest-repos, glacier-ng 4.0.0
in place), initramfs remade, booted: `/etc/glacier` from everest-repos,
12 packages.
### Audit and cleanup (2026-10-06, late)
At the user's request, before 3.1 step 4: a full security audit
(`SECURITY_REVIEW_2026-10-06.md`, bugs 30-35 below), unused code removed,
and junk cleaned up. The user's choices: delete only the `.old` backups
and stray binaries (every prototype/note file stays: libglacier-ng's
`src/{deb,gpkg,templ,report,test_*}.c`, `src/html`, `pkg/`, glacier-ng's
`todo`, `tree/`, `test-*`, `pkgs/`...); libglacier-ng's build outputs are
untracked (`.gitignore`: `/build/`; the deletions show in the next
commit); the `.old` backup rule STAYS (see "Backups" below for current ones). New tests:
`tests/audit_alpine.sh` (26 checks), `tests/test_abort_unlocked.lua`.
Full regression green; Everest rebuilt and booted. Everest's own `.old`
files were not touched (not one of the two repos).
### 3.1 status
Step 4 DONE (2026-10-07; REPO_DESIGN section 6 "As implemented"). The
user chose: the FIRST repo (repos.conf order) serving a name wins, at
its highest version; a dependency installed too old is upgraded in the
same transaction (keeps its auto/explicit mark); cached downloads are
deleted after the commit that installs them.
- libglacier-ng: `gl_scope_cache_dir(ctx, repo, out)` (transaction.c;
`<base>/cache/REPO`, checked like `repos` via the shared static
`checked_subdir`); `log.c` help lists `-f`/`-u`.
- glacier-ng `gpkg_common.lua`: `M.fetch(names, opts)`,
`M.upgrade(names, opts)`; `load_indexes` (refresh, then read the
accepted copies), `pick`, `close_deps`, `download` (SIZE-capped,
hashed), `matches_index` (manifest vs index entry), `install_plan`.
`M.install` is now `verify_all` + `install_copies(copies, original,
opts, meta)`, shared with -f/-u (`meta`: auto, may_replace, dep,
cache). `verify_and_copy(path, vouched)` takes the index's hash in
place of a sidecar. syspkg `-f`/`-u` modes; gpkg `-f`/`-u` (no
longer "not implemented").
- New suite `tests/fetch_alpine.sh` (84 checks; 16 Lua mutants + 1 C,
all caught except the download-level hash check, equivalent: kept as
defence in depth). The `gpkg -u "%s%n"` format check moved there from
audit_alpine.sh. Full regression green.
- Everest's libglacier-ng/glacier-ng 4.0.0 were REBUILT in place
2026-10-07 with -f/-u/-y (user: keep 4.0.0, no bump) and the
initramfs remade; in QEMU `gpkg -f hi` as a wheel user now says
`[x] No remote repositories are configured.` (shipped repos.conf has
no url; no curl in base). `container.sh initramfs` now `mkdir -p`s
`$G/lib/glacier` (gone from the tree). Not done: no published repo;
TLS library still open. Next: step 5 (`glacier-bootstrap -r`),
then 6 (user repos).
Step 3 DONE (2026-10-06, evening; REPO_DESIGN section 5 "As
implemented"). The user chose: a refresh-only flag `-y`, the date-form
SERIAL (YYYYMMDDNN), grepo in C.
- libglacier-ng: `src/grepo.c` (`grepo -s KEY.sec -r REPO [-n SERIAL]
[-e DAYS] DIR`, Makefile target in `all`, installed to bin, built from
source like gsign); `pkg.c`: `gl_index_parse(text, len, repo, out)` /
`gl_index_free` (strict; takes the signed BYTES; refuses NUL and '@'),
`gl_conf_repos(path, out, len)` ("NAME\tURL\n" lines), repos.conf
validation shared in `read_repos_conf`; `transaction.c`:
`gl_scope_state_dir(ctx, "repos", out)` (creates/checks
`<base>/repos`). `log.c` help lists `-y`.
- glacier-ng: `M.refresh(opts)` in `gpkg_common.lua` (fetch file:// or
curl https, verify with a key listed for THAT repo, parse, rollback /
same-serial / expiry checks, rename into `<base>/repos`, under the
scope lock); `syspkg -y` (needs root and `-n`; `-R` works), `gpkg -y`
(system repos.conf + keyring until step 6). Dockerfiles copy grepo to
/usr/local/bin. New suite `tests/repo_alpine.sh` (103 checks; 18
mutants, all caught). Full regression green. Everest's libglacier-ng
and glacier-ng packages rebuilt with it.
- Next: step 4 (`-f` with dependency closure, then `-u`; the cache in
`<base>/cache`, section 5's Fetch).
Step 2 DONE (2026-10-06, second part of the session; REPO_DESIGN
section 7 "As implemented" has the details):
- libglacier-ng `src/pkg.c`: `gl_conf_repo_keys(path, repo, out, len)`,
a strict repos.conf reader; new statuses `GL_CONF_ERR_INVALID = 4`,
`GL_CONF_ERR_SPACE = 5`.
- glacier-ng `lib/gpkg_common.lua`: `verify_all(paths, opts)` refuses a
base package in system scope without a trusted signature
(`signed_for`, `trusted_keys`, `untrusted`); `verify_and_copy` returns
the sidecar's bytes. glacier-bootstrap copies the sidecar's BYTES and
the `.sig`. `gbuild -k`/`--key`, `tests/mkpkg -k`.
- Every `Dockerfile.*` installs gsign + gsha256sum in /usr/local/bin and
a TEST keyring (`glacier-test`, secret `/root/glacier-test.sec`).
`syspkg_alpine.sh` / `overwrite_alpine.sh` make their own key.
- New suite `tests/keyring_alpine.sh` (68 checks; 17 mutants: 10 Lua/
shell, 7 C, all caught). Full regression green.
- Everest: `./everest.sh keygen` made `everest-2026`; the SECRET key is
`~/.config/everest/keys/everest-2026.sec` (user's choice: outside the
project), mounted read-only for `build`, which refuses without it.
Every package is built `gbuild -k`. everest-bootscripts 1.2.0 ships
`etc/glacier/repos.conf` (base: everest-2026) and
`etc/glacier/keys/everest-2026.pub`; `container.sh initramfs`
installs the same into the build container's `/etc/glacier`.
The signed `build all` was cut off after busybox (laptop died); the
other 9 were rebuilt `-k` the evening of 2026-10-06, `initramfs` ran
through glacier-bootstrap (signature checks passed for all 11), and
Everest booted in QEMU: repos.conf + key in `/etc/glacier`, index
lists all 11, syspkg/gpkg run as root and as a fresh user.
- Base KEEP_OLD backups can't be restored with `-lO` (no signature):
the user accepted that until the restore interface (REPO_DESIGN 10f).
Step 1 DONE (2026-10-06):
- `src/monocypher/`: Monocypher 4.0.3, vendored UNMODIFIED (provenance
and sha512 in its README; GitHub and monocypher.org copies matched).
Compiles clean under the project's `-Werror` flags.
- `src/sign.{c,h}`: signify formats (strict parse, canonical base64),
`gl_sign_read_pubkey`/`_read_sig`/`_check_seckey`/`_verify`/
`_verify_file`/`_file`/`_generate`, statuses `GL_SIGN_OK`..
`GL_SIGN_ERR_WRITE = 9`. Built into `libglacier_verify.so` (NOT into
the combined `libglacier-ng.so`).
- `src/gsign.c` (`-G`/`-S`/`-V`, lg_printf messages naming the file at
fault), Makefile targets `gsign` (in `all`, installed to bin like
gsha256sum).
- `glacier-ng/tests/sign_alpine.sh` (alpine:latest, mounts /libsrc):
round trips, signify interop both ways (Alpine `signify`), encrypted
and corrupted secret keys, malformed key/sig files, 64 MiB limit,
O_NOFOLLOW, key modes, a direct FFI call of `gl_sign_generate`.
Passes in full. (`luajit -e CODE ARG` runs ARG as a script, so the
FFI snippet is written to `/gen.lua` first.)
- Mutation checks: 12 mutants, all caught except m9 (padding position,
`j < 2`), which is EQUIVALENT: kept, it prevents `3 - pad`
underflowing. m5 (fchmod), m6 (final newline) and m10 (O_EXCL) needed
the umask-077, final-char and FFI checks respectively.
- `glacier-ng/lib/glacier_cdef.lua`: `gl_sign_*` declarations and
`M.sign_name()`; checked against the real `.so` (struct sizes 40/72,
every call on signify-made files). Full regression green after it.
Also open, for the user (raised 2026-10-06, not answered yet):
- libgcc: luajit is built `-static-libgcc` (Everest has no libgcc_s).
Claude's call; the alternative is a GCC runtime package in base
(ROADMAP section 6).
- Leftover `~/Projects/everest/pkgs/everest-bootscripts-1.0.0.gpkg`
(+ sidecar), superseded by 1.1.0. Not deleted: ask first.
### Everest's glacier packages (1.4)
- `~/Projects/everest/scripts/container.sh` builds zlib, libarchive
(gzip/tar only), libconfig, luajit (pinned commit,
`LDFLAGS=-static-libgcc`), libglacier-ng, glacier-ng (all base) and
doas (extra, OpenDoas, no PAM).
- glacier-ng layout: `usr/lib/glacier-ng/{syspkg,gpkg,gstore,lib/*.lua}`,
`lib/glacier -> /usr/lib/glacier` (absolute: relative links resolve
inside the store), `usr/bin/X -> ../lib/glacier-ng/X`,
`etc/glacier/glacier.conf`.
- everest-bootscripts 1.2.0 (1.1.0 before signing) ships `/etc/doas.conf` (0400): `permit
:wheel` plus nopass `gstore args -n` for BOTH `cmd gstore` and `cmd
/usr/bin/gstore` (OpenDoas matches `cmd` against what was typed).
- `templates/minimal.template` and `PACKAGES` in everest.sh list all 12
packages (everest-repos added 2026-10-07). `Dockerfile.build` has autoconf, automake, libtool, cmake,
pkgconf.
## The three projects
| Path | What | Git? |
|---|---|---|
| `~/Projects/libglacier-ng` | C library, design docs, this file | yes (`dev`) |
| `~/Projects/glacier-ng` | Frontend: `gpkg`, `gstore`, `syspkg`, `gbuild`, `gworld`, `lib/*.lua`, `tests/`, Dockerfiles, `pkgs/` | **no**: back files up before editing (`file.old`; if that exists, `.old2`, `.old3`, ...) |
| `~/Projects/everest` | The distro build: `everest.sh build\|initramfs\|boot`, packages, bootable QEMU image | **no**: same backup rule (but never inside `pkgsrc/*/etc/`, see Boot test) |
Nothing prebuilt is kept in either repo (the host-built `.so`s in
`glacier-ng/lib/glacier/` were removed 2026-10-06). Every image and test
builds libglacier-ng from source; the Dockerfiles create
`/app/lib/glacier/` with the musl `.so`s.
## What this project is
A from-scratch, security-focused Linux distribution (Everest) with its own
package manager (libglacier-ng + `gpkg`/`gstore`/`syspkg`) and build tooling
(`gbuild`/`gworld`). It's a C library, LuaJIT+FFI CLI tools and bash build
scripts. The user writes it; Claude implements and designs alongside them.
## How the user wants things done
These are preferences stated in sessions. Follow them.
- **Output format:** every message uses glacier's `lg_printf` levels:
`[x]` is error (2), `[!]` warning (1), `[i]` info (0), `[✓]` success (3).
Messages are capitalized. Exact strings the user chose include
`Cannot open system index: Permission denied` (no period) and
`Nothing to do.` (success level).
- The pre-trust-check `die()` in syspkg/gstore reproduces the `[x]`
format byte for byte, because it can't load `libglacier_log.so`
before that file has been verified.
- **Never say "Completed" after an error.** If the post-commit link pass
refused any path (`ctx->link_refusals`, counted only for packages
installed by that transaction), gpkg/syspkg end with
`[x] N paths were not linked.` (`1 path was not linked.`) and exit 1.
The transaction itself stays committed (2026-10-05).
- **No hint messages.** Don't print lines that point users to other flags
or commands (for example "use -lO to ..."). The user called these
clutter. A refusal states the fact, in one line. The user also removed a
`Run doas gstore -n ...` follow-up hint from gstore themselves.
- **What goes where.** ALL design docs (and ROADMAP, this file) live in
libglacier-ng, frontend ones included. Other than docs, libglacier-ng
holds only backend material (C sources, the build). Frontend code,
tests, Dockerfiles and everything the bootstrap runs on live in
glacier-ng. Clarified 2026-10-06.
- **Ask before settling a design choice that is the user's to make**, and
give a recommendation. They have answered every such question promptly.
Don't silently pick a layout or semantics.
- **Verify by running, in Docker or QEMU, never by inspection alone.** For
any security fix, also prove the test catches the bug: remove the
defence in a throwaway copy and confirm the test fails ("mutation
check"). Several defences here are verified that way.
- Use doas, not sudo. syspkg is never setuid.
- **Help text is terse** (2026-10-07): one short phrase per flag, no
mechanism details (`-f` "Install packages from a remote repository",
`-u` "Update system or individual packages", `-O` "Allow Glacier to
overwrite existing files").
## Decisions the user has made (don't relitigate)
- **Per-uid user layout** (`/glacier/usr/<uid>/...`, root-owned parent).
Both root and the user can create it: root runs `gstore -n UID`; a user
runs `doas gstore -n` through a doas rule pinned to `args -n`
(option "B").
- **`syspkg -n` creates the top-level tree itself** (GLACIER_BOOTSTRAP 9d).
- **`/etc` config files: option 1.** Copy on install, `.new` on conflict
(GLACIER_BOOTSTRAP 9c).
- **`-O` with `-l` = replace.** `-O` displaces non-glacier files, and
`-lO` also replaces an installed *different version* (upgrade or
downgrade from a local file). Plain `-l` with another version installed
refuses in one line.
- **`KEEP_OLD`** in `/etc/glacier/glacier.conf`: the number of backups of
replaced versions to keep per package (0 = none).
- It has **no default.** It ships as `KEEP_OLD = 1`.
- If it's missing or invalid, an operation that would replace a version
errors. Only then: other operations don't need it.
- "Backed up" means the old package's files are compressed into a
`.gpkg`. A restore interface comes later.
- **`gpkg -u` / `syspkg -u` upgrade from a REMOTE repository**, and
`gpkg -f` is `-l` but fetched by name from a remote repository. Done
2026-10-07 (3.1 step 4). Never against local files.
- **-f/-u details (2026-10-07):** first repo by priority wins (then its
highest version); a too-old installed dependency is upgraded in the
same transaction; cached downloads are deleted after commit.
- **Dependencies (ROADMAP 2.3, decided 2026-10-05):** see the spec in
ROADMAP 2.3. In short: minimum versions, conflicts both ways by name,
no override flag, report every problem, missing deps refuse until 3.1
brings automatic closure, auto/explicit recorded in the index now.
- **sysroot (IMAGE_DESIGN section 3) accepted as proposed**, plus the
library refuses a non-"/" sysroot for `GL_SCOPE_USR`. Order of work:
2.3 first, then 1.1.
- **System scope is world-readable** (dirs 0755, index files 0644): every
user must be able to run what syspkg installs (2026-10-05). User scope
stays 0700, and other users' programs stay unreachable to non-root
users. Root CAN run them by full path, but other users' links are
deliberately NOT on root's `PATH` (a user could get code run as root).
- **Link refusals after a commit** end with `[x] N paths were not linked.`
and exit 1, never "Completed" (the user's wording rule: don't say
"completed" if there was an error).
- **`syspkg -R DIR`** (not `-r`), `--root DIR`, `--root=DIR`; `-R`
IMPLIES `--strict`; `--strict` alone is opt-in on the host.
- **File collisions (2.6) are always refused**, every scope, strict or
not (folded into 1.2 at the user's go-ahead).
- **PATH:** per-user links, then `/glacier/sys/links/bin`, then the system
(`/usr/bin` on Everest), in the Docker images AND Everest's
`/etc/profile`. Non-base system packages must be on `PATH`; base
programs must be runnable by every user.
- `everest-bootscripts` is the name of the base `/etc` package. Its source
lives in `~/Projects/everest/pkgsrc/`.
- **glacier-bootstrap (2026-10-06):** template = `NAME=` + `PKG=name ver
[sha256:hex]` lines, no `SYS_PROFILE` (unknown key until ROADMAP 3.3);
it runs the `syspkg` on PATH (doas unless root); a template dependency
must be listed at that version OR LATER (the 2.3 rule); Everest's
initramfs is built through it.
- **Installing Everest (2026-10-07):** possible from a live Everest
system OR any Linux host; the LIVE SYSTEM is the preferred, officially
supported way (so base/the live medium must eventually carry what a
manual install needs). The user does the first QEMU disk install
themselves: don't start it.
- **Everest supports glibc as well as musl** (2026-10-07), Gentoo/Void
style: the user picks the libc at install time, one per system.
Today everything is built for musl (Alpine); glibc needs ROADMAP
4.1's profiles. How packages are kept apart is still open (ROADMAP 6).
- **Signing (2026-10-06, REPO_DESIGN 10e/10f):** Everest's SECRET key
stays outside the project (`~/.config/everest/keys`, mounted
read-only); a base KEEP_OLD backup refusing under `-lO` (unsigned) is
accepted until the restore interface.
## Core architecture
**`gl_context_t`** (transaction.h) is the central abstraction. Fields, in
order: `uid`, `root_path`, `index_path`, `store_path`, `links_path`,
`mode` (`GL_CTX_LIVE`/`GL_CTX_STAGE`), `scope`
(`GL_SCOPE_USR`/`GL_SCOPE_SYS`), `lock_fd`, then:
| Field | Set by | Meaning |
|---|---|---|
| `overwrite` | caller (after init, which zeroes it) | `-O`: the next link pass may displace foreign files |
| `sysroot` | `*_at` init | install root, `""` = the running system |
| `link_refusals` | link pass; commit copies the relink's count back | paths refused for this transaction's packages |
| `strict` | caller (after init, which zeroes it) | refuse those before committing instead |
New fields go at the END, and `lib/glacier_cdef.lua` must mirror the
struct exactly (FFI has no compiler to catch a mismatch).
**Status codes worth knowing:** `GL_TXN_ERR_DEPS = 9` covers every
pre-commit refusal (dependency, conflict, file collision, strict link
refusal); the library has already printed why, so the CLIs print nothing
more. Live-context equivalents: `GL_INSTALL_ERR_DEPS = 8`,
`GL_REMOVE_ERR_DEPS = 4`. `GL_INSTALL_ERR_SCOPE = 7` is base-in-user-
scope.
**Install root / sysroot** (ROADMAP 1.1, done 2026-10-05):
- `gl_init_live_context_at` / `gl_init_stage_context_at` / `gl_init_sys_at`
take a sysroot; the old functions wrap them with `"/"`.
`gl_normalize_sysroot` (common.h) refuses relative paths and `.`/`..`
components. **User scope refuses any sysroot** other than `/`.
- The context's paths are **INSTALL paths** (`T/glacier/sys/store`), so
every filesystem operation uses them unchanged: a forgotten translation
writes into T, never the host. That's deliberately the reverse of
IMAGE_DESIGN's wording (runtime paths stored, `gl_at` on every
operation), with the same effect and a safer failure mode.
- Translate ONLY where a path's runtime meaning matters:
`gl_runtime_path(ctx, p)` for link targets, `is_glacier_symlink`
prefixes and `.displaced` keys; `gl_at(ctx, runtime, out)` to check a
link's target (prune). `/usr` and `/etc` destinations are
`ctx->sysroot` + path.
- **Trust check** (`gl_check_sysroot` in transaction.c, also inside
every `*_at` init and `gl_init_sys_at`; helper `gl_sysroot_trusted` in
common.h): the root and every ancestor must exist and be root-owned,
non-symlink directories, not group/world-writable. So `/tmp/x` is
refused. Messages: `Refusing install root R: P is a symlink.` / `is
missing.` / `is not root-owned, or is group/world-writable.`
- Test: `tests/sysroot_alpine.sh` (host `/usr`, `/glacier`, `/etc`
byte-identical; `chroot T` runs a linked binary; 8 mutations caught).
- Not translated, on purpose: `KEEP_OLD` is read from the HOST's
`/etc/glacier/glacier.conf` even under `-R` (the build host's policy);
user scope's dependency view of "the system" is always the host's.
**Layout.** Each scope owns ONE base directory holding
`{index,store,links,stage,old,lock,backup}`. Every create, rename and lock
a transaction does stays inside it.
| Scope | Base dir | Ownership |
|---|---|---|
| `GL_SCOPE_USR` | `/glacier/usr/<uid>/` | uid, mode 0700, inside a root-owned 0755 `/glacier/usr` |
| `GL_SCOPE_SYS` | `/glacier/sys/` | root, world-readable: dirs 0755, index files 0644 (`gl_scope_dir_mode`/`gl_scope_file_mode` in transaction.h), whatever the umask |
- The helpers `gl_usr_base`, `gl_usr_base_check` and the layout comment
are in `common.h`.
- `gl_init_stage_context` refuses a user base that isn't a real directory
(lstat) owned by its uid with no group/other bits, or whose
`/glacier/usr` is loosened. `gl_usr_base_status` exports that check, so
gpkg can say why before any prompt.
**Transaction lifecycle.** `gl_init_live_context` / `gl_init_stage_context`
→ `gl_commit_transaction` or `gl_abort_transaction`.
- Stage init takes the lock FIRST, then clears any leftover stage.
- Stage contexts hardlink-seed `index`/`store` from live, so removing a
live package inside a transaction needs no special casing. Seeding
copies dotfiles: only `.` and `..` are skipped.
- A stage on another filesystem fails at seeding with `GL_TXN_ERR_XDEV`.
- **Commit:** first `gl_check_txn` (deps, conflicts, collisions, and
strict link refusals if `ctx->strict`); any problem → nothing changes,
`GL_TXN_ERR_DEPS`. Then rename-based promotion (stage → live, previous
live → `old/`), then fsync the base. Then `gl_relink_store` regenerates
links from a *fresh live context* and rescans the WHOLE store (every
commit relinks every package, so link-time logic must be idempotent);
its `link_refusals` is copied back into the caller's context. `old/` is
then deleted. (The index is rebuilt inside the stage by each
install/remove, and promoted with it.)
- **Live contexts** (no transaction; tests and third-party callers):
`gl_install_pkg` runs `gl_check_plan` itself, and `gl_remove_pkg`
removes the package's links by pruning (`gl_relink_store`), like a
commit.
- **Index files** are named to the second and seeding hardlinks them:
never open a seeded file for writing, always write elsewhere and
`rename()` (bug 25).
**Link routing** (per package, in `gl_link_pkg`):
| Scope | Repo | Destination |
|---|---|---|
| USR | any | `<base>/links` |
| SYS | `base` | `/usr` |
| SYS | anything else | `/glacier/sys/links` |
- A package's leading `usr/` is stripped.
- In system scope, a package's top-level `etc/` is never linked: it's
COPIED into `/etc` (see `/etc` config handling below).
**Repos.** `base` (the minimal system; boot-critical by definition, a
repo-level rule), `extra`, `community`. `extra` and `community` behave
identically.
**Hidden files in the store** (dot-prefixed, so linking, index rebuild and
the store scanners skip them, while seeding carries them through
transactions):
| File | Purpose |
|---|---|
| `<ver>/.glacier-manifest` | the package's manifest, saved at install, used by `gl_backup_pkg` |
| `<ver>/.glacier-fresh` | one-shot "installed in this transaction"; link conflicts are only reported when present |
| `<ver>/.glacier-overwrite` | one-shot `-O` permission for this package's next link pass |
| `<ver>/.glacier-auto` | installed as a dependency, not by name (index field 4 = `auto`); set by `gl_set_pkg_auto`, nothing sets it until 3.1 |
| `<store>/.displaced/<runtime abs path>` | originals displaced by `-O`, restored when the link that replaced them is pruned; keyed by RUNTIME path (no sysroot prefix) |
| `<store>/.etc-state/<path>` | the last version of each `/etc` file a package offered |
**Configuration.** `/etc/glacier/glacier.conf` (libconfig syntax).
- It's read with `gl_conf_get_int(path, key, &out)` in `pkg.c`. Missing
file, missing key and wrong type are distinct statuses, and there are no
defaults.
- The shipped copy is `glacier-ng/etc/glacier/glacier.conf`. Every
Dockerfile installs it.
## Safety invariants: read before touching links/symlink code
`/usr` is NOT glacier-exclusive territory. Treat every link destination as
possibly shared:
- **Ownership test.** `is_glacier_symlink(path, store_prefix)` is the only
test for "did glacier create this". Never touch anything it returns
false for, except through `-O`'s displace/restore.
- **Overwriting.** `gl_link_pkg` won't overwrite a non-glacier path without
the `-O` marker. It never descends through *another package's* directory
symlink into that package's store directory.
- **Pruning.** `gl_relink_store` PRUNES stale links; it never wipes a links
destination. A link is stale only if its own store target is gone
(checked with `lstat`, not by following it): a package symlink may
legitimately dangle.
- **Stage links.** The stage links tree is never seeded.
- **Extraction never writes through a symlink.**
- Symlink entries are created only after all regular content.
- Any entry whose parent is a symlink is refused (`crosses_symlink`,
plus `O_NOFOLLOW`).
- Any entry with `..` or an absolute path refuses the whole package.
- A failed extraction removes its half-made version directory.
- **Recursive deletes.** `gl_rmdir_recursive` skips only `.`/`..`, and
walks by descriptor (`openat(O_NOFOLLOW)`, `unlinkat`): an entry
swapped for a symlink is unlinked, never followed (bug 31). Every
caller passes a glacier-only path; keep it that way.
- **Root reading a user's tree** (`add_entry_at`, `gstore -b`) is
fd-relative with `O_NOFOLLOW` too (bug 32).
- **Reserved names.** A package's top-level `files/.glacier-*` entries
are refused (`GL_MARKER_PREFIX`, common.h): those names are glacier's
own version-dir files, and every marker write uses `O_NOFOLLOW`
(bug 30). The marker names are defined ONCE, in common.h.
- **Manifests are untrusted**: `gl_gpm2gpkg` refuses `@include` and
dependency/conflict entries without a valid name (bug 33). The CLIs
read manifests ONLY through the library (`manifest_of` in
gpkg_common.lua), never by pattern matching (bug 34). `gl_delete_user`
lstat-checks its target first, because a symlinked base would otherwise
have its *target* emptied.
- **Wide wrappers.** Don't add convenience wrappers that silently touch a
broader filesystem scope than asked. This codebase has been bitten twice
by exactly that.
## Feature reference (current behaviour)
**Integrity verification** (`gpkg_common.lua`: `verify_and_copy`,
`verify_all`):
- Each package is copied into a private `mktemp -d` directory while being
hashed with `libglacier_verify.so`. The result is checked against
`FILE.sha256` (`sha256:<hex>`).
- Everything after that (summary, manifest read, `gl_install_pkg`) reads
ONLY the copy.
- A missing or malformed sidecar counts as a mismatch and is refused before
the prompt; one bad package refuses the whole batch. No bypass exists.
- This lives in the CLIs' shared module, not the library (as
SECURITY_DESIGN section 4 specifies), so a third-party caller of
`gl_install_pkg` is not verified (ROADMAP 3.2).
- The base→USR refusal (`GL_INSTALL_ERR_SCOPE = 7`), by contrast, *is* in
the library.
**`-O` / displace:**
- A displaced foreign file or symlink moves to `<store>/.displaced/`:
renamed, or on `EXDEV` copied (keeping mode, owner and mtime) and then
unlinked.
- Directories, special files and paths that already have a backup are
still refused.
- In user scope `-O` is effectively a no-op, because the links tree is
rebuilt from scratch on every commit.
**Version replacement** (`-lO`, `M.install` in `gpkg_common.lua`):
1. Installed entries are read from the live index's `current` file
(`repo::name::version::auto|explicit` lines; files before 2026-10-05
stop at the version, and every parser accepts both).
2. A same name+repo with a different version is a replacement. Without
`-O` it's refused; with `-O`, `keep_old()` reads `KEEP_OLD`, and the
summary shows `old -> new`.
3. In the transaction: `gl_backup_pkg(live, repo, name, X.gpkg.part)`, then
`gl_remove_pkg` from the stage, then install.
4. After the commit: rename `.part` → `<scope>/backup/<repo>/<name>/<name>-<ver>.gpkg`,
write its sidecar, and run `gl_prune_backups(dir, KEEP_OLD)` (newest by
nanosecond mtime).
- On any abort, `finish()` deletes the `.part`.
- A backup is a complete installable `.gpkg`, so `-lO backup.gpkg`
restores it.
**`/etc` config handling** (`install_config` / `prune_config` in
istoreutils.c), system scope only:
| Situation | Result |
|---|---|
| `/etc/F` missing, no record | install |
| missing, record exists | the admin deleted it: stays deleted |
| unchanged since glacier wrote it | updated in place |
| admin-modified, or never glacier's | never touched; a package change is offered ONCE as `F.new` |
| no package ships it any more | removed if unchanged, kept otherwise |
User-scope `etc/` is still linked (ROADMAP 2.9).
**Top-level tree.** `gl_create_fs_tree(root, verbose)` (exported) creates
the merged-/usr tree: `/bin` and `/sbin` → `usr/bin`, `/lib` → `usr/lib`,
`/usr/sbin` → `bin`, all RELATIVE links. It also creates `/usr/{bin,lib}`,
`/etc`, `/dev`, `/proc`, `/sys`, `/run`, `/tmp` and `/var/tmp` (1777),
`/var`, `/root` (0700) and `/home`.
- It creates only what's missing, of any type.
- `gl_init_sys` runs it before writing the index. The index marks `-n` as
complete, so a second `syspkg -n` is refused: `Cannot create
/glacier/sys: already exists`.
- It takes a root prefix; `gl_init_sys_at` / `syspkg -R DIR -n` use it.
**Dependencies and conflicts** (ROADMAP 2.3; `src/deps.c`, built into
`libglacier_istoreutils.so`; rules in `deps.h`):
- `gl_check_plan(live, adds, n, "repo::name" removes, n)` runs before the
prompt (`check_plan` in `gpkg_common.lua`). `gl_check_txn(stage)`
compares stage with live, and `gl_commit_transaction` calls it itself
(`GL_TXN_ERR_DEPS = 9`). A live context runs the plan check inside
`gl_install_pkg`/`gl_remove_pkg` (`GL_INSTALL_ERR_DEPS = 8`,
`GL_REMOVE_ERR_DEPS = 4`). So no library caller can skip it.
- It reads each version dir's `.glacier-manifest`; a version without one
has no deps. Only what the transaction changes is checked.
- `-x` of a name installed in several repos is only checked at commit
(its repo is chosen after the prompt).
- Claude's own calls, not reviewed by the user: conflicts only apply
within one scope; only what a transaction changes is checked.
**Strict link check** (`check_links` in deps.c; `ctx->strict`): system
scope only, for this transaction's packages, mirrors `gl_link_pkg`: a
foreign file/symlink at a file's destination (unless `-O` and no earlier
`.displaced` copy), a non-directory where a directory goes, or a
directory over another package's symlink → `[x] Refusing to overwrite
non-glacier path: P` / `Refusing to merge directory ...`. A foreign
symlink where a directory goes (`usr/sbin -> bin`) is followed, as the
link pass does.
**File collisions** (ROADMAP 2.6, `check_collisions` in deps.c): two
packages whose entries land on the same path (by `dest_of`, which mirrors
`gl_link_pkg`) are refused in every scope, before the prompt and at
commit, when either is in the transaction: `[x] extra::a and extra::b
both ship /glacier/sys/links/bin/x.` Sharing a directory is fine; a file
or symlink where the other has a directory is a collision.
**Signatures** (REPO_DESIGN section 7; `gpkg_common.lua`): in system
scope, a package whose LIBRARY-read manifest (`gl_read_pkg_manifest`,
never `read_manifest_fields`) says base needs `FILE.sha256.sig` over the
exact sidecar bytes verified, by the key with that keynum among those
`/etc/glacier/repos.conf` lists for base (`gl_conf_repo_keys`). Else
`[x] base::NAME is not signed by a key trusted for base.` before the
prompt. repos.conf and keys (and all ancestors, real paths too) must be
root-owned, not group/world-writable. Under `-R`, the HOST's keyring
for this check; `-y`/`-f`/`-u` under `-R` use T's (inside T only).
No repos.conf = nothing trusted. gpkg / other repos: sidecar only.
**Repository indexes** (REPO_DESIGN sections 4-5; `M.refresh` in
`gpkg_common.lua`): `syspkg -y` / `gpkg -y` fetch `URL/repo.idx` + `.sig`
for every repos.conf entry with a url, check the signature over the
fetched bytes with a key listed for that repo, `gl_index_parse` the
same bytes, refuse SERIAL below the accepted one (or equal with other
bytes) and an expired VALID_UNTIL, then rename into
`<base>/repos/REPO.idx` (+ `.sig`). Any failure keeps the accepted
copy; every repo is tried; exit 1 if any failed.
**`-f` / `-u`** (REPO_DESIGN section 6; `M.fetch`/`M.upgrade`):
refresh first (any failure refuses), pick each name from the first
repo serving it, add missing deps (auto) and upgrade too-old ones,
download into `<base>/cache/REPO` (hash and SIZE from the index), copy
privately with the index's hash, require the manifest to equal the
index entry (`[x] extra::x 1.0.0 does not match its index.`), then the
`-l` path (`install_copies`): summary with `(dependency)`, 2.3 checks,
prompt, staged transaction, `gl_set_pkg_auto`, backups, cache files
deleted after commit. A base package fetched this way needs no `.sig`
(the base index vouches). `-f` of an installed other version needs
`-O`; `-u` and dependency upgrades don't. gpkg never fetches from base.
**Package symlinks** are extracted verbatim and linked like files.
Hardlink, device and fifo entries are skipped. `add_to_archive` (used by
backups and `gstore -b`) archives symlinks.
## CLIs (glacier-ng)
| Tool | Flags | Runs as |
|---|---|---|
| `gpkg` | `-l FILE...` `-f NAME...` `-u [NAME...]` `-x NAME...` `-y` `-O` `-S` `-V` `-h` `-v` | the user; per-user scope only, NEVER constructs `GL_SCOPE_SYS` |
| `syspkg` | `-n` `-l FILE...` `-f NAME...` `-u [NAME...]` `-x NAME...` `-y` `-O` `-S` `-V` `-R DIR` `--strict` `-h` `-v` | root via doas; `-h`/`-v` without root |
| `gstore` | `-n [UID]` `-b UID OUTFILE` `-x UID` `-s` (refuses: not implemented) `-V` `-h` `-v` | `-n` needs root (direct, or `doas gstore -n` for yourself) |
| `glacier-bootstrap` | `-d PKGDIR TEMPLATE TARGET` `-n` `-t` `-V` `-h` `-v` | the user; borrows root only for `doas syspkg` / `doas chroot` |
- **gpkg:** modifiers `-O`/`-S`/`-V` are applied in a pre-pass, so their
position doesn't matter (`-lO` == `-Ol`). It finds its user via
`/proc/self/loginuid`, falling back to `getuid()`. Unknown flags are
refused (`[x] Unknown option: -R`); gpkg has no `-R`/`--strict`.
- **Endings (both CLIs):** `[✓] Completed with no errors.` (exit 0), or
after link refusals `[x] N paths were not linked.` (exit 1;
`report_done` in gpkg_common.lua). Refusals before the prompt exit 1
with only the `[x]` lines.
- **syspkg, `-n`:** creates the tree plus `/glacier/sys`.
- **syspkg, `-R DIR` / `--root DIR` / `--root=DIR`** (last in a bundle:
`-lR DIR`): operate on the system under DIR, through the `*_at`
contexts. Refused first by `gl_check_sysroot`: DIR and every ancestor
must be root-owned, non-symlink directories, not group/world-writable
(`/tmp/...` is refused). Implies `--strict`.
- **syspkg, `--strict`:** anything the link pass would refuse fails the
whole transaction before the prompt (and at commit, `ctx->strict`);
`-O` still displaces.
- **syspkg as non-root:** `-n`/`-l`/`-x` are refused after argument
parsing with `Cannot open system index: Permission denied`.
- **gstore, `-n` without a UID, under doas:** acts for the invoking user
(loginuid, then `DOAS_USER`, then getuid). The doas rule
`permit nopass :wheel as root cmd /path/gstore args -n` is the
boundary that stops a user naming another UID.
- **Shared code:** `lib/gpkg_common.lua` (`M.install`, `M.remove`,
verification, replacement, backups).
- **The `-x` summary** comes from the live index; a name that isn't
installed fails before the prompt.
- **Hardened loading** (syspkg and gstore): `#!/usr/bin/luajit -E`,
`package.path` and every `ffi.load` pinned to their own `realpath`, and
when euid is 0, every loaded `.lua`/`.so`, and every ancestor of the
script's realpath AND of each loaded file's realpath, must be
root-owned and not group/world-writable (bug 29). The preamble is
DUPLICATED in both files on purpose: change one, change both.
- **gpkg** (2026-10-06) loads `lib/` from its own realpath, like
glacier-bootstrap (no ownership checks; it's unprivileged), so an
installed `/usr/bin/gpkg` works from any directory.
- **glacier-bootstrap** (GLACIER_BOOTSTRAP section 10): parse →
verify (copies each `PKGDIR/NAME-VER.gpkg` into a private `mktemp -d`
while hashing; sidecar + pin; writes the copy's sidecar from that hash)
→ resolve (reads manifests of the COPIES via `gl_read_pkg_manifest`:
name/version match, deps listed at >= version, conflicts) → `syspkg -R
T -n` (skipped if T is initialised with nothing installed) → `syspkg -R
T -l COPIES` (one strict transaction; its prompt passes through) →
checks T's index lists every package (syspkg exits 0 when declined) →
`syspkg -R T -y` if T's repos.conf has a url → `-t` smoke test.
Never writes into T itself. Exit: 1 usage/target, 2 template,
3 verify, 4 resolve, 5 install, 6 smoke, 7 repositories. Loads its `lib/`
from its realpath, no root checks. No `-r` / `--pin` yet.
- **Declarations:** `lib/glacier_cdef.lua` is the single source of truth
for every `ffi.cdef`. `tests/glacier_ffi.lua` is only a wrapper around
it. Declaration ORDER matters: a declaration using `gl_context_t` must
come after its typedef.
## Build tooling
- **gbuild** (bash, the only copy is `glacier-ng/gbuild`):
- `-n -V -g -c -d -m -D -C -b -i -o -p -j -F -t -x -u -r -k -s`
(`-k KEY.sec`: sign the sidecar with gsign, `$GSIGN_BIN` / build/bin
/ PATH, key tried before building);
`-v`/`--version` prints the version, `--verbose` turns on verbose
output.
- Autodetects the build system, with autotools checked BEFORE a bare
Makefile.
- Writes `NAME-VER.gpkg` plus a `.sha256` sidecar, using `gsha256sum`
from `$GSHA256SUM_BIN`, else `<script dir>/build/bin`, else PATH.
- **There is NO `-P` system-profile support, and gworld has no
`SYS_PROFILE` field** (ROADMAP 4.1). The target libc is decided by
where gbuild runs: Alpine for musl.
- **gworld** (Lua): batch orchestrator, reads `recipes/*.recipe`, sorts
topologically by `DEPS`, runs gbuild per package.
- **`~/Projects/everest`:** the gbuild command for each package lives in
`scripts/container.sh`; `everest.sh` runs it inside an Alpine image
(`Dockerfile.build`).
- **Kernel version:** linux `6.12.112`; plain `v6.12` doesn't build with
GCC 15.
- **Everest's image IS built through glacier** (2026-10-06): `container.sh
initramfs` copies glacier-ng to a root-owned `/opt/glacier-ng` (with
the musl-built `.so`s; syspkg's trust checks need that) and runs
`glacier-bootstrap -t -d /everest/pkgs templates/minimal.template
/target` (root-owned: not under `/tmp`). T's links name RUNTIME paths,
so anything reading through them from the build host must resolve
inside T (vmlinuz is read with `chroot T cat`). Only `root` exists in
the shipped `/etc/passwd`.
- gbuild ends with the package's contents list. Its "To install with
gpkg: gpkg -l ..." hint was removed 2026-10-06 at the user's request.
## libglacier-ng build
- **Separate `.so` per module** (`libglacier_log`, `_pkg`, `_istoreutils`,
`_transaction`, `_verify`; `_dag` was removed 2026-10-06, unused). This split is deliberate: a
third-party program links only what it needs.
- **The combined `libglacier-ng.so`** (`make libglacier-ng`) exists only
for the FFI tests.
- **`libglacier_verify.so`** is private: no public header.
- **Tools** in `build/bin`: `gsha256sum`, `gsign`, `grepo` (gsign and
grepo compile the sources they need in, no rpath). Maintainer tools,
later "glacier-tools": Everest's libglacier-ng package drops
`usr/bin`.
- **Warnings:** `config.mk` builds with `-Werror`. `-Wno-format-truncation`
is set because that warning is noise; `-Wstringop-truncation` stays
fatal.
- **Makefile quirk:** `make log` builds only `log.o`; the shared object
comes from `make log_test` (or plain `make`).
## Testing
Run everything in throwaway containers. Several suites write to `/usr`,
`/etc` and `/glacier`.
**FFI suite** (`glacier-ng/tests/*.lua`; 11 files, all passing):
```sh
docker run --rm --cap-add SYS_ADMIN -v ~/Projects/glacier-ng:/src:ro \
-v ~/Projects/libglacier-ng:/libsrc:ro alpine:latest sh -c '
apk add --no-cache build-base luajit luajit-dev libarchive-dev libconfig-dev linux-headers tar >/dev/null
cp -a /libsrc /build && cd /build && make clean && make && make libglacier-ng
cp -a /src /g && cd /g/tests && GLACIER_TEST_CONFIRM=yes \
LIBGLACIER_SO=/build/build/lib/shared/libglacier-ng.so luajit run_all.lua'
```
GNU `tar` is required: the hostile-archive fixtures use `tar --append`.
**Shell suites** (`glacier-ng/tests/*.sh`):
| Suite | Image |
|---|---|
| `sign_alpine.sh` | `alpine:latest` with `/libsrc` and `/t.sh` mounted |
| `syspkg_alpine.sh`, `overwrite_alpine.sh` | `alpine:latest` with `/src`, `/libsrc` and `/t.sh` mounted (see each header); `overwrite_alpine.sh` runs both as-is and with `--tmpfs /glacier:exec` |
| `verify_alpine.sh`, `user_layout_alpine.sh`, `upgrade_alpine.sh`, `sys_perms_alpine.sh`, `deps_alpine.sh`, `sysroot_alpine.sh`, `root_alpine.sh`, `bootstrap_alpine.sh`, `installed_alpine.sh`, `keyring_alpine.sh`, `repo_alpine.sh`, `audit_alpine.sh`, `fetch_alpine.sh` | `glacier-alpine_root` |
Build the `glacier-alpine_root` image with:
```sh
cd ~/Projects/glacier-ng && docker build -f Dockerfile.alpine_root \
--build-context libglacier=../libglacier-ng -t glacier-alpine_root .
docker run --rm -v "$PWD/tests/NAME.sh:/t.sh:ro" glacier-alpine_root sh /t.sh
```
`glacier-ng/lib/glacier/` does not exist in the tree (the `.so`s are
built per image); syspkg_alpine/overwrite_alpine `mkdir -p` it
(2026-10-07). If Docker fails with `veth ... operation not supported`,
the host kernel was upgraded without a reboot (modules missing): use
`docker build --network host` and `docker run --network=host` (or
`--network none` where nothing is fetched) until the user reboots.
Mutating a Lua file syspkg loads: mount it elsewhere and `cp` it over
`/app/lib/...` inside the container (a bind mount is user-owned, so
syspkg's trust check refuses it).
**Demo packages:** `glacier-ng/tests/mkpkg [-o DIR] [-k KEY.sec]
[-d NAME:VER]... [-c NAME:VER]... NAME VERSION REPO` writes a one-script
`.gpkg` plus its sidecar (`/app/tests/mkpkg` in the images); a base one
needs `-k /root/glacier-test.sec` (the images' test key) for syspkg. The user plays with changes
in `./run_docker.sh Dockerfile.alpine_root` using it; for an image,
`mkdir -p /mnt/img && ./syspkg -R /mnt/img -n`.
**Mutation checks, how:** `cp -a` libglacier-ng (and glacier-ng, for Lua
changes) into the scratchpad, break the defence there, then
`docker build -f Dockerfile.alpine_root --build-context libglacier=COPY
-t glacier-mut .` from the (copied) glacier-ng and run the suite against
`glacier-mut`; delete the image after. Docker tags must be lowercase. A
mutation that doesn't compile under `-Werror` proves nothing: rewrite it
so it compiles. Watch for checks that pass for the wrong reason (e.g. a
refusal caused by a missing directory instead of the defence). In shell
suites, `sh -c '... $OUT ...'` sees an EMPTY `$OUT` (not exported), so a
negative check written that way passes vacuously; use a function
(`lacks` in bootstrap_alpine.sh). Found 2026-10-06 by a surviving
mutation. The host shell is zsh: an unquoted `$VAR` holding `-v a:b`
is NOT word-split, so a docker command built that way silently runs
without the mount (a mutant "survived" that way, 2026-10-06). To
mutate a Lua CLI without rebuilding, bind-mount the mutated file over
`/app/NAME` (`-v $PWD/mut:/app/glacier-bootstrap:ro`).
**Full regression** = the FFI suite + `syspkg_alpine.sh` +
`overwrite_alpine.sh` (both ways) + `sign_alpine.sh` + the thirteen `glacier-alpine_root`
suites, all in parallel; the two output-based suites (`syspkg_alpine`,
`user_layout_alpine`) have no PASS lines, so grep their output for
`Unknown|traceback|failed|GL_` instead (only `--bogus` is expected).
**Interactive images:** `./run_docker.sh Dockerfile.{alpine,alpine_root,arch,debian}`.
`run_docker.sh` passes the `libglacier` build context. `alpine`, `arch`
and `debian` default to user `glacier` (uid 1000, in wheel); `alpine_root`
defaults to root. All of them have:
- root-owned `/app`;
- no `/glacier` until `gstore -n` / `syspkg -n`;
- the doas rule plus `permit nopass glacier`;
- `/etc/profile.d/glacier.sh` putting `/glacier/usr/<uid>/links/bin` then
`/glacier/sys/links/bin` on `PATH` (and `lib` on `LD_LIBRARY_PATH`)
unconditionally, since `/glacier` only appears after login (fixed
2026-10-05: it had no system links and checked `-d` at login);
- `glacier.conf` installed.
**Boot test:** `cd ~/Projects/everest && ./everest.sh boot` boots QEMU
(KVM). Press Enter for a root shell; `poweroff -f` to leave. To test as
a non-root user, `adduser -D -H -h /tmp tester` at runtime, then
`su - tester -c ...` (nothing shipped has a user besides root).
Everest's `/etc/profile` sets `PATH` to per-user links, then
`/glacier/sys/links/bin`, then `/usr/bin` (2026-10-05). Backups of
everest-bootscripts files go NEXT TO `etc/`, never inside it: the package
copies `etc/` wholesale. After changing a package:
`./everest.sh build PKG && ./everest.sh initramfs`, then boot.
To drive the boot non-interactively, fork QEMU on a pty from Python
(`pty.fork()`), wait for `Please press Enter`, send lines and match the
`# `/`$ ` prompt (no `expect` on the host).
## Bug history: so these don't get reintroduced
All fixed in the current tree.
1. `gl_abort_transaction` never released the lock (wrong variable). Now
`open(O_CREAT)` + `flock()`, and lock files are never unlinked.
2. No `EXDEV` check before renaming. Commit `stat()`s both sides and
returns `GL_TXN_ERR_XDEV`.
3. Segfault on install: the `transaction` target didn't link
`istoreutils`/`log`.
4. Stale links post-commit: `gl_link_pkg` ran during staged installs,
embedding stage paths. It now runs only for LIVE contexts, and links
regenerate post-commit.
5. Dangling links were never cleaned on removal → `prune_stale_links`.
6. `GL_TXN_ERR_RENAME` after the first commit: a leftover `old/`. It's now
wiped first.
7. `-Wstringop-truncation`: `strncpy(..., PATH_MAX - 1)` could leave
strings unterminated. Use `snprintf(dst, sizeof dst, "%s", ...)`.
8. gbuild detected a bare `Makefile` before `configure`. The fix had been
lost from the tree and was re-applied 2026-10-04.
9. The index had no version field; it's now `repo::pkgname::version`.
10. [The code this was about is NOT in the tree: ROADMAP 4.1.] A recipe
field silently never parsed and fell back to a plausible default.
Remember the class.
11. `gsha256sum.c` included `sha256.c` instead of `.h`.
12. gbuild told users to run `gpkg -f` (a stub) instead of `gpkg -l`.
13. `gpkg -f`'s stub referenced an undefined `uid`.
14. Path traversal from manifests/archives: `PKG_NAME`/`PKG_REPO` must
pass `gl_valid_pkg_ident`, and `..`/absolute entries refuse the
package. Finding 1 (self-declared `base`) stays open until signing.
15. Stage seeding and `gl_rmdir_recursive` skipped ALL dot-entries:
dotfiles vanished, and `old/` couldn't be removed (bug 6 again).
16. Per-user transactions could never work for a real user (shared
writable parents needed). Fixed by the per-uid layout. Found alongside:
- `gstore -n` never created parent directories;
- the stage was wiped before the lock was taken;
- cross-filesystem seeding was reported as `MKDIR`;
- `gl_restore_istore` doubled the uid and accepted `..` entries;
- gstore loaded the combined `.so` via `LIBGLACIER_SO` from the
environment.
17. Package symlinks were silently dropped at extraction, and only regular
files were linked, so musl's loader was never installed. Alpine-based
tests passed only because Alpine provides its own loader.
18. `add_to_archive` (`gstore -b`, backups) silently dropped symlinks.
19. gpkg acted on `-l` as soon as it parsed it, so `-lO` ignored its `O`.
20. `tests/glacier_ffi.lua` duplicated the cdefs and had drifted (it was
missing `GL_INSTALL_ERR_SCOPE`). It's now a wrapper.
21. Link conflicts were re-reported on every system commit, for every
package (it now uses the `.glacier-fresh` marker).
22. A failed extraction left a half-made version directory, so in a live
context every retry reported `ALREADY_INSTALLED`.
23. Backups written in the same second were pruned in the wrong order
(whole-second mtimes).
24. System scope was root 0700 (index, store, links, every version dir),
so no non-root user could run anything syspkg installed, through
`/usr` or `/glacier/sys/links`. Found 2026-10-05. Also fixed: every
mode glacier creates was filtered by the caller's umask (`gl_mkdirp`,
seeding, extracted files), so syspkg under umask 077 made even a
0755 store private. `gl_mkdirp`/`gl_mkdir_exact` now chmod what they
create, and extracted entries get `archive mode & 07755` explicitly
(never group/world-writable). Test: `tests/sys_perms_alpine.sh`
(mutation-checked). No migration: a store made before the fix keeps
its 0700 dirs until reinstalled.
25. `gl_rebuild_index` `fopen(name, "w")`'d the new index file. Index
names have one-second resolution and seeding HARDLINKS live's index
files into the stage, so a staged install in the same second as the
last commit rewrote the LIVE index in place; an aborted transaction
then left live listing the stage's packages (and history files were
overwritten). It now writes to `tmpfile()` and only `rename()`s onto
the name. Found 2026-10-05 by `deps_alpine.sh`; regression test
`tests/test_index_abort.lua` (mutation-checked). Rule: never open a
seeded file for writing; replace it by rename.
26. Live-context removal (`gl_unlink_pkg`) `unlink()`ed the destination
of every regular file in the package WITHOUT `is_glacier_symlink`:
a foreign file the package had been refused to replace was deleted,
and the package's own symlink entries were never unlinked. The CLIs
always stage, so only library callers were exposed. Removed;
live removal now prunes like a commit (`gl_relink_store`). Found
2026-10-05; `tests/test_live_remove_foreign.lua` (fails on the old
code).
27. gpkg's `handle_flag` had no `else`: an unknown flag (`-R`) was
silently ignored and the NEXT word reported as the unknown option.
Now `[x] Unknown option: -R` (capitalized, as syspkg). 2026-10-05.
28. gbuild's `clone_source` printed its "Shallow clone failed" warning
(and the full clone's/checkout's output) to stdout, which
`SOURCE_DIR="$(clone_source)"` captured, so building from a commit
hash (`-r <sha>`) always failed. Those now go to stderr. Found
2026-10-06 building luajit.
29. syspkg's and gstore's trust checks walked only the ancestors of the
script's own realpath. Installed, `lib/glacier -> /usr/lib/glacier`
and the `.so`s link into libglacier-ng's store dir, so a non-root-
owned directory there was never checked: root loaded `.so`s from it.
Both now also check every ancestor of each loaded file's realpath.
Found 2026-10-06 by `tests/installed_alpine.sh` (both mutations
caught).
30. A package could ship top-level `files/.glacier-*` entries. As a
symlink, `.glacier-fresh` made syspkg truncate its target as root
(an unsigned extra package emptied `/root/victim`, reproduced);
`.glacier-overwrite` self-granted `-O`. Reserved names refused,
marker writes `O_NOFOLLOW`. 2026-10-06 audit.
31. `gl_rmdir_recursive` (lstat, then opendir by path) could be steered
into a symlink's target when root deleted a user's tree (`gstore
-x`). Now fd-relative. Race test in audit_alpine.sh.
32. `add_to_archive` (same class, `gstore -b`). Now fd-relative; no test
fails on the old code (window too narrow; review only).
33. Manifests could `@include` any file (read as root); a dependency
entry without a name cut the list short. Both refused.
34. gpkg_common's `read_manifest_fields` (tar + pattern match) could
disagree with the library: a decoy comment hid a replacement. Now
`manifest_of` (`gl_read_pkg_manifest`).
35. Format strings: gstore's `argv[0]`/UID and gpkg `-u`'s operand
reached `lg_printf` as the format; `%d` was given Lua numbers;
gstore UIDs wrapped past uid_t; `gl_abort_transaction` wiped a stage
without holding the lock. All fixed (`lg_printf` now has a printf
format attribute).
## Known gaps (not bugs, not yet scheduled)
- No migration for stores made before bug 24's fix (0700 dirs stay).
- Extracted entries keep setuid/setgid bits (`& 07755`), as before.
- No size limit on extraction (a decompression bomb fills the disk).
- Bug 32's fix has no test that fails on the old code.
- Hash verification is still CLI-only (ROADMAP 3.2); everything else
added 2026-10-05 is enforced in the library.
- A dependency refusal for `-x` of a name installed in several repos
comes at commit, after the prompt (its repo is chosen after it).
## Backups of glacier-ng / everest files (no git there)
The rule stands (back up before editing: `file.old`, then `.old2`...).
All of glacier-ng's backups were deleted 2026-10-06 at the user's
request. The 2026-10-07 session made new ones: `lib/glacier_cdef.lua.old`,
`lib/gpkg_common.lua.old`, `gpkg.old`, `syspkg.old`,
`tests/{audit,overwrite,syspkg,repo}_alpine.sh.old`,
`lib/gpkg_common.lua.old2`/`.old3`, `tests/repo_alpine.sh.old2`/`.old3`,
`tests/fetch_alpine.sh.old`, `lib/gpkg_common.lua.old4`,
`glacier-bootstrap.old`, `tests/bootstrap_alpine.sh.old`,
`tests/repo_alpine.sh.old4`, `tests/fetch_alpine.sh.old2`; Everest:
`scripts/container.sh.old4`, `everest.sh.old3`, `README.md.old3`,
`templates/minimal.template.old2`, `pkgsrc/everest-bootscripts/glacier.old`.
Everest still has its earlier ones (`everest.sh.old`/`.old2`,
`scripts/container.sh.old`/`.old2`/`.old3`, `README.md.old`/`.old2`,
`Dockerfile.build.old`, `templates/minimal.template.old`,
`pkgsrc/everest-bootscripts/profile.old`). Leave them unless the user
says to clean up.
## Conventions
- **C style:** tabs, but the codebase is inconsistent; match the
surrounding function.
- **Paths:** `PATH_MAX` stack buffers plus `snprintf` for path
construction.
- **Lua requires:** gpkg uses bare CWD-relative requires.
syspkg and gstore pin their paths (see CLIs).
- **Lua numbers in C varargs:** a Lua number reaches C varargs as a
double, so never pass one to `lg_printf("%d")`. Use `%s` with
`tostring()`, or `ffi.new("int", n)`.
- **Shell commands from Lua:** always `shell_quote` interpolated paths
(syspkg runs as root).
- **Verify by running.** Several real bugs here were found only by
executing something.