1048 lines
58 KiB
Markdown
1048 lines
58 KiB
Markdown
# CLAUDE.md
|
|
|
|
Context for Claude Code sessions on libglacier-ng: the package manager
|
|
for Everest Linux. Kept as a handoff, so a fresh session starts with full
|
|
continuity instead of re-deriving everything. Last full rewrite:
|
|
2026-10-04; brought fully up to date at the end of the 2026-10-07
|
|
session. **Update it as you go.**
|
|
|
|
## Start here
|
|
|
|
1. `ROADMAP.txt`: what's left, in priority order, plus the decisions that
|
|
are still open (section 6). Pick work from it.
|
|
2. This file: how things work, what the user wants, how to test.
|
|
3. The design docs, which are authoritative for their topics and not
|
|
repeated here:
|
|
- `SECURITY_DESIGN.txt`: integrity verification, scope separation,
|
|
refusal matrix, syspkg's privilege model.
|
|
- `PACKAGE_SCOPES.txt`: layout, link routing, `/etc` config handling.
|
|
- `IMAGE_DESIGN.txt`: system images (bootstrap OUTPUT). Sections 3
|
|
(sysroot) and 4 (`syspkg --root`) are IMPLEMENTED (milestones 1-2);
|
|
the rest is still a proposal. Open decisions are in section 9.
|
|
- `GLACIER_BOOTSTRAP.txt`: `glacier-bootstrap`. Local mode is
|
|
IMPLEMENTED (ROADMAP 1.3; section 10 says how it differs from the
|
|
plan); remote mode and `--pin` are still a plan. Open: 9a, 9f.
|
|
- `REPO_DESIGN.txt`: remote repos + signing (ROADMAP 3.1). Sections
|
|
1 and 10 = the user's decisions (TLS library still open); steps
|
|
1-4 of section 9 DONE (see "3.1 status"); sections 7, 5 and 6
|
|
describe steps 2, 3 and 4 as implemented.
|
|
- `SECURITY_REVIEW_2026-10-06.md`: the full audit (findings, fixes,
|
|
what was removed, what's still open).
|
|
- `SECURITY_REVIEW_2026-07-16.md`: older review findings.
|
|
|
|
Nothing from the 2026-10-04 through 2026-10-07 sessions is
|
|
committed. The user said "do not commit yet" (2026-10-05), and on
|
|
2026-10-06 that their git server is down: they will commit themselves
|
|
once it's back. libglacier-ng is on branch
|
|
`dev` with a large uncommitted diff (`src/`, the docs, `build/`). Commit
|
|
only when the user asks; when they do, propose a few logical commits.
|
|
|
|
2026-10-05 session: system-scope permissions fix (bug 24), ROADMAP 2.3
|
|
dependencies (`src/deps.c`), index corruption on abort (bug 25), ROADMAP
|
|
1.1 sysroot, live removal deleting foreign files (bug 26), ROADMAP 1.2
|
|
(`syspkg -R`, `--strict`) + 2.6 (file collisions), gpkg ignoring unknown
|
|
flags (bug 27).
|
|
|
|
2026-10-06 session: ROADMAP 1.3 `glacier-bootstrap` local mode, and
|
|
Everest's initramfs switched to it (boots in QEMU). Tidy-up (gbuild's
|
|
hint removed). ROADMAP 1.4 DONE (packages for glacier itself; details in
|
|
ROADMAP 1.4 and section 7): full regression green, new
|
|
`tests/installed_alpine.sh` for the installed layout, which found bug 29
|
|
(fixed, mutation-checked, Everest rebuilt and booted with the fix).
|
|
|
|
The installer (1.5) is DEFERRED by the user: manual installation is
|
|
one of Everest's main features. Then ROADMAP 3.1 was designed with the
|
|
user (REPO_DESIGN.txt, every decision taken; only the TLS library for
|
|
curl in base is open) and steps 1, 2 and (that evening, after the
|
|
laptop died mid-rebuild) 3 of its section 9 were done.
|
|
|
|
2026-10-07 session: ROADMAP 3.1 step 4 (`-f`, `-u`, dependency
|
|
closure, the cache; see "3.1 status"). Help text for -f/-u/-O
|
|
shortened at the user's request. Refresh output (user's wording): `[i] Syncing
|
|
all repositories...`, then `[✓] All repositories synced successfully.`
|
|
if every repo succeeded; the per-repo success/up-to-date lines only
|
|
with `-V`; errors always (repo_alpine.sh checks both ways).
|
|
Then (same day) glacier-bootstrap configures repositories (user: it must
|
|
leave a semi-functional system, pacstrap-like): see "Bootstrap
|
|
repositories" below.
|
|
|
|
2026-10-09: help text. `gsign`, `grepo`, `gsha256sum` (C, `getopt_long`,
|
|
static `help()` in each tool's own file), `glacier-bootstrap` and `gworld`
|
|
now have `-h/--help` and `-v/--version` (`v4.0.0`) in gpkg's format
|
|
(`%-25s` `{-x --long} ARG` lines, GPL footer), with long forms for every
|
|
flag. `-h` goes to stdout, exit 0; usage errors stay one usage block on
|
|
stderr, exit 1. repo_alpine.sh checks the help of grepo/gsign/gsha256sum.
|
|
Backups: `glacier-bootstrap.old2`, `gworld.old`, `tests/repo_alpine.sh.old5`.
|
|
|
|
### Bootstrap repositories (2026-10-07)
|
|
|
|
The user chose: the config comes from a NEW base package
|
|
(`everest-repos` 1.0.0: `etc/glacier/repos.conf` + `keys/everest-2026.pub`,
|
|
source `~/Projects/everest/pkgsrc/everest-repos`; everest-bootscripts
|
|
1.3.0 no longer ships `etc/glacier`, its old copy is
|
|
`pkgsrc/everest-bootscripts/glacier.old`); bootstrap runs `syspkg -R T
|
|
-y` after installing and a failed fetch is an error (exit 7, no
|
|
"Bootstrapped"); under `-R`, `-y`/`-f`/`-u` use T's OWN keyring
|
|
(`keyring(sysroot)` in gpkg_common.lua; `require_trusted` also refuses a
|
|
file whose realpath leaves T). `-R -l` signatures stay the host's
|
|
(`HOST_KEYRING`). No url in T: `[!] No remote repositories are
|
|
configured in T.`, exit 0 (Everest today: base isn't published).
|
|
Copying the host's repos.conf (pacstrap's mirrorlist) left out for now.
|
|
The user said everything a template lists for base also goes in the
|
|
base repo (the published repo = the template's base packages); no url
|
|
yet. Everest rebuilt (bootscripts 1.3.0, everest-repos, glacier-ng 4.0.0
|
|
in place), initramfs remade, booted: `/etc/glacier` from everest-repos,
|
|
12 packages.
|
|
|
|
### Audit and cleanup (2026-10-06, late)
|
|
|
|
At the user's request, before 3.1 step 4: a full security audit
|
|
(`SECURITY_REVIEW_2026-10-06.md`, bugs 30-35 below), unused code removed,
|
|
and junk cleaned up. The user's choices: delete only the `.old` backups
|
|
and stray binaries (every prototype/note file stays: libglacier-ng's
|
|
`src/{deb,gpkg,templ,report,test_*}.c`, `src/html`, `pkg/`, glacier-ng's
|
|
`todo`, `tree/`, `test-*`, `pkgs/`...); libglacier-ng's build outputs are
|
|
untracked (`.gitignore`: `/build/`; the deletions show in the next
|
|
commit); the `.old` backup rule STAYS (see "Backups" below for current ones). New tests:
|
|
`tests/audit_alpine.sh` (26 checks), `tests/test_abort_unlocked.lua`.
|
|
Full regression green; Everest rebuilt and booted. Everest's own `.old`
|
|
files were not touched (not one of the two repos).
|
|
|
|
### 3.1 status
|
|
|
|
Step 4 DONE (2026-10-07; REPO_DESIGN section 6 "As implemented"). The
|
|
user chose: the FIRST repo (repos.conf order) serving a name wins, at
|
|
its highest version; a dependency installed too old is upgraded in the
|
|
same transaction (keeps its auto/explicit mark); cached downloads are
|
|
deleted after the commit that installs them.
|
|
- libglacier-ng: `gl_scope_cache_dir(ctx, repo, out)` (transaction.c;
|
|
`<base>/cache/REPO`, checked like `repos` via the shared static
|
|
`checked_subdir`); `log.c` help lists `-f`/`-u`.
|
|
- glacier-ng `gpkg_common.lua`: `M.fetch(names, opts)`,
|
|
`M.upgrade(names, opts)`; `load_indexes` (refresh, then read the
|
|
accepted copies), `pick`, `close_deps`, `download` (SIZE-capped,
|
|
hashed), `matches_index` (manifest vs index entry), `install_plan`.
|
|
`M.install` is now `verify_all` + `install_copies(copies, original,
|
|
opts, meta)`, shared with -f/-u (`meta`: auto, may_replace, dep,
|
|
cache). `verify_and_copy(path, vouched)` takes the index's hash in
|
|
place of a sidecar. syspkg `-f`/`-u` modes; gpkg `-f`/`-u` (no
|
|
longer "not implemented").
|
|
- New suite `tests/fetch_alpine.sh` (84 checks; 16 Lua mutants + 1 C,
|
|
all caught except the download-level hash check, equivalent: kept as
|
|
defence in depth). The `gpkg -u "%s%n"` format check moved there from
|
|
audit_alpine.sh. Full regression green.
|
|
- Everest's libglacier-ng/glacier-ng 4.0.0 were REBUILT in place
|
|
2026-10-07 with -f/-u/-y (user: keep 4.0.0, no bump) and the
|
|
initramfs remade; in QEMU `gpkg -f hi` as a wheel user now says
|
|
`[x] No remote repositories are configured.` (shipped repos.conf has
|
|
no url; no curl in base). `container.sh initramfs` now `mkdir -p`s
|
|
`$G/lib/glacier` (gone from the tree). Not done: no published repo;
|
|
TLS library still open. Next: step 5 (`glacier-bootstrap -r`),
|
|
then 6 (user repos).
|
|
|
|
Step 3 DONE (2026-10-06, evening; REPO_DESIGN section 5 "As
|
|
implemented"). The user chose: a refresh-only flag `-y`, the date-form
|
|
SERIAL (YYYYMMDDNN), grepo in C.
|
|
- libglacier-ng: `src/grepo.c` (`grepo -s KEY.sec -r REPO [-n SERIAL]
|
|
[-e DAYS] DIR`, Makefile target in `all`, installed to bin, built from
|
|
source like gsign); `pkg.c`: `gl_index_parse(text, len, repo, out)` /
|
|
`gl_index_free` (strict; takes the signed BYTES; refuses NUL and '@'),
|
|
`gl_conf_repos(path, out, len)` ("NAME\tURL\n" lines), repos.conf
|
|
validation shared in `read_repos_conf`; `transaction.c`:
|
|
`gl_scope_state_dir(ctx, "repos", out)` (creates/checks
|
|
`<base>/repos`). `log.c` help lists `-y`.
|
|
- glacier-ng: `M.refresh(opts)` in `gpkg_common.lua` (fetch file:// or
|
|
curl https, verify with a key listed for THAT repo, parse, rollback /
|
|
same-serial / expiry checks, rename into `<base>/repos`, under the
|
|
scope lock); `syspkg -y` (needs root and `-n`; `-R` works), `gpkg -y`
|
|
(system repos.conf + keyring until step 6). Dockerfiles copy grepo to
|
|
/usr/local/bin. New suite `tests/repo_alpine.sh` (103 checks; 18
|
|
mutants, all caught). Full regression green. Everest's libglacier-ng
|
|
and glacier-ng packages rebuilt with it.
|
|
- Next: step 4 (`-f` with dependency closure, then `-u`; the cache in
|
|
`<base>/cache`, section 5's Fetch).
|
|
|
|
Step 2 DONE (2026-10-06, second part of the session; REPO_DESIGN
|
|
section 7 "As implemented" has the details):
|
|
- libglacier-ng `src/pkg.c`: `gl_conf_repo_keys(path, repo, out, len)`,
|
|
a strict repos.conf reader; new statuses `GL_CONF_ERR_INVALID = 4`,
|
|
`GL_CONF_ERR_SPACE = 5`.
|
|
- glacier-ng `lib/gpkg_common.lua`: `verify_all(paths, opts)` refuses a
|
|
base package in system scope without a trusted signature
|
|
(`signed_for`, `trusted_keys`, `untrusted`); `verify_and_copy` returns
|
|
the sidecar's bytes. glacier-bootstrap copies the sidecar's BYTES and
|
|
the `.sig`. `gbuild -k`/`--key`, `tests/mkpkg -k`.
|
|
- Every `Dockerfile.*` installs gsign + gsha256sum in /usr/local/bin and
|
|
a TEST keyring (`glacier-test`, secret `/root/glacier-test.sec`).
|
|
`syspkg_alpine.sh` / `overwrite_alpine.sh` make their own key.
|
|
- New suite `tests/keyring_alpine.sh` (68 checks; 17 mutants: 10 Lua/
|
|
shell, 7 C, all caught). Full regression green.
|
|
- Everest: `./everest.sh keygen` made `everest-2026`; the SECRET key is
|
|
`~/.config/everest/keys/everest-2026.sec` (user's choice: outside the
|
|
project), mounted read-only for `build`, which refuses without it.
|
|
Every package is built `gbuild -k`. everest-bootscripts 1.2.0 ships
|
|
`etc/glacier/repos.conf` (base: everest-2026) and
|
|
`etc/glacier/keys/everest-2026.pub`; `container.sh initramfs`
|
|
installs the same into the build container's `/etc/glacier`.
|
|
The signed `build all` was cut off after busybox (laptop died); the
|
|
other 9 were rebuilt `-k` the evening of 2026-10-06, `initramfs` ran
|
|
through glacier-bootstrap (signature checks passed for all 11), and
|
|
Everest booted in QEMU: repos.conf + key in `/etc/glacier`, index
|
|
lists all 11, syspkg/gpkg run as root and as a fresh user.
|
|
- Base KEEP_OLD backups can't be restored with `-lO` (no signature):
|
|
the user accepted that until the restore interface (REPO_DESIGN 10f).
|
|
|
|
Step 1 DONE (2026-10-06):
|
|
- `src/monocypher/`: Monocypher 4.0.3, vendored UNMODIFIED (provenance
|
|
and sha512 in its README; GitHub and monocypher.org copies matched).
|
|
Compiles clean under the project's `-Werror` flags.
|
|
- `src/sign.{c,h}`: signify formats (strict parse, canonical base64),
|
|
`gl_sign_read_pubkey`/`_read_sig`/`_check_seckey`/`_verify`/
|
|
`_verify_file`/`_file`/`_generate`, statuses `GL_SIGN_OK`..
|
|
`GL_SIGN_ERR_WRITE = 9`. Built into `libglacier_verify.so` (NOT into
|
|
the combined `libglacier-ng.so`).
|
|
- `src/gsign.c` (`-G`/`-S`/`-V`, lg_printf messages naming the file at
|
|
fault), Makefile targets `gsign` (in `all`, installed to bin like
|
|
gsha256sum).
|
|
- `glacier-ng/tests/sign_alpine.sh` (alpine:latest, mounts /libsrc):
|
|
round trips, signify interop both ways (Alpine `signify`), encrypted
|
|
and corrupted secret keys, malformed key/sig files, 64 MiB limit,
|
|
O_NOFOLLOW, key modes, a direct FFI call of `gl_sign_generate`.
|
|
Passes in full. (`luajit -e CODE ARG` runs ARG as a script, so the
|
|
FFI snippet is written to `/gen.lua` first.)
|
|
- Mutation checks: 12 mutants, all caught except m9 (padding position,
|
|
`j < 2`), which is EQUIVALENT: kept, it prevents `3 - pad`
|
|
underflowing. m5 (fchmod), m6 (final newline) and m10 (O_EXCL) needed
|
|
the umask-077, final-char and FFI checks respectively.
|
|
- `glacier-ng/lib/glacier_cdef.lua`: `gl_sign_*` declarations and
|
|
`M.sign_name()`; checked against the real `.so` (struct sizes 40/72,
|
|
every call on signify-made files). Full regression green after it.
|
|
|
|
Also open, for the user (raised 2026-10-06, not answered yet):
|
|
- libgcc: luajit is built `-static-libgcc` (Everest has no libgcc_s).
|
|
Claude's call; the alternative is a GCC runtime package in base
|
|
(ROADMAP section 6).
|
|
- Leftover `~/Projects/everest/pkgs/everest-bootscripts-1.0.0.gpkg`
|
|
(+ sidecar), superseded by 1.1.0. Not deleted: ask first.
|
|
|
|
### Everest's glacier packages (1.4)
|
|
|
|
- `~/Projects/everest/scripts/container.sh` builds zlib, libarchive
|
|
(gzip/tar only), libconfig, luajit (pinned commit,
|
|
`LDFLAGS=-static-libgcc`), libglacier-ng, glacier-ng (all base) and
|
|
doas (extra, OpenDoas, no PAM).
|
|
- glacier-ng layout: `usr/lib/glacier-ng/{syspkg,gpkg,gstore,lib/*.lua}`,
|
|
`lib/glacier -> /usr/lib/glacier` (absolute: relative links resolve
|
|
inside the store), `usr/bin/X -> ../lib/glacier-ng/X`,
|
|
`etc/glacier/glacier.conf`.
|
|
- everest-bootscripts 1.2.0 (1.1.0 before signing) ships `/etc/doas.conf` (0400): `permit
|
|
:wheel` plus nopass `gstore args -n` for BOTH `cmd gstore` and `cmd
|
|
/usr/bin/gstore` (OpenDoas matches `cmd` against what was typed).
|
|
- `templates/minimal.template` and `PACKAGES` in everest.sh list all 12
|
|
packages (everest-repos added 2026-10-07). `Dockerfile.build` has autoconf, automake, libtool, cmake,
|
|
pkgconf.
|
|
|
|
## The three projects
|
|
|
|
| Path | What | Git? |
|
|
|---|---|---|
|
|
| `~/Projects/libglacier-ng` | C library, design docs, this file | yes (`dev`) |
|
|
| `~/Projects/glacier-ng` | Frontend: `gpkg`, `gstore`, `syspkg`, `gbuild`, `gworld`, `lib/*.lua`, `tests/`, Dockerfiles, `pkgs/` | **no**: back files up before editing (`file.old`; if that exists, `.old2`, `.old3`, ...) |
|
|
| `~/Projects/everest` | The distro build: `everest.sh build\|initramfs\|boot`, packages, bootable QEMU image | **no**: same backup rule (but never inside `pkgsrc/*/etc/`, see Boot test) |
|
|
|
|
Nothing prebuilt is kept in either repo (the host-built `.so`s in
|
|
`glacier-ng/lib/glacier/` were removed 2026-10-06). Every image and test
|
|
builds libglacier-ng from source; the Dockerfiles create
|
|
`/app/lib/glacier/` with the musl `.so`s.
|
|
|
|
## What this project is
|
|
|
|
A from-scratch, security-focused Linux distribution (Everest) with its own
|
|
package manager (libglacier-ng + `gpkg`/`gstore`/`syspkg`) and build tooling
|
|
(`gbuild`/`gworld`). It's a C library, LuaJIT+FFI CLI tools and bash build
|
|
scripts. The user writes it; Claude implements and designs alongside them.
|
|
|
|
## How the user wants things done
|
|
|
|
These are preferences stated in sessions. Follow them.
|
|
|
|
- **Output format:** every message uses glacier's `lg_printf` levels:
|
|
`[x]` is error (2), `[!]` warning (1), `[i]` info (0), `[✓]` success (3).
|
|
Messages are capitalized. Exact strings the user chose include
|
|
`Cannot open system index: Permission denied` (no period) and
|
|
`Nothing to do.` (success level).
|
|
- The pre-trust-check `die()` in syspkg/gstore reproduces the `[x]`
|
|
format byte for byte, because it can't load `libglacier_log.so`
|
|
before that file has been verified.
|
|
- **Never say "Completed" after an error.** If the post-commit link pass
|
|
refused any path (`ctx->link_refusals`, counted only for packages
|
|
installed by that transaction), gpkg/syspkg end with
|
|
`[x] N paths were not linked.` (`1 path was not linked.`) and exit 1.
|
|
The transaction itself stays committed (2026-10-05).
|
|
- **No hint messages.** Don't print lines that point users to other flags
|
|
or commands (for example "use -lO to ..."). The user called these
|
|
clutter. A refusal states the fact, in one line. The user also removed a
|
|
`Run doas gstore -n ...` follow-up hint from gstore themselves.
|
|
- **What goes where.** ALL design docs (and ROADMAP, this file) live in
|
|
libglacier-ng, frontend ones included. Other than docs, libglacier-ng
|
|
holds only backend material (C sources, the build). Frontend code,
|
|
tests, Dockerfiles and everything the bootstrap runs on live in
|
|
glacier-ng. Clarified 2026-10-06.
|
|
- **Ask before settling a design choice that is the user's to make**, and
|
|
give a recommendation. They have answered every such question promptly.
|
|
Don't silently pick a layout or semantics.
|
|
- **Verify by running, in Docker or QEMU, never by inspection alone.** For
|
|
any security fix, also prove the test catches the bug: remove the
|
|
defence in a throwaway copy and confirm the test fails ("mutation
|
|
check"). Several defences here are verified that way.
|
|
- Use doas, not sudo. syspkg is never setuid.
|
|
- **Help text is terse** (2026-10-07): one short phrase per flag, no
|
|
mechanism details (`-f` "Install packages from a remote repository",
|
|
`-u` "Update system or individual packages", `-O` "Allow Glacier to
|
|
overwrite existing files").
|
|
|
|
## Decisions the user has made (don't relitigate)
|
|
|
|
- **Per-uid user layout** (`/glacier/usr/<uid>/...`, root-owned parent).
|
|
Both root and the user can create it: root runs `gstore -n UID`; a user
|
|
runs `doas gstore -n` through a doas rule pinned to `args -n`
|
|
(option "B").
|
|
- **`syspkg -n` creates the top-level tree itself** (GLACIER_BOOTSTRAP 9d).
|
|
- **`/etc` config files: option 1.** Copy on install, `.new` on conflict
|
|
(GLACIER_BOOTSTRAP 9c).
|
|
- **`-O` with `-l` = replace.** `-O` displaces non-glacier files, and
|
|
`-lO` also replaces an installed *different version* (upgrade or
|
|
downgrade from a local file). Plain `-l` with another version installed
|
|
refuses in one line.
|
|
- **`KEEP_OLD`** in `/etc/glacier/glacier.conf`: the number of backups of
|
|
replaced versions to keep per package (0 = none).
|
|
- It has **no default.** It ships as `KEEP_OLD = 1`.
|
|
- If it's missing or invalid, an operation that would replace a version
|
|
errors. Only then: other operations don't need it.
|
|
- "Backed up" means the old package's files are compressed into a
|
|
`.gpkg`. A restore interface comes later.
|
|
- **`gpkg -u` / `syspkg -u` upgrade from a REMOTE repository**, and
|
|
`gpkg -f` is `-l` but fetched by name from a remote repository. Done
|
|
2026-10-07 (3.1 step 4). Never against local files.
|
|
- **-f/-u details (2026-10-07):** first repo by priority wins (then its
|
|
highest version); a too-old installed dependency is upgraded in the
|
|
same transaction; cached downloads are deleted after commit.
|
|
- **Dependencies (ROADMAP 2.3, decided 2026-10-05):** see the spec in
|
|
ROADMAP 2.3. In short: minimum versions, conflicts both ways by name,
|
|
no override flag, report every problem, missing deps refuse until 3.1
|
|
brings automatic closure, auto/explicit recorded in the index now.
|
|
- **sysroot (IMAGE_DESIGN section 3) accepted as proposed**, plus the
|
|
library refuses a non-"/" sysroot for `GL_SCOPE_USR`. Order of work:
|
|
2.3 first, then 1.1.
|
|
- **System scope is world-readable** (dirs 0755, index files 0644): every
|
|
user must be able to run what syspkg installs (2026-10-05). User scope
|
|
stays 0700, and other users' programs stay unreachable to non-root
|
|
users. Root CAN run them by full path, but other users' links are
|
|
deliberately NOT on root's `PATH` (a user could get code run as root).
|
|
- **Link refusals after a commit** end with `[x] N paths were not linked.`
|
|
and exit 1, never "Completed" (the user's wording rule: don't say
|
|
"completed" if there was an error).
|
|
- **`syspkg -R DIR`** (not `-r`), `--root DIR`, `--root=DIR`; `-R`
|
|
IMPLIES `--strict`; `--strict` alone is opt-in on the host.
|
|
- **File collisions (2.6) are always refused**, every scope, strict or
|
|
not (folded into 1.2 at the user's go-ahead).
|
|
- **PATH:** per-user links, then `/glacier/sys/links/bin`, then the system
|
|
(`/usr/bin` on Everest), in the Docker images AND Everest's
|
|
`/etc/profile`. Non-base system packages must be on `PATH`; base
|
|
programs must be runnable by every user.
|
|
- `everest-bootscripts` is the name of the base `/etc` package. Its source
|
|
lives in `~/Projects/everest/pkgsrc/`.
|
|
- **glacier-bootstrap (2026-10-06):** template = `NAME=` + `PKG=name ver
|
|
[sha256:hex]` lines, no `SYS_PROFILE` (unknown key until ROADMAP 3.3);
|
|
it runs the `syspkg` on PATH (doas unless root); a template dependency
|
|
must be listed at that version OR LATER (the 2.3 rule); Everest's
|
|
initramfs is built through it.
|
|
- **Installing Everest (2026-10-07):** possible from a live Everest
|
|
system OR any Linux host; the LIVE SYSTEM is the preferred, officially
|
|
supported way (so base/the live medium must eventually carry what a
|
|
manual install needs). The user does the first QEMU disk install
|
|
themselves: don't start it.
|
|
- **Everest supports glibc as well as musl** (2026-10-07), Gentoo/Void
|
|
style: the user picks the libc at install time, one per system.
|
|
Today everything is built for musl (Alpine); glibc needs ROADMAP
|
|
4.1's profiles. How packages are kept apart is still open (ROADMAP 6).
|
|
- **Signing (2026-10-06, REPO_DESIGN 10e/10f):** Everest's SECRET key
|
|
stays outside the project (`~/.config/everest/keys`, mounted
|
|
read-only); a base KEEP_OLD backup refusing under `-lO` (unsigned) is
|
|
accepted until the restore interface.
|
|
|
|
## Core architecture
|
|
|
|
**`gl_context_t`** (transaction.h) is the central abstraction. Fields, in
|
|
order: `uid`, `root_path`, `index_path`, `store_path`, `links_path`,
|
|
`mode` (`GL_CTX_LIVE`/`GL_CTX_STAGE`), `scope`
|
|
(`GL_SCOPE_USR`/`GL_SCOPE_SYS`), `lock_fd`, then:
|
|
|
|
| Field | Set by | Meaning |
|
|
|---|---|---|
|
|
| `overwrite` | caller (after init, which zeroes it) | `-O`: the next link pass may displace foreign files |
|
|
| `sysroot` | `*_at` init | install root, `""` = the running system |
|
|
| `link_refusals` | link pass; commit copies the relink's count back | paths refused for this transaction's packages |
|
|
| `strict` | caller (after init, which zeroes it) | refuse those before committing instead |
|
|
|
|
New fields go at the END, and `lib/glacier_cdef.lua` must mirror the
|
|
struct exactly (FFI has no compiler to catch a mismatch).
|
|
|
|
**Status codes worth knowing:** `GL_TXN_ERR_DEPS = 9` covers every
|
|
pre-commit refusal (dependency, conflict, file collision, strict link
|
|
refusal); the library has already printed why, so the CLIs print nothing
|
|
more. Live-context equivalents: `GL_INSTALL_ERR_DEPS = 8`,
|
|
`GL_REMOVE_ERR_DEPS = 4`. `GL_INSTALL_ERR_SCOPE = 7` is base-in-user-
|
|
scope.
|
|
|
|
**Install root / sysroot** (ROADMAP 1.1, done 2026-10-05):
|
|
- `gl_init_live_context_at` / `gl_init_stage_context_at` / `gl_init_sys_at`
|
|
take a sysroot; the old functions wrap them with `"/"`.
|
|
`gl_normalize_sysroot` (common.h) refuses relative paths and `.`/`..`
|
|
components. **User scope refuses any sysroot** other than `/`.
|
|
- The context's paths are **INSTALL paths** (`T/glacier/sys/store`), so
|
|
every filesystem operation uses them unchanged: a forgotten translation
|
|
writes into T, never the host. That's deliberately the reverse of
|
|
IMAGE_DESIGN's wording (runtime paths stored, `gl_at` on every
|
|
operation), with the same effect and a safer failure mode.
|
|
- Translate ONLY where a path's runtime meaning matters:
|
|
`gl_runtime_path(ctx, p)` for link targets, `is_glacier_symlink`
|
|
prefixes and `.displaced` keys; `gl_at(ctx, runtime, out)` to check a
|
|
link's target (prune). `/usr` and `/etc` destinations are
|
|
`ctx->sysroot` + path.
|
|
- **Trust check** (`gl_check_sysroot` in transaction.c, also inside
|
|
every `*_at` init and `gl_init_sys_at`; helper `gl_sysroot_trusted` in
|
|
common.h): the root and every ancestor must exist and be root-owned,
|
|
non-symlink directories, not group/world-writable. So `/tmp/x` is
|
|
refused. Messages: `Refusing install root R: P is a symlink.` / `is
|
|
missing.` / `is not root-owned, or is group/world-writable.`
|
|
- Test: `tests/sysroot_alpine.sh` (host `/usr`, `/glacier`, `/etc`
|
|
byte-identical; `chroot T` runs a linked binary; 8 mutations caught).
|
|
- Not translated, on purpose: `KEEP_OLD` is read from the HOST's
|
|
`/etc/glacier/glacier.conf` even under `-R` (the build host's policy);
|
|
user scope's dependency view of "the system" is always the host's.
|
|
|
|
**Layout.** Each scope owns ONE base directory holding
|
|
`{index,store,links,stage,old,lock,backup}`. Every create, rename and lock
|
|
a transaction does stays inside it.
|
|
|
|
| Scope | Base dir | Ownership |
|
|
|---|---|---|
|
|
| `GL_SCOPE_USR` | `/glacier/usr/<uid>/` | uid, mode 0700, inside a root-owned 0755 `/glacier/usr` |
|
|
| `GL_SCOPE_SYS` | `/glacier/sys/` | root, world-readable: dirs 0755, index files 0644 (`gl_scope_dir_mode`/`gl_scope_file_mode` in transaction.h), whatever the umask |
|
|
|
|
- The helpers `gl_usr_base`, `gl_usr_base_check` and the layout comment
|
|
are in `common.h`.
|
|
- `gl_init_stage_context` refuses a user base that isn't a real directory
|
|
(lstat) owned by its uid with no group/other bits, or whose
|
|
`/glacier/usr` is loosened. `gl_usr_base_status` exports that check, so
|
|
gpkg can say why before any prompt.
|
|
|
|
**Transaction lifecycle.** `gl_init_live_context` / `gl_init_stage_context`
|
|
→ `gl_commit_transaction` or `gl_abort_transaction`.
|
|
- Stage init takes the lock FIRST, then clears any leftover stage.
|
|
- Stage contexts hardlink-seed `index`/`store` from live, so removing a
|
|
live package inside a transaction needs no special casing. Seeding
|
|
copies dotfiles: only `.` and `..` are skipped.
|
|
- A stage on another filesystem fails at seeding with `GL_TXN_ERR_XDEV`.
|
|
- **Commit:** first `gl_check_txn` (deps, conflicts, collisions, and
|
|
strict link refusals if `ctx->strict`); any problem → nothing changes,
|
|
`GL_TXN_ERR_DEPS`. Then rename-based promotion (stage → live, previous
|
|
live → `old/`), then fsync the base. Then `gl_relink_store` regenerates
|
|
links from a *fresh live context* and rescans the WHOLE store (every
|
|
commit relinks every package, so link-time logic must be idempotent);
|
|
its `link_refusals` is copied back into the caller's context. `old/` is
|
|
then deleted. (The index is rebuilt inside the stage by each
|
|
install/remove, and promoted with it.)
|
|
- **Live contexts** (no transaction; tests and third-party callers):
|
|
`gl_install_pkg` runs `gl_check_plan` itself, and `gl_remove_pkg`
|
|
removes the package's links by pruning (`gl_relink_store`), like a
|
|
commit.
|
|
- **Index files** are named to the second and seeding hardlinks them:
|
|
never open a seeded file for writing, always write elsewhere and
|
|
`rename()` (bug 25).
|
|
|
|
**Link routing** (per package, in `gl_link_pkg`):
|
|
|
|
| Scope | Repo | Destination |
|
|
|---|---|---|
|
|
| USR | any | `<base>/links` |
|
|
| SYS | `base` | `/usr` |
|
|
| SYS | anything else | `/glacier/sys/links` |
|
|
|
|
- A package's leading `usr/` is stripped.
|
|
- In system scope, a package's top-level `etc/` is never linked: it's
|
|
COPIED into `/etc` (see `/etc` config handling below).
|
|
|
|
**Repos.** `base` (the minimal system; boot-critical by definition, a
|
|
repo-level rule), `extra`, `community`. `extra` and `community` behave
|
|
identically.
|
|
|
|
**Hidden files in the store** (dot-prefixed, so linking, index rebuild and
|
|
the store scanners skip them, while seeding carries them through
|
|
transactions):
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `<ver>/.glacier-manifest` | the package's manifest, saved at install, used by `gl_backup_pkg` |
|
|
| `<ver>/.glacier-fresh` | one-shot "installed in this transaction"; link conflicts are only reported when present |
|
|
| `<ver>/.glacier-overwrite` | one-shot `-O` permission for this package's next link pass |
|
|
| `<ver>/.glacier-auto` | installed as a dependency, not by name (index field 4 = `auto`); set by `gl_set_pkg_auto`, nothing sets it until 3.1 |
|
|
| `<store>/.displaced/<runtime abs path>` | originals displaced by `-O`, restored when the link that replaced them is pruned; keyed by RUNTIME path (no sysroot prefix) |
|
|
| `<store>/.etc-state/<path>` | the last version of each `/etc` file a package offered |
|
|
|
|
**Configuration.** `/etc/glacier/glacier.conf` (libconfig syntax).
|
|
- It's read with `gl_conf_get_int(path, key, &out)` in `pkg.c`. Missing
|
|
file, missing key and wrong type are distinct statuses, and there are no
|
|
defaults.
|
|
- The shipped copy is `glacier-ng/etc/glacier/glacier.conf`. Every
|
|
Dockerfile installs it.
|
|
|
|
## Safety invariants: read before touching links/symlink code
|
|
|
|
`/usr` is NOT glacier-exclusive territory. Treat every link destination as
|
|
possibly shared:
|
|
|
|
- **Ownership test.** `is_glacier_symlink(path, store_prefix)` is the only
|
|
test for "did glacier create this". Never touch anything it returns
|
|
false for, except through `-O`'s displace/restore.
|
|
- **Overwriting.** `gl_link_pkg` won't overwrite a non-glacier path without
|
|
the `-O` marker. It never descends through *another package's* directory
|
|
symlink into that package's store directory.
|
|
- **Pruning.** `gl_relink_store` PRUNES stale links; it never wipes a links
|
|
destination. A link is stale only if its own store target is gone
|
|
(checked with `lstat`, not by following it): a package symlink may
|
|
legitimately dangle.
|
|
- **Stage links.** The stage links tree is never seeded.
|
|
- **Extraction never writes through a symlink.**
|
|
- Symlink entries are created only after all regular content.
|
|
- Any entry whose parent is a symlink is refused (`crosses_symlink`,
|
|
plus `O_NOFOLLOW`).
|
|
- Any entry with `..` or an absolute path refuses the whole package.
|
|
- A failed extraction removes its half-made version directory.
|
|
- **Recursive deletes.** `gl_rmdir_recursive` skips only `.`/`..`, and
|
|
walks by descriptor (`openat(O_NOFOLLOW)`, `unlinkat`): an entry
|
|
swapped for a symlink is unlinked, never followed (bug 31). Every
|
|
caller passes a glacier-only path; keep it that way.
|
|
- **Root reading a user's tree** (`add_entry_at`, `gstore -b`) is
|
|
fd-relative with `O_NOFOLLOW` too (bug 32).
|
|
- **Reserved names.** A package's top-level `files/.glacier-*` entries
|
|
are refused (`GL_MARKER_PREFIX`, common.h): those names are glacier's
|
|
own version-dir files, and every marker write uses `O_NOFOLLOW`
|
|
(bug 30). The marker names are defined ONCE, in common.h.
|
|
- **Manifests are untrusted**: `gl_gpm2gpkg` refuses `@include` and
|
|
dependency/conflict entries without a valid name (bug 33). The CLIs
|
|
read manifests ONLY through the library (`manifest_of` in
|
|
gpkg_common.lua), never by pattern matching (bug 34). `gl_delete_user`
|
|
lstat-checks its target first, because a symlinked base would otherwise
|
|
have its *target* emptied.
|
|
- **Wide wrappers.** Don't add convenience wrappers that silently touch a
|
|
broader filesystem scope than asked. This codebase has been bitten twice
|
|
by exactly that.
|
|
|
|
## Feature reference (current behaviour)
|
|
|
|
**Integrity verification** (`gpkg_common.lua`: `verify_and_copy`,
|
|
`verify_all`):
|
|
- Each package is copied into a private `mktemp -d` directory while being
|
|
hashed with `libglacier_verify.so`. The result is checked against
|
|
`FILE.sha256` (`sha256:<hex>`).
|
|
- Everything after that (summary, manifest read, `gl_install_pkg`) reads
|
|
ONLY the copy.
|
|
- A missing or malformed sidecar counts as a mismatch and is refused before
|
|
the prompt; one bad package refuses the whole batch. No bypass exists.
|
|
- This lives in the CLIs' shared module, not the library (as
|
|
SECURITY_DESIGN section 4 specifies), so a third-party caller of
|
|
`gl_install_pkg` is not verified (ROADMAP 3.2).
|
|
- The base→USR refusal (`GL_INSTALL_ERR_SCOPE = 7`), by contrast, *is* in
|
|
the library.
|
|
|
|
**`-O` / displace:**
|
|
- A displaced foreign file or symlink moves to `<store>/.displaced/`:
|
|
renamed, or on `EXDEV` copied (keeping mode, owner and mtime) and then
|
|
unlinked.
|
|
- Directories, special files and paths that already have a backup are
|
|
still refused.
|
|
- In user scope `-O` is effectively a no-op, because the links tree is
|
|
rebuilt from scratch on every commit.
|
|
|
|
**Version replacement** (`-lO`, `M.install` in `gpkg_common.lua`):
|
|
1. Installed entries are read from the live index's `current` file
|
|
(`repo::name::version::auto|explicit` lines; files before 2026-10-05
|
|
stop at the version, and every parser accepts both).
|
|
2. A same name+repo with a different version is a replacement. Without
|
|
`-O` it's refused; with `-O`, `keep_old()` reads `KEEP_OLD`, and the
|
|
summary shows `old -> new`.
|
|
3. In the transaction: `gl_backup_pkg(live, repo, name, X.gpkg.part)`, then
|
|
`gl_remove_pkg` from the stage, then install.
|
|
4. After the commit: rename `.part` → `<scope>/backup/<repo>/<name>/<name>-<ver>.gpkg`,
|
|
write its sidecar, and run `gl_prune_backups(dir, KEEP_OLD)` (newest by
|
|
nanosecond mtime).
|
|
- On any abort, `finish()` deletes the `.part`.
|
|
- A backup is a complete installable `.gpkg`, so `-lO backup.gpkg`
|
|
restores it.
|
|
|
|
**`/etc` config handling** (`install_config` / `prune_config` in
|
|
istoreutils.c), system scope only:
|
|
|
|
| Situation | Result |
|
|
|---|---|
|
|
| `/etc/F` missing, no record | install |
|
|
| missing, record exists | the admin deleted it: stays deleted |
|
|
| unchanged since glacier wrote it | updated in place |
|
|
| admin-modified, or never glacier's | never touched; a package change is offered ONCE as `F.new` |
|
|
| no package ships it any more | removed if unchanged, kept otherwise |
|
|
|
|
User-scope `etc/` is still linked (ROADMAP 2.9).
|
|
|
|
**Top-level tree.** `gl_create_fs_tree(root, verbose)` (exported) creates
|
|
the merged-/usr tree: `/bin` and `/sbin` → `usr/bin`, `/lib` → `usr/lib`,
|
|
`/usr/sbin` → `bin`, all RELATIVE links. It also creates `/usr/{bin,lib}`,
|
|
`/etc`, `/dev`, `/proc`, `/sys`, `/run`, `/tmp` and `/var/tmp` (1777),
|
|
`/var`, `/root` (0700) and `/home`.
|
|
- It creates only what's missing, of any type.
|
|
- `gl_init_sys` runs it before writing the index. The index marks `-n` as
|
|
complete, so a second `syspkg -n` is refused: `Cannot create
|
|
/glacier/sys: already exists`.
|
|
- It takes a root prefix; `gl_init_sys_at` / `syspkg -R DIR -n` use it.
|
|
|
|
**Dependencies and conflicts** (ROADMAP 2.3; `src/deps.c`, built into
|
|
`libglacier_istoreutils.so`; rules in `deps.h`):
|
|
- `gl_check_plan(live, adds, n, "repo::name" removes, n)` runs before the
|
|
prompt (`check_plan` in `gpkg_common.lua`). `gl_check_txn(stage)`
|
|
compares stage with live, and `gl_commit_transaction` calls it itself
|
|
(`GL_TXN_ERR_DEPS = 9`). A live context runs the plan check inside
|
|
`gl_install_pkg`/`gl_remove_pkg` (`GL_INSTALL_ERR_DEPS = 8`,
|
|
`GL_REMOVE_ERR_DEPS = 4`). So no library caller can skip it.
|
|
- It reads each version dir's `.glacier-manifest`; a version without one
|
|
has no deps. Only what the transaction changes is checked.
|
|
- `-x` of a name installed in several repos is only checked at commit
|
|
(its repo is chosen after the prompt).
|
|
- Claude's own calls, not reviewed by the user: conflicts only apply
|
|
within one scope; only what a transaction changes is checked.
|
|
|
|
**Strict link check** (`check_links` in deps.c; `ctx->strict`): system
|
|
scope only, for this transaction's packages, mirrors `gl_link_pkg`: a
|
|
foreign file/symlink at a file's destination (unless `-O` and no earlier
|
|
`.displaced` copy), a non-directory where a directory goes, or a
|
|
directory over another package's symlink → `[x] Refusing to overwrite
|
|
non-glacier path: P` / `Refusing to merge directory ...`. A foreign
|
|
symlink where a directory goes (`usr/sbin -> bin`) is followed, as the
|
|
link pass does.
|
|
|
|
**File collisions** (ROADMAP 2.6, `check_collisions` in deps.c): two
|
|
packages whose entries land on the same path (by `dest_of`, which mirrors
|
|
`gl_link_pkg`) are refused in every scope, before the prompt and at
|
|
commit, when either is in the transaction: `[x] extra::a and extra::b
|
|
both ship /glacier/sys/links/bin/x.` Sharing a directory is fine; a file
|
|
or symlink where the other has a directory is a collision.
|
|
|
|
**Signatures** (REPO_DESIGN section 7; `gpkg_common.lua`): in system
|
|
scope, a package whose LIBRARY-read manifest (`gl_read_pkg_manifest`,
|
|
never `read_manifest_fields`) says base needs `FILE.sha256.sig` over the
|
|
exact sidecar bytes verified, by the key with that keynum among those
|
|
`/etc/glacier/repos.conf` lists for base (`gl_conf_repo_keys`). Else
|
|
`[x] base::NAME is not signed by a key trusted for base.` before the
|
|
prompt. repos.conf and keys (and all ancestors, real paths too) must be
|
|
root-owned, not group/world-writable. Under `-R`, the HOST's keyring
|
|
for this check; `-y`/`-f`/`-u` under `-R` use T's (inside T only).
|
|
No repos.conf = nothing trusted. gpkg / other repos: sidecar only.
|
|
|
|
**Repository indexes** (REPO_DESIGN sections 4-5; `M.refresh` in
|
|
`gpkg_common.lua`): `syspkg -y` / `gpkg -y` fetch `URL/repo.idx` + `.sig`
|
|
for every repos.conf entry with a url, check the signature over the
|
|
fetched bytes with a key listed for that repo, `gl_index_parse` the
|
|
same bytes, refuse SERIAL below the accepted one (or equal with other
|
|
bytes) and an expired VALID_UNTIL, then rename into
|
|
`<base>/repos/REPO.idx` (+ `.sig`). Any failure keeps the accepted
|
|
copy; every repo is tried; exit 1 if any failed.
|
|
|
|
**`-f` / `-u`** (REPO_DESIGN section 6; `M.fetch`/`M.upgrade`):
|
|
refresh first (any failure refuses), pick each name from the first
|
|
repo serving it, add missing deps (auto) and upgrade too-old ones,
|
|
download into `<base>/cache/REPO` (hash and SIZE from the index), copy
|
|
privately with the index's hash, require the manifest to equal the
|
|
index entry (`[x] extra::x 1.0.0 does not match its index.`), then the
|
|
`-l` path (`install_copies`): summary with `(dependency)`, 2.3 checks,
|
|
prompt, staged transaction, `gl_set_pkg_auto`, backups, cache files
|
|
deleted after commit. A base package fetched this way needs no `.sig`
|
|
(the base index vouches). `-f` of an installed other version needs
|
|
`-O`; `-u` and dependency upgrades don't. gpkg never fetches from base.
|
|
|
|
**Package symlinks** are extracted verbatim and linked like files.
|
|
Hardlink, device and fifo entries are skipped. `add_to_archive` (used by
|
|
backups and `gstore -b`) archives symlinks.
|
|
|
|
## CLIs (glacier-ng)
|
|
|
|
| Tool | Flags | Runs as |
|
|
|---|---|---|
|
|
| `gpkg` | `-l FILE...` `-f NAME...` `-u [NAME...]` `-x NAME...` `-y` `-O` `-S` `-V` `-h` `-v` | the user; per-user scope only, NEVER constructs `GL_SCOPE_SYS` |
|
|
| `syspkg` | `-n` `-l FILE...` `-f NAME...` `-u [NAME...]` `-x NAME...` `-y` `-O` `-S` `-V` `-R DIR` `--strict` `-h` `-v` | root via doas; `-h`/`-v` without root |
|
|
| `gstore` | `-n [UID]` `-b UID OUTFILE` `-x UID` `-s` (refuses: not implemented) `-V` `-h` `-v` | `-n` needs root (direct, or `doas gstore -n` for yourself) |
|
|
| `glacier-bootstrap` | `-d PKGDIR TEMPLATE TARGET` `-n` `-t` `-V` `-h` `-v` | the user; borrows root only for `doas syspkg` / `doas chroot` |
|
|
|
|
- **gpkg:** modifiers `-O`/`-S`/`-V` are applied in a pre-pass, so their
|
|
position doesn't matter (`-lO` == `-Ol`). It finds its user via
|
|
`/proc/self/loginuid`, falling back to `getuid()`. Unknown flags are
|
|
refused (`[x] Unknown option: -R`); gpkg has no `-R`/`--strict`.
|
|
- **Endings (both CLIs):** `[✓] Completed with no errors.` (exit 0), or
|
|
after link refusals `[x] N paths were not linked.` (exit 1;
|
|
`report_done` in gpkg_common.lua). Refusals before the prompt exit 1
|
|
with only the `[x]` lines.
|
|
- **syspkg, `-n`:** creates the tree plus `/glacier/sys`.
|
|
- **syspkg, `-R DIR` / `--root DIR` / `--root=DIR`** (last in a bundle:
|
|
`-lR DIR`): operate on the system under DIR, through the `*_at`
|
|
contexts. Refused first by `gl_check_sysroot`: DIR and every ancestor
|
|
must be root-owned, non-symlink directories, not group/world-writable
|
|
(`/tmp/...` is refused). Implies `--strict`.
|
|
- **syspkg, `--strict`:** anything the link pass would refuse fails the
|
|
whole transaction before the prompt (and at commit, `ctx->strict`);
|
|
`-O` still displaces.
|
|
- **syspkg as non-root:** `-n`/`-l`/`-x` are refused after argument
|
|
parsing with `Cannot open system index: Permission denied`.
|
|
- **gstore, `-n` without a UID, under doas:** acts for the invoking user
|
|
(loginuid, then `DOAS_USER`, then getuid). The doas rule
|
|
`permit nopass :wheel as root cmd /path/gstore args -n` is the
|
|
boundary that stops a user naming another UID.
|
|
- **Shared code:** `lib/gpkg_common.lua` (`M.install`, `M.remove`,
|
|
verification, replacement, backups).
|
|
- **The `-x` summary** comes from the live index; a name that isn't
|
|
installed fails before the prompt.
|
|
- **Hardened loading** (syspkg and gstore): `#!/usr/bin/luajit -E`,
|
|
`package.path` and every `ffi.load` pinned to their own `realpath`, and
|
|
when euid is 0, every loaded `.lua`/`.so`, and every ancestor of the
|
|
script's realpath AND of each loaded file's realpath, must be
|
|
root-owned and not group/world-writable (bug 29). The preamble is
|
|
DUPLICATED in both files on purpose: change one, change both.
|
|
- **gpkg** (2026-10-06) loads `lib/` from its own realpath, like
|
|
glacier-bootstrap (no ownership checks; it's unprivileged), so an
|
|
installed `/usr/bin/gpkg` works from any directory.
|
|
- **glacier-bootstrap** (GLACIER_BOOTSTRAP section 10): parse →
|
|
verify (copies each `PKGDIR/NAME-VER.gpkg` into a private `mktemp -d`
|
|
while hashing; sidecar + pin; writes the copy's sidecar from that hash)
|
|
→ resolve (reads manifests of the COPIES via `gl_read_pkg_manifest`:
|
|
name/version match, deps listed at >= version, conflicts) → `syspkg -R
|
|
T -n` (skipped if T is initialised with nothing installed) → `syspkg -R
|
|
T -l COPIES` (one strict transaction; its prompt passes through) →
|
|
checks T's index lists every package (syspkg exits 0 when declined) →
|
|
`syspkg -R T -y` if T's repos.conf has a url → `-t` smoke test.
|
|
Never writes into T itself. Exit: 1 usage/target, 2 template,
|
|
3 verify, 4 resolve, 5 install, 6 smoke, 7 repositories. Loads its `lib/`
|
|
from its realpath, no root checks. No `-r` / `--pin` yet.
|
|
- **Declarations:** `lib/glacier_cdef.lua` is the single source of truth
|
|
for every `ffi.cdef`. `tests/glacier_ffi.lua` is only a wrapper around
|
|
it. Declaration ORDER matters: a declaration using `gl_context_t` must
|
|
come after its typedef.
|
|
|
|
## Build tooling
|
|
|
|
- **gbuild** (bash, the only copy is `glacier-ng/gbuild`):
|
|
- `-n -V -g -c -d -m -D -C -b -i -o -p -j -F -t -x -u -r -k -s`
|
|
(`-k KEY.sec`: sign the sidecar with gsign, `$GSIGN_BIN` / build/bin
|
|
/ PATH, key tried before building);
|
|
`-v`/`--version` prints the version, `--verbose` turns on verbose
|
|
output.
|
|
- Autodetects the build system, with autotools checked BEFORE a bare
|
|
Makefile.
|
|
- Writes `NAME-VER.gpkg` plus a `.sha256` sidecar, using `gsha256sum`
|
|
from `$GSHA256SUM_BIN`, else `<script dir>/build/bin`, else PATH.
|
|
- **There is NO `-P` system-profile support, and gworld has no
|
|
`SYS_PROFILE` field** (ROADMAP 4.1). The target libc is decided by
|
|
where gbuild runs: Alpine for musl.
|
|
- **gworld** (Lua): batch orchestrator, reads `recipes/*.recipe`, sorts
|
|
topologically by `DEPS`, runs gbuild per package.
|
|
- **`~/Projects/everest`:** the gbuild command for each package lives in
|
|
`scripts/container.sh`; `everest.sh` runs it inside an Alpine image
|
|
(`Dockerfile.build`).
|
|
- **Kernel version:** linux `6.12.112`; plain `v6.12` doesn't build with
|
|
GCC 15.
|
|
- **Everest's image IS built through glacier** (2026-10-06): `container.sh
|
|
initramfs` copies glacier-ng to a root-owned `/opt/glacier-ng` (with
|
|
the musl-built `.so`s; syspkg's trust checks need that) and runs
|
|
`glacier-bootstrap -t -d /everest/pkgs templates/minimal.template
|
|
/target` (root-owned: not under `/tmp`). T's links name RUNTIME paths,
|
|
so anything reading through them from the build host must resolve
|
|
inside T (vmlinuz is read with `chroot T cat`). Only `root` exists in
|
|
the shipped `/etc/passwd`.
|
|
- gbuild ends with the package's contents list. Its "To install with
|
|
gpkg: gpkg -l ..." hint was removed 2026-10-06 at the user's request.
|
|
|
|
## libglacier-ng build
|
|
|
|
- **Separate `.so` per module** (`libglacier_log`, `_pkg`, `_istoreutils`,
|
|
`_transaction`, `_verify`; `_dag` was removed 2026-10-06, unused). This split is deliberate: a
|
|
third-party program links only what it needs.
|
|
- **The combined `libglacier-ng.so`** (`make libglacier-ng`) exists only
|
|
for the FFI tests.
|
|
- **`libglacier_verify.so`** is private: no public header.
|
|
- **Tools** in `build/bin`: `gsha256sum`, `gsign`, `grepo` (gsign and
|
|
grepo compile the sources they need in, no rpath). Maintainer tools,
|
|
later "glacier-tools": Everest's libglacier-ng package drops
|
|
`usr/bin`.
|
|
- **Warnings:** `config.mk` builds with `-Werror`. `-Wno-format-truncation`
|
|
is set because that warning is noise; `-Wstringop-truncation` stays
|
|
fatal.
|
|
- **Makefile quirk:** `make log` builds only `log.o`; the shared object
|
|
comes from `make log_test` (or plain `make`).
|
|
|
|
## Testing
|
|
|
|
Run everything in throwaway containers. Several suites write to `/usr`,
|
|
`/etc` and `/glacier`.
|
|
|
|
**FFI suite** (`glacier-ng/tests/*.lua`; 11 files, all passing):
|
|
```sh
|
|
docker run --rm --cap-add SYS_ADMIN -v ~/Projects/glacier-ng:/src:ro \
|
|
-v ~/Projects/libglacier-ng:/libsrc:ro alpine:latest sh -c '
|
|
apk add --no-cache build-base luajit luajit-dev libarchive-dev libconfig-dev linux-headers tar >/dev/null
|
|
cp -a /libsrc /build && cd /build && make clean && make && make libglacier-ng
|
|
cp -a /src /g && cd /g/tests && GLACIER_TEST_CONFIRM=yes \
|
|
LIBGLACIER_SO=/build/build/lib/shared/libglacier-ng.so luajit run_all.lua'
|
|
```
|
|
GNU `tar` is required: the hostile-archive fixtures use `tar --append`.
|
|
|
|
**Shell suites** (`glacier-ng/tests/*.sh`):
|
|
|
|
| Suite | Image |
|
|
|---|---|
|
|
| `sign_alpine.sh` | `alpine:latest` with `/libsrc` and `/t.sh` mounted |
|
|
| `syspkg_alpine.sh`, `overwrite_alpine.sh` | `alpine:latest` with `/src`, `/libsrc` and `/t.sh` mounted (see each header); `overwrite_alpine.sh` runs both as-is and with `--tmpfs /glacier:exec` |
|
|
| `verify_alpine.sh`, `user_layout_alpine.sh`, `upgrade_alpine.sh`, `sys_perms_alpine.sh`, `deps_alpine.sh`, `sysroot_alpine.sh`, `root_alpine.sh`, `bootstrap_alpine.sh`, `installed_alpine.sh`, `keyring_alpine.sh`, `repo_alpine.sh`, `audit_alpine.sh`, `fetch_alpine.sh` | `glacier-alpine_root` |
|
|
|
|
Build the `glacier-alpine_root` image with:
|
|
```sh
|
|
cd ~/Projects/glacier-ng && docker build -f Dockerfile.alpine_root \
|
|
--build-context libglacier=../libglacier-ng -t glacier-alpine_root .
|
|
docker run --rm -v "$PWD/tests/NAME.sh:/t.sh:ro" glacier-alpine_root sh /t.sh
|
|
```
|
|
|
|
`glacier-ng/lib/glacier/` does not exist in the tree (the `.so`s are
|
|
built per image); syspkg_alpine/overwrite_alpine `mkdir -p` it
|
|
(2026-10-07). If Docker fails with `veth ... operation not supported`,
|
|
the host kernel was upgraded without a reboot (modules missing): use
|
|
`docker build --network host` and `docker run --network=host` (or
|
|
`--network none` where nothing is fetched) until the user reboots.
|
|
Mutating a Lua file syspkg loads: mount it elsewhere and `cp` it over
|
|
`/app/lib/...` inside the container (a bind mount is user-owned, so
|
|
syspkg's trust check refuses it).
|
|
|
|
**Demo packages:** `glacier-ng/tests/mkpkg [-o DIR] [-k KEY.sec]
|
|
[-d NAME:VER]... [-c NAME:VER]... NAME VERSION REPO` writes a one-script
|
|
`.gpkg` plus its sidecar (`/app/tests/mkpkg` in the images); a base one
|
|
needs `-k /root/glacier-test.sec` (the images' test key) for syspkg. The user plays with changes
|
|
in `./run_docker.sh Dockerfile.alpine_root` using it; for an image,
|
|
`mkdir -p /mnt/img && ./syspkg -R /mnt/img -n`.
|
|
|
|
**Mutation checks, how:** `cp -a` libglacier-ng (and glacier-ng, for Lua
|
|
changes) into the scratchpad, break the defence there, then
|
|
`docker build -f Dockerfile.alpine_root --build-context libglacier=COPY
|
|
-t glacier-mut .` from the (copied) glacier-ng and run the suite against
|
|
`glacier-mut`; delete the image after. Docker tags must be lowercase. A
|
|
mutation that doesn't compile under `-Werror` proves nothing: rewrite it
|
|
so it compiles. Watch for checks that pass for the wrong reason (e.g. a
|
|
refusal caused by a missing directory instead of the defence). In shell
|
|
suites, `sh -c '... $OUT ...'` sees an EMPTY `$OUT` (not exported), so a
|
|
negative check written that way passes vacuously; use a function
|
|
(`lacks` in bootstrap_alpine.sh). Found 2026-10-06 by a surviving
|
|
mutation. The host shell is zsh: an unquoted `$VAR` holding `-v a:b`
|
|
is NOT word-split, so a docker command built that way silently runs
|
|
without the mount (a mutant "survived" that way, 2026-10-06). To
|
|
mutate a Lua CLI without rebuilding, bind-mount the mutated file over
|
|
`/app/NAME` (`-v $PWD/mut:/app/glacier-bootstrap:ro`).
|
|
|
|
**Full regression** = the FFI suite + `syspkg_alpine.sh` +
|
|
`overwrite_alpine.sh` (both ways) + `sign_alpine.sh` + the thirteen `glacier-alpine_root`
|
|
suites, all in parallel; the two output-based suites (`syspkg_alpine`,
|
|
`user_layout_alpine`) have no PASS lines, so grep their output for
|
|
`Unknown|traceback|failed|GL_` instead (only `--bogus` is expected).
|
|
|
|
**Interactive images:** `./run_docker.sh Dockerfile.{alpine,alpine_root,arch,debian}`.
|
|
`run_docker.sh` passes the `libglacier` build context. `alpine`, `arch`
|
|
and `debian` default to user `glacier` (uid 1000, in wheel); `alpine_root`
|
|
defaults to root. All of them have:
|
|
- root-owned `/app`;
|
|
- no `/glacier` until `gstore -n` / `syspkg -n`;
|
|
- the doas rule plus `permit nopass glacier`;
|
|
- `/etc/profile.d/glacier.sh` putting `/glacier/usr/<uid>/links/bin` then
|
|
`/glacier/sys/links/bin` on `PATH` (and `lib` on `LD_LIBRARY_PATH`)
|
|
unconditionally, since `/glacier` only appears after login (fixed
|
|
2026-10-05: it had no system links and checked `-d` at login);
|
|
- `glacier.conf` installed.
|
|
|
|
**Boot test:** `cd ~/Projects/everest && ./everest.sh boot` boots QEMU
|
|
(KVM). Press Enter for a root shell; `poweroff -f` to leave. To test as
|
|
a non-root user, `adduser -D -H -h /tmp tester` at runtime, then
|
|
`su - tester -c ...` (nothing shipped has a user besides root).
|
|
Everest's `/etc/profile` sets `PATH` to per-user links, then
|
|
`/glacier/sys/links/bin`, then `/usr/bin` (2026-10-05). Backups of
|
|
everest-bootscripts files go NEXT TO `etc/`, never inside it: the package
|
|
copies `etc/` wholesale. After changing a package:
|
|
`./everest.sh build PKG && ./everest.sh initramfs`, then boot.
|
|
To drive the boot non-interactively, fork QEMU on a pty from Python
|
|
(`pty.fork()`), wait for `Please press Enter`, send lines and match the
|
|
`# `/`$ ` prompt (no `expect` on the host).
|
|
|
|
## Bug history: so these don't get reintroduced
|
|
|
|
All fixed in the current tree.
|
|
|
|
1. `gl_abort_transaction` never released the lock (wrong variable). Now
|
|
`open(O_CREAT)` + `flock()`, and lock files are never unlinked.
|
|
2. No `EXDEV` check before renaming. Commit `stat()`s both sides and
|
|
returns `GL_TXN_ERR_XDEV`.
|
|
3. Segfault on install: the `transaction` target didn't link
|
|
`istoreutils`/`log`.
|
|
4. Stale links post-commit: `gl_link_pkg` ran during staged installs,
|
|
embedding stage paths. It now runs only for LIVE contexts, and links
|
|
regenerate post-commit.
|
|
5. Dangling links were never cleaned on removal → `prune_stale_links`.
|
|
6. `GL_TXN_ERR_RENAME` after the first commit: a leftover `old/`. It's now
|
|
wiped first.
|
|
7. `-Wstringop-truncation`: `strncpy(..., PATH_MAX - 1)` could leave
|
|
strings unterminated. Use `snprintf(dst, sizeof dst, "%s", ...)`.
|
|
8. gbuild detected a bare `Makefile` before `configure`. The fix had been
|
|
lost from the tree and was re-applied 2026-10-04.
|
|
9. The index had no version field; it's now `repo::pkgname::version`.
|
|
10. [The code this was about is NOT in the tree: ROADMAP 4.1.] A recipe
|
|
field silently never parsed and fell back to a plausible default.
|
|
Remember the class.
|
|
11. `gsha256sum.c` included `sha256.c` instead of `.h`.
|
|
12. gbuild told users to run `gpkg -f` (a stub) instead of `gpkg -l`.
|
|
13. `gpkg -f`'s stub referenced an undefined `uid`.
|
|
14. Path traversal from manifests/archives: `PKG_NAME`/`PKG_REPO` must
|
|
pass `gl_valid_pkg_ident`, and `..`/absolute entries refuse the
|
|
package. Finding 1 (self-declared `base`) stays open until signing.
|
|
15. Stage seeding and `gl_rmdir_recursive` skipped ALL dot-entries:
|
|
dotfiles vanished, and `old/` couldn't be removed (bug 6 again).
|
|
16. Per-user transactions could never work for a real user (shared
|
|
writable parents needed). Fixed by the per-uid layout. Found alongside:
|
|
- `gstore -n` never created parent directories;
|
|
- the stage was wiped before the lock was taken;
|
|
- cross-filesystem seeding was reported as `MKDIR`;
|
|
- `gl_restore_istore` doubled the uid and accepted `..` entries;
|
|
- gstore loaded the combined `.so` via `LIBGLACIER_SO` from the
|
|
environment.
|
|
17. Package symlinks were silently dropped at extraction, and only regular
|
|
files were linked, so musl's loader was never installed. Alpine-based
|
|
tests passed only because Alpine provides its own loader.
|
|
18. `add_to_archive` (`gstore -b`, backups) silently dropped symlinks.
|
|
19. gpkg acted on `-l` as soon as it parsed it, so `-lO` ignored its `O`.
|
|
20. `tests/glacier_ffi.lua` duplicated the cdefs and had drifted (it was
|
|
missing `GL_INSTALL_ERR_SCOPE`). It's now a wrapper.
|
|
21. Link conflicts were re-reported on every system commit, for every
|
|
package (it now uses the `.glacier-fresh` marker).
|
|
22. A failed extraction left a half-made version directory, so in a live
|
|
context every retry reported `ALREADY_INSTALLED`.
|
|
23. Backups written in the same second were pruned in the wrong order
|
|
(whole-second mtimes).
|
|
24. System scope was root 0700 (index, store, links, every version dir),
|
|
so no non-root user could run anything syspkg installed, through
|
|
`/usr` or `/glacier/sys/links`. Found 2026-10-05. Also fixed: every
|
|
mode glacier creates was filtered by the caller's umask (`gl_mkdirp`,
|
|
seeding, extracted files), so syspkg under umask 077 made even a
|
|
0755 store private. `gl_mkdirp`/`gl_mkdir_exact` now chmod what they
|
|
create, and extracted entries get `archive mode & 07755` explicitly
|
|
(never group/world-writable). Test: `tests/sys_perms_alpine.sh`
|
|
(mutation-checked). No migration: a store made before the fix keeps
|
|
its 0700 dirs until reinstalled.
|
|
25. `gl_rebuild_index` `fopen(name, "w")`'d the new index file. Index
|
|
names have one-second resolution and seeding HARDLINKS live's index
|
|
files into the stage, so a staged install in the same second as the
|
|
last commit rewrote the LIVE index in place; an aborted transaction
|
|
then left live listing the stage's packages (and history files were
|
|
overwritten). It now writes to `tmpfile()` and only `rename()`s onto
|
|
the name. Found 2026-10-05 by `deps_alpine.sh`; regression test
|
|
`tests/test_index_abort.lua` (mutation-checked). Rule: never open a
|
|
seeded file for writing; replace it by rename.
|
|
26. Live-context removal (`gl_unlink_pkg`) `unlink()`ed the destination
|
|
of every regular file in the package WITHOUT `is_glacier_symlink`:
|
|
a foreign file the package had been refused to replace was deleted,
|
|
and the package's own symlink entries were never unlinked. The CLIs
|
|
always stage, so only library callers were exposed. Removed;
|
|
live removal now prunes like a commit (`gl_relink_store`). Found
|
|
2026-10-05; `tests/test_live_remove_foreign.lua` (fails on the old
|
|
code).
|
|
27. gpkg's `handle_flag` had no `else`: an unknown flag (`-R`) was
|
|
silently ignored and the NEXT word reported as the unknown option.
|
|
Now `[x] Unknown option: -R` (capitalized, as syspkg). 2026-10-05.
|
|
28. gbuild's `clone_source` printed its "Shallow clone failed" warning
|
|
(and the full clone's/checkout's output) to stdout, which
|
|
`SOURCE_DIR="$(clone_source)"` captured, so building from a commit
|
|
hash (`-r <sha>`) always failed. Those now go to stderr. Found
|
|
2026-10-06 building luajit.
|
|
29. syspkg's and gstore's trust checks walked only the ancestors of the
|
|
script's own realpath. Installed, `lib/glacier -> /usr/lib/glacier`
|
|
and the `.so`s link into libglacier-ng's store dir, so a non-root-
|
|
owned directory there was never checked: root loaded `.so`s from it.
|
|
Both now also check every ancestor of each loaded file's realpath.
|
|
Found 2026-10-06 by `tests/installed_alpine.sh` (both mutations
|
|
caught).
|
|
|
|
30. A package could ship top-level `files/.glacier-*` entries. As a
|
|
symlink, `.glacier-fresh` made syspkg truncate its target as root
|
|
(an unsigned extra package emptied `/root/victim`, reproduced);
|
|
`.glacier-overwrite` self-granted `-O`. Reserved names refused,
|
|
marker writes `O_NOFOLLOW`. 2026-10-06 audit.
|
|
31. `gl_rmdir_recursive` (lstat, then opendir by path) could be steered
|
|
into a symlink's target when root deleted a user's tree (`gstore
|
|
-x`). Now fd-relative. Race test in audit_alpine.sh.
|
|
32. `add_to_archive` (same class, `gstore -b`). Now fd-relative; no test
|
|
fails on the old code (window too narrow; review only).
|
|
33. Manifests could `@include` any file (read as root); a dependency
|
|
entry without a name cut the list short. Both refused.
|
|
34. gpkg_common's `read_manifest_fields` (tar + pattern match) could
|
|
disagree with the library: a decoy comment hid a replacement. Now
|
|
`manifest_of` (`gl_read_pkg_manifest`).
|
|
35. Format strings: gstore's `argv[0]`/UID and gpkg `-u`'s operand
|
|
reached `lg_printf` as the format; `%d` was given Lua numbers;
|
|
gstore UIDs wrapped past uid_t; `gl_abort_transaction` wiped a stage
|
|
without holding the lock. All fixed (`lg_printf` now has a printf
|
|
format attribute).
|
|
|
|
## Known gaps (not bugs, not yet scheduled)
|
|
|
|
- No migration for stores made before bug 24's fix (0700 dirs stay).
|
|
- Extracted entries keep setuid/setgid bits (`& 07755`), as before.
|
|
- No size limit on extraction (a decompression bomb fills the disk).
|
|
- Bug 32's fix has no test that fails on the old code.
|
|
- Hash verification is still CLI-only (ROADMAP 3.2); everything else
|
|
added 2026-10-05 is enforced in the library.
|
|
- A dependency refusal for `-x` of a name installed in several repos
|
|
comes at commit, after the prompt (its repo is chosen after it).
|
|
|
|
## Backups of glacier-ng / everest files (no git there)
|
|
|
|
The rule stands (back up before editing: `file.old`, then `.old2`...).
|
|
All of glacier-ng's backups were deleted 2026-10-06 at the user's
|
|
request. The 2026-10-07 session made new ones: `lib/glacier_cdef.lua.old`,
|
|
`lib/gpkg_common.lua.old`, `gpkg.old`, `syspkg.old`,
|
|
`tests/{audit,overwrite,syspkg,repo}_alpine.sh.old`,
|
|
`lib/gpkg_common.lua.old2`/`.old3`, `tests/repo_alpine.sh.old2`/`.old3`,
|
|
`tests/fetch_alpine.sh.old`, `lib/gpkg_common.lua.old4`,
|
|
`glacier-bootstrap.old`, `tests/bootstrap_alpine.sh.old`,
|
|
`tests/repo_alpine.sh.old4`, `tests/fetch_alpine.sh.old2`; Everest:
|
|
`scripts/container.sh.old4`, `everest.sh.old3`, `README.md.old3`,
|
|
`templates/minimal.template.old2`, `pkgsrc/everest-bootscripts/glacier.old`.
|
|
Everest still has its earlier ones (`everest.sh.old`/`.old2`,
|
|
`scripts/container.sh.old`/`.old2`/`.old3`, `README.md.old`/`.old2`,
|
|
`Dockerfile.build.old`, `templates/minimal.template.old`,
|
|
`pkgsrc/everest-bootscripts/profile.old`). Leave them unless the user
|
|
says to clean up.
|
|
|
|
## Conventions
|
|
|
|
- **C style:** tabs, but the codebase is inconsistent; match the
|
|
surrounding function.
|
|
- **Paths:** `PATH_MAX` stack buffers plus `snprintf` for path
|
|
construction.
|
|
- **Lua requires:** gpkg uses bare CWD-relative requires.
|
|
syspkg and gstore pin their paths (see CLIs).
|
|
- **Lua numbers in C varargs:** a Lua number reaches C varargs as a
|
|
double, so never pass one to `lg_printf("%d")`. Use `%s` with
|
|
`tostring()`, or `ffi.new("int", n)`.
|
|
- **Shell commands from Lua:** always `shell_quote` interpolated paths
|
|
(syspkg runs as root).
|
|
- **Verify by running.** Several real bugs here were found only by
|
|
executing something.
|