10/14
This commit is contained in:
1 parent
ee4bcc3b77
commit
ae54f74070
53 files changed
+8916
-1424
No files matched your search
@@ -3,7 +3,8 @@
|
||||
Context for Claude Code sessions on libglacier-ng: the package manager
|
||||
for Everest Linux. Kept as a handoff, so a fresh session starts with full
|
||||
continuity instead of re-deriving everything. Last full rewrite:
|
||||
2026-10-04, at the end of a long session. **Update it as you go.**
|
||||
2026-10-04; brought fully up to date at the end of the 2026-10-07
|
||||
session. **Update it as you go.**
|
||||
|
||||
## Start here
|
||||
|
||||
@@ -15,15 +16,231 @@ continuity instead of re-deriving everything. Last full rewrite:
|
||||
- `SECURITY_DESIGN.txt`: integrity verification, scope separation,
|
||||
refusal matrix, syspkg's privilege model.
|
||||
- `PACKAGE_SCOPES.txt`: layout, link routing, `/etc` config handling.
|
||||
- `IMAGE_DESIGN.txt`: PROPOSAL for system images (bootstrap OUTPUT,
|
||||
the sysroot library change). Open decisions are in section 9.
|
||||
- `GLACIER_BOOTSTRAP.txt`: PLAN for `glacier-bootstrap`. Prerequisites
|
||||
are in section 7, decisions in section 9 (9c and 9d are decided).
|
||||
- `IMAGE_DESIGN.txt`: system images (bootstrap OUTPUT). Sections 3
|
||||
(sysroot) and 4 (`syspkg --root`) are IMPLEMENTED (milestones 1-2);
|
||||
the rest is still a proposal. Open decisions are in section 9.
|
||||
- `GLACIER_BOOTSTRAP.txt`: `glacier-bootstrap`. Local mode is
|
||||
IMPLEMENTED (ROADMAP 1.3; section 10 says how it differs from the
|
||||
plan); remote mode and `--pin` are still a plan. Open: 9a, 9f.
|
||||
- `REPO_DESIGN.txt`: remote repos + signing (ROADMAP 3.1). Sections
|
||||
1 and 10 = the user's decisions (TLS library still open); steps
|
||||
1-4 of section 9 DONE (see "3.1 status"); sections 7, 5 and 6
|
||||
describe steps 2, 3 and 4 as implemented.
|
||||
- `SECURITY_REVIEW_2026-10-06.md`: the full audit (findings, fixes,
|
||||
what was removed, what's still open).
|
||||
- `SECURITY_REVIEW_2026-07-16.md`: older review findings.
|
||||
|
||||
Nothing from the 2026-10-04 session is committed. libglacier-ng is on
|
||||
branch `dev` with a large uncommitted diff (`src/`, the docs). Commit only
|
||||
when the user asks.
|
||||
Nothing from the 2026-10-04 through 2026-10-07 sessions is
|
||||
committed. The user said "do not commit yet" (2026-10-05), and on
|
||||
2026-10-06 that their git server is down: they will commit themselves
|
||||
once it's back. libglacier-ng is on branch
|
||||
`dev` with a large uncommitted diff (`src/`, the docs, `build/`). Commit
|
||||
only when the user asks; when they do, propose a few logical commits.
|
||||
|
||||
2026-10-05 session: system-scope permissions fix (bug 24), ROADMAP 2.3
|
||||
dependencies (`src/deps.c`), index corruption on abort (bug 25), ROADMAP
|
||||
1.1 sysroot, live removal deleting foreign files (bug 26), ROADMAP 1.2
|
||||
(`syspkg -R`, `--strict`) + 2.6 (file collisions), gpkg ignoring unknown
|
||||
flags (bug 27).
|
||||
|
||||
2026-10-06 session: ROADMAP 1.3 `glacier-bootstrap` local mode, and
|
||||
Everest's initramfs switched to it (boots in QEMU). Tidy-up (gbuild's
|
||||
hint removed). ROADMAP 1.4 DONE (packages for glacier itself; details in
|
||||
ROADMAP 1.4 and section 7): full regression green, new
|
||||
`tests/installed_alpine.sh` for the installed layout, which found bug 29
|
||||
(fixed, mutation-checked, Everest rebuilt and booted with the fix).
|
||||
|
||||
The installer (1.5) is DEFERRED by the user: manual installation is
|
||||
one of Everest's main features. Then ROADMAP 3.1 was designed with the
|
||||
user (REPO_DESIGN.txt, every decision taken; only the TLS library for
|
||||
curl in base is open) and steps 1, 2 and (that evening, after the
|
||||
laptop died mid-rebuild) 3 of its section 9 were done.
|
||||
|
||||
2026-10-07 session: ROADMAP 3.1 step 4 (`-f`, `-u`, dependency
|
||||
closure, the cache; see "3.1 status"). Help text for -f/-u/-O
|
||||
shortened at the user's request. Refresh output (user's wording): `[i] Syncing
|
||||
all repositories...`, then `[✓] All repositories synced successfully.`
|
||||
if every repo succeeded; the per-repo success/up-to-date lines only
|
||||
with `-V`; errors always (repo_alpine.sh checks both ways).
|
||||
Then (same day) glacier-bootstrap configures repositories (user: it must
|
||||
leave a semi-functional system, pacstrap-like): see "Bootstrap
|
||||
repositories" below.
|
||||
|
||||
2026-10-09: help text. `gsign`, `grepo`, `gsha256sum` (C, `getopt_long`,
|
||||
static `help()` in each tool's own file), `glacier-bootstrap` and `gworld`
|
||||
now have `-h/--help` and `-v/--version` (`v4.0.0`) in gpkg's format
|
||||
(`%-25s` `{-x --long} ARG` lines, GPL footer), with long forms for every
|
||||
flag. `-h` goes to stdout, exit 0; usage errors stay one usage block on
|
||||
stderr, exit 1. repo_alpine.sh checks the help of grepo/gsign/gsha256sum.
|
||||
Backups: `glacier-bootstrap.old2`, `gworld.old`, `tests/repo_alpine.sh.old5`.
|
||||
|
||||
### Bootstrap repositories (2026-10-07)
|
||||
|
||||
The user chose: the config comes from a NEW base package
|
||||
(`everest-repos` 1.0.0: `etc/glacier/repos.conf` + `keys/everest-2026.pub`,
|
||||
source `~/Projects/everest/pkgsrc/everest-repos`; everest-bootscripts
|
||||
1.3.0 no longer ships `etc/glacier`, its old copy is
|
||||
`pkgsrc/everest-bootscripts/glacier.old`); bootstrap runs `syspkg -R T
|
||||
-y` after installing and a failed fetch is an error (exit 7, no
|
||||
"Bootstrapped"); under `-R`, `-y`/`-f`/`-u` use T's OWN keyring
|
||||
(`keyring(sysroot)` in gpkg_common.lua; `require_trusted` also refuses a
|
||||
file whose realpath leaves T). `-R -l` signatures stay the host's
|
||||
(`HOST_KEYRING`). No url in T: `[!] No remote repositories are
|
||||
configured in T.`, exit 0 (Everest today: base isn't published).
|
||||
Copying the host's repos.conf (pacstrap's mirrorlist) left out for now.
|
||||
The user said everything a template lists for base also goes in the
|
||||
base repo (the published repo = the template's base packages); no url
|
||||
yet. Everest rebuilt (bootscripts 1.3.0, everest-repos, glacier-ng 4.0.0
|
||||
in place), initramfs remade, booted: `/etc/glacier` from everest-repos,
|
||||
12 packages.
|
||||
|
||||
### Audit and cleanup (2026-10-06, late)
|
||||
|
||||
At the user's request, before 3.1 step 4: a full security audit
|
||||
(`SECURITY_REVIEW_2026-10-06.md`, bugs 30-35 below), unused code removed,
|
||||
and junk cleaned up. The user's choices: delete only the `.old` backups
|
||||
and stray binaries (every prototype/note file stays: libglacier-ng's
|
||||
`src/{deb,gpkg,templ,report,test_*}.c`, `src/html`, `pkg/`, glacier-ng's
|
||||
`todo`, `tree/`, `test-*`, `pkgs/`...); libglacier-ng's build outputs are
|
||||
untracked (`.gitignore`: `/build/`; the deletions show in the next
|
||||
commit); the `.old` backup rule STAYS (see "Backups" below for current ones). New tests:
|
||||
`tests/audit_alpine.sh` (26 checks), `tests/test_abort_unlocked.lua`.
|
||||
Full regression green; Everest rebuilt and booted. Everest's own `.old`
|
||||
files were not touched (not one of the two repos).
|
||||
|
||||
### 3.1 status
|
||||
|
||||
Step 4 DONE (2026-10-07; REPO_DESIGN section 6 "As implemented"). The
|
||||
user chose: the FIRST repo (repos.conf order) serving a name wins, at
|
||||
its highest version; a dependency installed too old is upgraded in the
|
||||
same transaction (keeps its auto/explicit mark); cached downloads are
|
||||
deleted after the commit that installs them.
|
||||
- libglacier-ng: `gl_scope_cache_dir(ctx, repo, out)` (transaction.c;
|
||||
`<base>/cache/REPO`, checked like `repos` via the shared static
|
||||
`checked_subdir`); `log.c` help lists `-f`/`-u`.
|
||||
- glacier-ng `gpkg_common.lua`: `M.fetch(names, opts)`,
|
||||
`M.upgrade(names, opts)`; `load_indexes` (refresh, then read the
|
||||
accepted copies), `pick`, `close_deps`, `download` (SIZE-capped,
|
||||
hashed), `matches_index` (manifest vs index entry), `install_plan`.
|
||||
`M.install` is now `verify_all` + `install_copies(copies, original,
|
||||
opts, meta)`, shared with -f/-u (`meta`: auto, may_replace, dep,
|
||||
cache). `verify_and_copy(path, vouched)` takes the index's hash in
|
||||
place of a sidecar. syspkg `-f`/`-u` modes; gpkg `-f`/`-u` (no
|
||||
longer "not implemented").
|
||||
- New suite `tests/fetch_alpine.sh` (84 checks; 16 Lua mutants + 1 C,
|
||||
all caught except the download-level hash check, equivalent: kept as
|
||||
defence in depth). The `gpkg -u "%s%n"` format check moved there from
|
||||
audit_alpine.sh. Full regression green.
|
||||
- Everest's libglacier-ng/glacier-ng 4.0.0 were REBUILT in place
|
||||
2026-10-07 with -f/-u/-y (user: keep 4.0.0, no bump) and the
|
||||
initramfs remade; in QEMU `gpkg -f hi` as a wheel user now says
|
||||
`[x] No remote repositories are configured.` (shipped repos.conf has
|
||||
no url; no curl in base). `container.sh initramfs` now `mkdir -p`s
|
||||
`$G/lib/glacier` (gone from the tree). Not done: no published repo;
|
||||
TLS library still open. Next: step 5 (`glacier-bootstrap -r`),
|
||||
then 6 (user repos).
|
||||
|
||||
Step 3 DONE (2026-10-06, evening; REPO_DESIGN section 5 "As
|
||||
implemented"). The user chose: a refresh-only flag `-y`, the date-form
|
||||
SERIAL (YYYYMMDDNN), grepo in C.
|
||||
- libglacier-ng: `src/grepo.c` (`grepo -s KEY.sec -r REPO [-n SERIAL]
|
||||
[-e DAYS] DIR`, Makefile target in `all`, installed to bin, built from
|
||||
source like gsign); `pkg.c`: `gl_index_parse(text, len, repo, out)` /
|
||||
`gl_index_free` (strict; takes the signed BYTES; refuses NUL and '@'),
|
||||
`gl_conf_repos(path, out, len)` ("NAME\tURL\n" lines), repos.conf
|
||||
validation shared in `read_repos_conf`; `transaction.c`:
|
||||
`gl_scope_state_dir(ctx, "repos", out)` (creates/checks
|
||||
`<base>/repos`). `log.c` help lists `-y`.
|
||||
- glacier-ng: `M.refresh(opts)` in `gpkg_common.lua` (fetch file:// or
|
||||
curl https, verify with a key listed for THAT repo, parse, rollback /
|
||||
same-serial / expiry checks, rename into `<base>/repos`, under the
|
||||
scope lock); `syspkg -y` (needs root and `-n`; `-R` works), `gpkg -y`
|
||||
(system repos.conf + keyring until step 6). Dockerfiles copy grepo to
|
||||
/usr/local/bin. New suite `tests/repo_alpine.sh` (103 checks; 18
|
||||
mutants, all caught). Full regression green. Everest's libglacier-ng
|
||||
and glacier-ng packages rebuilt with it.
|
||||
- Next: step 4 (`-f` with dependency closure, then `-u`; the cache in
|
||||
`<base>/cache`, section 5's Fetch).
|
||||
|
||||
Step 2 DONE (2026-10-06, second part of the session; REPO_DESIGN
|
||||
section 7 "As implemented" has the details):
|
||||
- libglacier-ng `src/pkg.c`: `gl_conf_repo_keys(path, repo, out, len)`,
|
||||
a strict repos.conf reader; new statuses `GL_CONF_ERR_INVALID = 4`,
|
||||
`GL_CONF_ERR_SPACE = 5`.
|
||||
- glacier-ng `lib/gpkg_common.lua`: `verify_all(paths, opts)` refuses a
|
||||
base package in system scope without a trusted signature
|
||||
(`signed_for`, `trusted_keys`, `untrusted`); `verify_and_copy` returns
|
||||
the sidecar's bytes. glacier-bootstrap copies the sidecar's BYTES and
|
||||
the `.sig`. `gbuild -k`/`--key`, `tests/mkpkg -k`.
|
||||
- Every `Dockerfile.*` installs gsign + gsha256sum in /usr/local/bin and
|
||||
a TEST keyring (`glacier-test`, secret `/root/glacier-test.sec`).
|
||||
`syspkg_alpine.sh` / `overwrite_alpine.sh` make their own key.
|
||||
- New suite `tests/keyring_alpine.sh` (68 checks; 17 mutants: 10 Lua/
|
||||
shell, 7 C, all caught). Full regression green.
|
||||
- Everest: `./everest.sh keygen` made `everest-2026`; the SECRET key is
|
||||
`~/.config/everest/keys/everest-2026.sec` (user's choice: outside the
|
||||
project), mounted read-only for `build`, which refuses without it.
|
||||
Every package is built `gbuild -k`. everest-bootscripts 1.2.0 ships
|
||||
`etc/glacier/repos.conf` (base: everest-2026) and
|
||||
`etc/glacier/keys/everest-2026.pub`; `container.sh initramfs`
|
||||
installs the same into the build container's `/etc/glacier`.
|
||||
The signed `build all` was cut off after busybox (laptop died); the
|
||||
other 9 were rebuilt `-k` the evening of 2026-10-06, `initramfs` ran
|
||||
through glacier-bootstrap (signature checks passed for all 11), and
|
||||
Everest booted in QEMU: repos.conf + key in `/etc/glacier`, index
|
||||
lists all 11, syspkg/gpkg run as root and as a fresh user.
|
||||
- Base KEEP_OLD backups can't be restored with `-lO` (no signature):
|
||||
the user accepted that until the restore interface (REPO_DESIGN 10f).
|
||||
|
||||
Step 1 DONE (2026-10-06):
|
||||
- `src/monocypher/`: Monocypher 4.0.3, vendored UNMODIFIED (provenance
|
||||
and sha512 in its README; GitHub and monocypher.org copies matched).
|
||||
Compiles clean under the project's `-Werror` flags.
|
||||
- `src/sign.{c,h}`: signify formats (strict parse, canonical base64),
|
||||
`gl_sign_read_pubkey`/`_read_sig`/`_check_seckey`/`_verify`/
|
||||
`_verify_file`/`_file`/`_generate`, statuses `GL_SIGN_OK`..
|
||||
`GL_SIGN_ERR_WRITE = 9`. Built into `libglacier_verify.so` (NOT into
|
||||
the combined `libglacier-ng.so`).
|
||||
- `src/gsign.c` (`-G`/`-S`/`-V`, lg_printf messages naming the file at
|
||||
fault), Makefile targets `gsign` (in `all`, installed to bin like
|
||||
gsha256sum).
|
||||
- `glacier-ng/tests/sign_alpine.sh` (alpine:latest, mounts /libsrc):
|
||||
round trips, signify interop both ways (Alpine `signify`), encrypted
|
||||
and corrupted secret keys, malformed key/sig files, 64 MiB limit,
|
||||
O_NOFOLLOW, key modes, a direct FFI call of `gl_sign_generate`.
|
||||
Passes in full. (`luajit -e CODE ARG` runs ARG as a script, so the
|
||||
FFI snippet is written to `/gen.lua` first.)
|
||||
- Mutation checks: 12 mutants, all caught except m9 (padding position,
|
||||
`j < 2`), which is EQUIVALENT: kept, it prevents `3 - pad`
|
||||
underflowing. m5 (fchmod), m6 (final newline) and m10 (O_EXCL) needed
|
||||
the umask-077, final-char and FFI checks respectively.
|
||||
- `glacier-ng/lib/glacier_cdef.lua`: `gl_sign_*` declarations and
|
||||
`M.sign_name()`; checked against the real `.so` (struct sizes 40/72,
|
||||
every call on signify-made files). Full regression green after it.
|
||||
|
||||
Also open, for the user (raised 2026-10-06, not answered yet):
|
||||
- libgcc: luajit is built `-static-libgcc` (Everest has no libgcc_s).
|
||||
Claude's call; the alternative is a GCC runtime package in base
|
||||
(ROADMAP section 6).
|
||||
- Leftover `~/Projects/everest/pkgs/everest-bootscripts-1.0.0.gpkg`
|
||||
(+ sidecar), superseded by 1.1.0. Not deleted: ask first.
|
||||
|
||||
### Everest's glacier packages (1.4)
|
||||
|
||||
- `~/Projects/everest/scripts/container.sh` builds zlib, libarchive
|
||||
(gzip/tar only), libconfig, luajit (pinned commit,
|
||||
`LDFLAGS=-static-libgcc`), libglacier-ng, glacier-ng (all base) and
|
||||
doas (extra, OpenDoas, no PAM).
|
||||
- glacier-ng layout: `usr/lib/glacier-ng/{syspkg,gpkg,gstore,lib/*.lua}`,
|
||||
`lib/glacier -> /usr/lib/glacier` (absolute: relative links resolve
|
||||
inside the store), `usr/bin/X -> ../lib/glacier-ng/X`,
|
||||
`etc/glacier/glacier.conf`.
|
||||
- everest-bootscripts 1.2.0 (1.1.0 before signing) ships `/etc/doas.conf` (0400): `permit
|
||||
:wheel` plus nopass `gstore args -n` for BOTH `cmd gstore` and `cmd
|
||||
/usr/bin/gstore` (OpenDoas matches `cmd` against what was typed).
|
||||
- `templates/minimal.template` and `PACKAGES` in everest.sh list all 12
|
||||
packages (everest-repos added 2026-10-07). `Dockerfile.build` has autoconf, automake, libtool, cmake,
|
||||
pkgconf.
|
||||
|
||||
## The three projects
|
||||
|
||||
@@ -31,11 +248,12 @@ when the user asks.
|
||||
|---|---|---|
|
||||
| `~/Projects/libglacier-ng` | C library, design docs, this file | yes (`dev`) |
|
||||
| `~/Projects/glacier-ng` | Frontend: `gpkg`, `gstore`, `syspkg`, `gbuild`, `gworld`, `lib/*.lua`, `tests/`, Dockerfiles, `pkgs/` | **no**: back files up before editing (`file.old`; if that exists, `.old2`, `.old3`, ...) |
|
||||
| `~/Projects/everest` | The distro build: `everest.sh build\|initramfs\|boot`, packages, bootable QEMU image | no |
|
||||
| `~/Projects/everest` | The distro build: `everest.sh build\|initramfs\|boot`, packages, bootable QEMU image | **no**: same backup rule (but never inside `pkgsrc/*/etc/`, see Boot test) |
|
||||
|
||||
`glacier-ng/lib/glacier/` holds `.so`s built on the host (glibc). They
|
||||
won't load on musl. Every Alpine image and test builds libglacier-ng from
|
||||
source instead.
|
||||
Nothing prebuilt is kept in either repo (the host-built `.so`s in
|
||||
`glacier-ng/lib/glacier/` were removed 2026-10-06). Every image and test
|
||||
builds libglacier-ng from source; the Dockerfiles create
|
||||
`/app/lib/glacier/` with the musl `.so`s.
|
||||
|
||||
## What this project is
|
||||
|
||||
@@ -56,10 +274,20 @@ These are preferences stated in sessions. Follow them.
|
||||
- The pre-trust-check `die()` in syspkg/gstore reproduces the `[x]`
|
||||
format byte for byte, because it can't load `libglacier_log.so`
|
||||
before that file has been verified.
|
||||
- **Never say "Completed" after an error.** If the post-commit link pass
|
||||
refused any path (`ctx->link_refusals`, counted only for packages
|
||||
installed by that transaction), gpkg/syspkg end with
|
||||
`[x] N paths were not linked.` (`1 path was not linked.`) and exit 1.
|
||||
The transaction itself stays committed (2026-10-05).
|
||||
- **No hint messages.** Don't print lines that point users to other flags
|
||||
or commands (for example "use -lO to ..."). The user called these
|
||||
clutter. A refusal states the fact, in one line. The user also removed a
|
||||
`Run doas gstore -n ...` follow-up hint from gstore themselves.
|
||||
- **What goes where.** ALL design docs (and ROADMAP, this file) live in
|
||||
libglacier-ng, frontend ones included. Other than docs, libglacier-ng
|
||||
holds only backend material (C sources, the build). Frontend code,
|
||||
tests, Dockerfiles and everything the bootstrap runs on live in
|
||||
glacier-ng. Clarified 2026-10-06.
|
||||
- **Ask before settling a design choice that is the user's to make**, and
|
||||
give a recommendation. They have answered every such question promptly.
|
||||
Don't silently pick a layout or semantics.
|
||||
@@ -68,6 +296,10 @@ These are preferences stated in sessions. Follow them.
|
||||
defence in a throwaway copy and confirm the test fails ("mutation
|
||||
check"). Several defences here are verified that way.
|
||||
- Use doas, not sudo. syspkg is never setuid.
|
||||
- **Help text is terse** (2026-10-07): one short phrase per flag, no
|
||||
mechanism details (`-f` "Install packages from a remote repository",
|
||||
`-u` "Update system or individual packages", `-O` "Allow Glacier to
|
||||
overwrite existing files").
|
||||
|
||||
## Decisions the user has made (don't relitigate)
|
||||
|
||||
@@ -90,18 +322,105 @@ These are preferences stated in sessions. Follow them.
|
||||
- "Backed up" means the old package's files are compressed into a
|
||||
`.gpkg`. A restore interface comes later.
|
||||
- **`gpkg -u` / `syspkg -u` upgrade from a REMOTE repository**, and
|
||||
`gpkg -f` is `-l` but fetched by name from a remote repository. Both wait
|
||||
for remote repos + signing (ROADMAP 3.1). Don't implement them against
|
||||
local files.
|
||||
`gpkg -f` is `-l` but fetched by name from a remote repository. Done
|
||||
2026-10-07 (3.1 step 4). Never against local files.
|
||||
- **-f/-u details (2026-10-07):** first repo by priority wins (then its
|
||||
highest version); a too-old installed dependency is upgraded in the
|
||||
same transaction; cached downloads are deleted after commit.
|
||||
- **Dependencies (ROADMAP 2.3, decided 2026-10-05):** see the spec in
|
||||
ROADMAP 2.3. In short: minimum versions, conflicts both ways by name,
|
||||
no override flag, report every problem, missing deps refuse until 3.1
|
||||
brings automatic closure, auto/explicit recorded in the index now.
|
||||
- **sysroot (IMAGE_DESIGN section 3) accepted as proposed**, plus the
|
||||
library refuses a non-"/" sysroot for `GL_SCOPE_USR`. Order of work:
|
||||
2.3 first, then 1.1.
|
||||
- **System scope is world-readable** (dirs 0755, index files 0644): every
|
||||
user must be able to run what syspkg installs (2026-10-05). User scope
|
||||
stays 0700, and other users' programs stay unreachable to non-root
|
||||
users. Root CAN run them by full path, but other users' links are
|
||||
deliberately NOT on root's `PATH` (a user could get code run as root).
|
||||
- **Link refusals after a commit** end with `[x] N paths were not linked.`
|
||||
and exit 1, never "Completed" (the user's wording rule: don't say
|
||||
"completed" if there was an error).
|
||||
- **`syspkg -R DIR`** (not `-r`), `--root DIR`, `--root=DIR`; `-R`
|
||||
IMPLIES `--strict`; `--strict` alone is opt-in on the host.
|
||||
- **File collisions (2.6) are always refused**, every scope, strict or
|
||||
not (folded into 1.2 at the user's go-ahead).
|
||||
- **PATH:** per-user links, then `/glacier/sys/links/bin`, then the system
|
||||
(`/usr/bin` on Everest), in the Docker images AND Everest's
|
||||
`/etc/profile`. Non-base system packages must be on `PATH`; base
|
||||
programs must be runnable by every user.
|
||||
- `everest-bootscripts` is the name of the base `/etc` package. Its source
|
||||
lives in `~/Projects/everest/pkgsrc/`.
|
||||
- **glacier-bootstrap (2026-10-06):** template = `NAME=` + `PKG=name ver
|
||||
[sha256:hex]` lines, no `SYS_PROFILE` (unknown key until ROADMAP 3.3);
|
||||
it runs the `syspkg` on PATH (doas unless root); a template dependency
|
||||
must be listed at that version OR LATER (the 2.3 rule); Everest's
|
||||
initramfs is built through it.
|
||||
- **Installing Everest (2026-10-07):** possible from a live Everest
|
||||
system OR any Linux host; the LIVE SYSTEM is the preferred, officially
|
||||
supported way (so base/the live medium must eventually carry what a
|
||||
manual install needs). The user does the first QEMU disk install
|
||||
themselves: don't start it.
|
||||
- **Everest supports glibc as well as musl** (2026-10-07), Gentoo/Void
|
||||
style: the user picks the libc at install time, one per system.
|
||||
Today everything is built for musl (Alpine); glibc needs ROADMAP
|
||||
4.1's profiles. How packages are kept apart is still open (ROADMAP 6).
|
||||
- **Signing (2026-10-06, REPO_DESIGN 10e/10f):** Everest's SECRET key
|
||||
stays outside the project (`~/.config/everest/keys`, mounted
|
||||
read-only); a base KEEP_OLD backup refusing under `-lO` (unsigned) is
|
||||
accepted until the restore interface.
|
||||
|
||||
## Core architecture
|
||||
|
||||
**`gl_context_t`** (transaction.h) is the central abstraction. Fields:
|
||||
`uid`, `root_path`, `index_path`, `store_path`, `links_path`, `mode`
|
||||
(`GL_CTX_LIVE`/`GL_CTX_STAGE`), `scope` (`GL_SCOPE_USR`/`GL_SCOPE_SYS`),
|
||||
`lock_fd`, and `overwrite` (last field, zeroed by the init functions).
|
||||
**`gl_context_t`** (transaction.h) is the central abstraction. Fields, in
|
||||
order: `uid`, `root_path`, `index_path`, `store_path`, `links_path`,
|
||||
`mode` (`GL_CTX_LIVE`/`GL_CTX_STAGE`), `scope`
|
||||
(`GL_SCOPE_USR`/`GL_SCOPE_SYS`), `lock_fd`, then:
|
||||
|
||||
| Field | Set by | Meaning |
|
||||
|---|---|---|
|
||||
| `overwrite` | caller (after init, which zeroes it) | `-O`: the next link pass may displace foreign files |
|
||||
| `sysroot` | `*_at` init | install root, `""` = the running system |
|
||||
| `link_refusals` | link pass; commit copies the relink's count back | paths refused for this transaction's packages |
|
||||
| `strict` | caller (after init, which zeroes it) | refuse those before committing instead |
|
||||
|
||||
New fields go at the END, and `lib/glacier_cdef.lua` must mirror the
|
||||
struct exactly (FFI has no compiler to catch a mismatch).
|
||||
|
||||
**Status codes worth knowing:** `GL_TXN_ERR_DEPS = 9` covers every
|
||||
pre-commit refusal (dependency, conflict, file collision, strict link
|
||||
refusal); the library has already printed why, so the CLIs print nothing
|
||||
more. Live-context equivalents: `GL_INSTALL_ERR_DEPS = 8`,
|
||||
`GL_REMOVE_ERR_DEPS = 4`. `GL_INSTALL_ERR_SCOPE = 7` is base-in-user-
|
||||
scope.
|
||||
|
||||
**Install root / sysroot** (ROADMAP 1.1, done 2026-10-05):
|
||||
- `gl_init_live_context_at` / `gl_init_stage_context_at` / `gl_init_sys_at`
|
||||
take a sysroot; the old functions wrap them with `"/"`.
|
||||
`gl_normalize_sysroot` (common.h) refuses relative paths and `.`/`..`
|
||||
components. **User scope refuses any sysroot** other than `/`.
|
||||
- The context's paths are **INSTALL paths** (`T/glacier/sys/store`), so
|
||||
every filesystem operation uses them unchanged: a forgotten translation
|
||||
writes into T, never the host. That's deliberately the reverse of
|
||||
IMAGE_DESIGN's wording (runtime paths stored, `gl_at` on every
|
||||
operation), with the same effect and a safer failure mode.
|
||||
- Translate ONLY where a path's runtime meaning matters:
|
||||
`gl_runtime_path(ctx, p)` for link targets, `is_glacier_symlink`
|
||||
prefixes and `.displaced` keys; `gl_at(ctx, runtime, out)` to check a
|
||||
link's target (prune). `/usr` and `/etc` destinations are
|
||||
`ctx->sysroot` + path.
|
||||
- **Trust check** (`gl_check_sysroot` in transaction.c, also inside
|
||||
every `*_at` init and `gl_init_sys_at`; helper `gl_sysroot_trusted` in
|
||||
common.h): the root and every ancestor must exist and be root-owned,
|
||||
non-symlink directories, not group/world-writable. So `/tmp/x` is
|
||||
refused. Messages: `Refusing install root R: P is a symlink.` / `is
|
||||
missing.` / `is not root-owned, or is group/world-writable.`
|
||||
- Test: `tests/sysroot_alpine.sh` (host `/usr`, `/glacier`, `/etc`
|
||||
byte-identical; `chroot T` runs a linked binary; 8 mutations caught).
|
||||
- Not translated, on purpose: `KEEP_OLD` is read from the HOST's
|
||||
`/etc/glacier/glacier.conf` even under `-R` (the build host's policy);
|
||||
user scope's dependency view of "the system" is always the host's.
|
||||
|
||||
**Layout.** Each scope owns ONE base directory holding
|
||||
`{index,store,links,stage,old,lock,backup}`. Every create, rename and lock
|
||||
@@ -110,7 +429,7 @@ a transaction does stays inside it.
|
||||
| Scope | Base dir | Ownership |
|
||||
|---|---|---|
|
||||
| `GL_SCOPE_USR` | `/glacier/usr/<uid>/` | uid, mode 0700, inside a root-owned 0755 `/glacier/usr` |
|
||||
| `GL_SCOPE_SYS` | `/glacier/sys/` | root |
|
||||
| `GL_SCOPE_SYS` | `/glacier/sys/` | root, world-readable: dirs 0755, index files 0644 (`gl_scope_dir_mode`/`gl_scope_file_mode` in transaction.h), whatever the umask |
|
||||
|
||||
- The helpers `gl_usr_base`, `gl_usr_base_check` and the layout comment
|
||||
are in `common.h`.
|
||||
@@ -126,11 +445,22 @@ a transaction does stays inside it.
|
||||
live package inside a transaction needs no special casing. Seeding
|
||||
copies dotfiles: only `.` and `..` are skipped.
|
||||
- A stage on another filesystem fails at seeding with `GL_TXN_ERR_XDEV`.
|
||||
- **Commit:** rename-based promotion (stage → live, previous live → `old/`),
|
||||
then fsync the base. Then `gl_relink_store` + `gl_rebuild_index`
|
||||
regenerate links and index from a *fresh live context* and rescan the
|
||||
WHOLE store (every commit relinks every package, so link-time logic must
|
||||
be idempotent). `old/` is then deleted.
|
||||
- **Commit:** first `gl_check_txn` (deps, conflicts, collisions, and
|
||||
strict link refusals if `ctx->strict`); any problem → nothing changes,
|
||||
`GL_TXN_ERR_DEPS`. Then rename-based promotion (stage → live, previous
|
||||
live → `old/`), then fsync the base. Then `gl_relink_store` regenerates
|
||||
links from a *fresh live context* and rescans the WHOLE store (every
|
||||
commit relinks every package, so link-time logic must be idempotent);
|
||||
its `link_refusals` is copied back into the caller's context. `old/` is
|
||||
then deleted. (The index is rebuilt inside the stage by each
|
||||
install/remove, and promoted with it.)
|
||||
- **Live contexts** (no transaction; tests and third-party callers):
|
||||
`gl_install_pkg` runs `gl_check_plan` itself, and `gl_remove_pkg`
|
||||
removes the package's links by pruning (`gl_relink_store`), like a
|
||||
commit.
|
||||
- **Index files** are named to the second and seeding hardlinks them:
|
||||
never open a seeded file for writing, always write elsewhere and
|
||||
`rename()` (bug 25).
|
||||
|
||||
**Link routing** (per package, in `gl_link_pkg`):
|
||||
|
||||
@@ -157,7 +487,8 @@ transactions):
|
||||
| `<ver>/.glacier-manifest` | the package's manifest, saved at install, used by `gl_backup_pkg` |
|
||||
| `<ver>/.glacier-fresh` | one-shot "installed in this transaction"; link conflicts are only reported when present |
|
||||
| `<ver>/.glacier-overwrite` | one-shot `-O` permission for this package's next link pass |
|
||||
| `<store>/.displaced/<abs path>` | originals displaced by `-O`, restored when the link that replaced them is pruned |
|
||||
| `<ver>/.glacier-auto` | installed as a dependency, not by name (index field 4 = `auto`); set by `gl_set_pkg_auto`, nothing sets it until 3.1 |
|
||||
| `<store>/.displaced/<runtime abs path>` | originals displaced by `-O`, restored when the link that replaced them is pruned; keyed by RUNTIME path (no sysroot prefix) |
|
||||
| `<store>/.etc-state/<path>` | the last version of each `/etc` file a package offered |
|
||||
|
||||
**Configuration.** `/etc/glacier/glacier.conf` (libconfig syntax).
|
||||
@@ -189,8 +520,20 @@ possibly shared:
|
||||
plus `O_NOFOLLOW`).
|
||||
- Any entry with `..` or an absolute path refuses the whole package.
|
||||
- A failed extraction removes its half-made version directory.
|
||||
- **Recursive deletes.** `gl_rmdir_recursive` skips only `.`/`..`. Every
|
||||
caller passes a glacier-only path; keep it that way. `gl_delete_user`
|
||||
- **Recursive deletes.** `gl_rmdir_recursive` skips only `.`/`..`, and
|
||||
walks by descriptor (`openat(O_NOFOLLOW)`, `unlinkat`): an entry
|
||||
swapped for a symlink is unlinked, never followed (bug 31). Every
|
||||
caller passes a glacier-only path; keep it that way.
|
||||
- **Root reading a user's tree** (`add_entry_at`, `gstore -b`) is
|
||||
fd-relative with `O_NOFOLLOW` too (bug 32).
|
||||
- **Reserved names.** A package's top-level `files/.glacier-*` entries
|
||||
are refused (`GL_MARKER_PREFIX`, common.h): those names are glacier's
|
||||
own version-dir files, and every marker write uses `O_NOFOLLOW`
|
||||
(bug 30). The marker names are defined ONCE, in common.h.
|
||||
- **Manifests are untrusted**: `gl_gpm2gpkg` refuses `@include` and
|
||||
dependency/conflict entries without a valid name (bug 33). The CLIs
|
||||
read manifests ONLY through the library (`manifest_of` in
|
||||
gpkg_common.lua), never by pattern matching (bug 34). `gl_delete_user`
|
||||
lstat-checks its target first, because a symlinked base would otherwise
|
||||
have its *target* emptied.
|
||||
- **Wide wrappers.** Don't add convenience wrappers that silently touch a
|
||||
@@ -225,7 +568,8 @@ possibly shared:
|
||||
|
||||
**Version replacement** (`-lO`, `M.install` in `gpkg_common.lua`):
|
||||
1. Installed entries are read from the live index's `current` file
|
||||
(`repo::name::version` lines).
|
||||
(`repo::name::version::auto|explicit` lines; files before 2026-10-05
|
||||
stop at the version, and every parser accepts both).
|
||||
2. A same name+repo with a different version is a replacement. Without
|
||||
`-O` it's refused; with `-O`, `keep_old()` reads `KEEP_OLD`, and the
|
||||
summary shows `old -> new`.
|
||||
@@ -260,7 +604,70 @@ the merged-/usr tree: `/bin` and `/sbin` → `usr/bin`, `/lib` → `usr/lib`,
|
||||
- `gl_init_sys` runs it before writing the index. The index marks `-n` as
|
||||
complete, so a second `syspkg -n` is refused: `Cannot create
|
||||
/glacier/sys: already exists`.
|
||||
- It takes a root prefix, ready for `syspkg --root`.
|
||||
- It takes a root prefix; `gl_init_sys_at` / `syspkg -R DIR -n` use it.
|
||||
|
||||
**Dependencies and conflicts** (ROADMAP 2.3; `src/deps.c`, built into
|
||||
`libglacier_istoreutils.so`; rules in `deps.h`):
|
||||
- `gl_check_plan(live, adds, n, "repo::name" removes, n)` runs before the
|
||||
prompt (`check_plan` in `gpkg_common.lua`). `gl_check_txn(stage)`
|
||||
compares stage with live, and `gl_commit_transaction` calls it itself
|
||||
(`GL_TXN_ERR_DEPS = 9`). A live context runs the plan check inside
|
||||
`gl_install_pkg`/`gl_remove_pkg` (`GL_INSTALL_ERR_DEPS = 8`,
|
||||
`GL_REMOVE_ERR_DEPS = 4`). So no library caller can skip it.
|
||||
- It reads each version dir's `.glacier-manifest`; a version without one
|
||||
has no deps. Only what the transaction changes is checked.
|
||||
- `-x` of a name installed in several repos is only checked at commit
|
||||
(its repo is chosen after the prompt).
|
||||
- Claude's own calls, not reviewed by the user: conflicts only apply
|
||||
within one scope; only what a transaction changes is checked.
|
||||
|
||||
**Strict link check** (`check_links` in deps.c; `ctx->strict`): system
|
||||
scope only, for this transaction's packages, mirrors `gl_link_pkg`: a
|
||||
foreign file/symlink at a file's destination (unless `-O` and no earlier
|
||||
`.displaced` copy), a non-directory where a directory goes, or a
|
||||
directory over another package's symlink → `[x] Refusing to overwrite
|
||||
non-glacier path: P` / `Refusing to merge directory ...`. A foreign
|
||||
symlink where a directory goes (`usr/sbin -> bin`) is followed, as the
|
||||
link pass does.
|
||||
|
||||
**File collisions** (ROADMAP 2.6, `check_collisions` in deps.c): two
|
||||
packages whose entries land on the same path (by `dest_of`, which mirrors
|
||||
`gl_link_pkg`) are refused in every scope, before the prompt and at
|
||||
commit, when either is in the transaction: `[x] extra::a and extra::b
|
||||
both ship /glacier/sys/links/bin/x.` Sharing a directory is fine; a file
|
||||
or symlink where the other has a directory is a collision.
|
||||
|
||||
**Signatures** (REPO_DESIGN section 7; `gpkg_common.lua`): in system
|
||||
scope, a package whose LIBRARY-read manifest (`gl_read_pkg_manifest`,
|
||||
never `read_manifest_fields`) says base needs `FILE.sha256.sig` over the
|
||||
exact sidecar bytes verified, by the key with that keynum among those
|
||||
`/etc/glacier/repos.conf` lists for base (`gl_conf_repo_keys`). Else
|
||||
`[x] base::NAME is not signed by a key trusted for base.` before the
|
||||
prompt. repos.conf and keys (and all ancestors, real paths too) must be
|
||||
root-owned, not group/world-writable. Under `-R`, the HOST's keyring
|
||||
for this check; `-y`/`-f`/`-u` under `-R` use T's (inside T only).
|
||||
No repos.conf = nothing trusted. gpkg / other repos: sidecar only.
|
||||
|
||||
**Repository indexes** (REPO_DESIGN sections 4-5; `M.refresh` in
|
||||
`gpkg_common.lua`): `syspkg -y` / `gpkg -y` fetch `URL/repo.idx` + `.sig`
|
||||
for every repos.conf entry with a url, check the signature over the
|
||||
fetched bytes with a key listed for that repo, `gl_index_parse` the
|
||||
same bytes, refuse SERIAL below the accepted one (or equal with other
|
||||
bytes) and an expired VALID_UNTIL, then rename into
|
||||
`<base>/repos/REPO.idx` (+ `.sig`). Any failure keeps the accepted
|
||||
copy; every repo is tried; exit 1 if any failed.
|
||||
|
||||
**`-f` / `-u`** (REPO_DESIGN section 6; `M.fetch`/`M.upgrade`):
|
||||
refresh first (any failure refuses), pick each name from the first
|
||||
repo serving it, add missing deps (auto) and upgrade too-old ones,
|
||||
download into `<base>/cache/REPO` (hash and SIZE from the index), copy
|
||||
privately with the index's hash, require the manifest to equal the
|
||||
index entry (`[x] extra::x 1.0.0 does not match its index.`), then the
|
||||
`-l` path (`install_copies`): summary with `(dependency)`, 2.3 checks,
|
||||
prompt, staged transaction, `gl_set_pkg_auto`, backups, cache files
|
||||
deleted after commit. A base package fetched this way needs no `.sig`
|
||||
(the base index vouches). `-f` of an installed other version needs
|
||||
`-O`; `-u` and dependency upgrades don't. gpkg never fetches from base.
|
||||
|
||||
**Package symlinks** are extracted verbatim and linked like files.
|
||||
Hardlink, device and fifo entries are skipped. `add_to_archive` (used by
|
||||
@@ -270,14 +677,28 @@ backups and `gstore -b`) archives symlinks.
|
||||
|
||||
| Tool | Flags | Runs as |
|
||||
|---|---|---|
|
||||
| `gpkg` | `-l FILE...` `-x NAME...` `-O` `-S` `-V` `-h` `-v`; `-f`/`-u` are stubs | the user; per-user scope only, NEVER constructs `GL_SCOPE_SYS` |
|
||||
| `syspkg` | `-n` `-l FILE...` `-x NAME...` `-O` `-S` `-V` `-h` `-v` | root via doas; `-h`/`-v` without root |
|
||||
| `gstore` | `-n [UID]` `-b UID OUTFILE` `-x UID` `-s` (stub) `-V` `-h` `-v` | `-n` needs root (direct, or `doas gstore -n` for yourself) |
|
||||
| `gpkg` | `-l FILE...` `-f NAME...` `-u [NAME...]` `-x NAME...` `-y` `-O` `-S` `-V` `-h` `-v` | the user; per-user scope only, NEVER constructs `GL_SCOPE_SYS` |
|
||||
| `syspkg` | `-n` `-l FILE...` `-f NAME...` `-u [NAME...]` `-x NAME...` `-y` `-O` `-S` `-V` `-R DIR` `--strict` `-h` `-v` | root via doas; `-h`/`-v` without root |
|
||||
| `gstore` | `-n [UID]` `-b UID OUTFILE` `-x UID` `-s` (refuses: not implemented) `-V` `-h` `-v` | `-n` needs root (direct, or `doas gstore -n` for yourself) |
|
||||
| `glacier-bootstrap` | `-d PKGDIR TEMPLATE TARGET` `-n` `-t` `-V` `-h` `-v` | the user; borrows root only for `doas syspkg` / `doas chroot` |
|
||||
|
||||
- **gpkg:** modifiers `-O`/`-S`/`-V` are applied in a pre-pass, so their
|
||||
position doesn't matter (`-lO` == `-Ol`). It finds its user via
|
||||
`/proc/self/loginuid`, falling back to `getuid()`.
|
||||
`/proc/self/loginuid`, falling back to `getuid()`. Unknown flags are
|
||||
refused (`[x] Unknown option: -R`); gpkg has no `-R`/`--strict`.
|
||||
- **Endings (both CLIs):** `[✓] Completed with no errors.` (exit 0), or
|
||||
after link refusals `[x] N paths were not linked.` (exit 1;
|
||||
`report_done` in gpkg_common.lua). Refusals before the prompt exit 1
|
||||
with only the `[x]` lines.
|
||||
- **syspkg, `-n`:** creates the tree plus `/glacier/sys`.
|
||||
- **syspkg, `-R DIR` / `--root DIR` / `--root=DIR`** (last in a bundle:
|
||||
`-lR DIR`): operate on the system under DIR, through the `*_at`
|
||||
contexts. Refused first by `gl_check_sysroot`: DIR and every ancestor
|
||||
must be root-owned, non-symlink directories, not group/world-writable
|
||||
(`/tmp/...` is refused). Implies `--strict`.
|
||||
- **syspkg, `--strict`:** anything the link pass would refuse fails the
|
||||
whole transaction before the prompt (and at commit, `ctx->strict`);
|
||||
`-O` still displaces.
|
||||
- **syspkg as non-root:** `-n`/`-l`/`-x` are refused after argument
|
||||
parsing with `Cannot open system index: Permission denied`.
|
||||
- **gstore, `-n` without a UID, under doas:** acts for the invoking user
|
||||
@@ -290,11 +711,25 @@ backups and `gstore -b`) archives symlinks.
|
||||
installed fails before the prompt.
|
||||
- **Hardened loading** (syspkg and gstore): `#!/usr/bin/luajit -E`,
|
||||
`package.path` and every `ffi.load` pinned to their own `realpath`, and
|
||||
when euid is 0, every ancestor directory and every loaded `.lua`/`.so`
|
||||
must be root-owned and not group/world-writable. The preamble is
|
||||
when euid is 0, every loaded `.lua`/`.so`, and every ancestor of the
|
||||
script's realpath AND of each loaded file's realpath, must be
|
||||
root-owned and not group/world-writable (bug 29). The preamble is
|
||||
DUPLICATED in both files on purpose: change one, change both.
|
||||
- **gpkg** uses the older CWD-relative convention (`require("lib/...")`,
|
||||
`ffi.load("./lib/glacier/...")`): run it from its own directory.
|
||||
- **gpkg** (2026-10-06) loads `lib/` from its own realpath, like
|
||||
glacier-bootstrap (no ownership checks; it's unprivileged), so an
|
||||
installed `/usr/bin/gpkg` works from any directory.
|
||||
- **glacier-bootstrap** (GLACIER_BOOTSTRAP section 10): parse →
|
||||
verify (copies each `PKGDIR/NAME-VER.gpkg` into a private `mktemp -d`
|
||||
while hashing; sidecar + pin; writes the copy's sidecar from that hash)
|
||||
→ resolve (reads manifests of the COPIES via `gl_read_pkg_manifest`:
|
||||
name/version match, deps listed at >= version, conflicts) → `syspkg -R
|
||||
T -n` (skipped if T is initialised with nothing installed) → `syspkg -R
|
||||
T -l COPIES` (one strict transaction; its prompt passes through) →
|
||||
checks T's index lists every package (syspkg exits 0 when declined) →
|
||||
`syspkg -R T -y` if T's repos.conf has a url → `-t` smoke test.
|
||||
Never writes into T itself. Exit: 1 usage/target, 2 template,
|
||||
3 verify, 4 resolve, 5 install, 6 smoke, 7 repositories. Loads its `lib/`
|
||||
from its realpath, no root checks. No `-r` / `--pin` yet.
|
||||
- **Declarations:** `lib/glacier_cdef.lua` is the single source of truth
|
||||
for every `ffi.cdef`. `tests/glacier_ffi.lua` is only a wrapper around
|
||||
it. Declaration ORDER matters: a declaration using `gl_context_t` must
|
||||
@@ -303,7 +738,9 @@ backups and `gstore -b`) archives symlinks.
|
||||
## Build tooling
|
||||
|
||||
- **gbuild** (bash, the only copy is `glacier-ng/gbuild`):
|
||||
- `-n -V -g -c -d -m -D -C -b -i -o -p -j -F -t -x -u -r -s`;
|
||||
- `-n -V -g -c -d -m -D -C -b -i -o -p -j -F -t -x -u -r -k -s`
|
||||
(`-k KEY.sec`: sign the sidecar with gsign, `$GSIGN_BIN` / build/bin
|
||||
/ PATH, key tried before building);
|
||||
`-v`/`--version` prints the version, `--verbose` turns on verbose
|
||||
output.
|
||||
- Autodetects the build system, with autotools checked BEFORE a bare
|
||||
@@ -320,15 +757,29 @@ backups and `gstore -b`) archives symlinks.
|
||||
(`Dockerfile.build`).
|
||||
- **Kernel version:** linux `6.12.112`; plain `v6.12` doesn't build with
|
||||
GCC 15.
|
||||
- **Everest's image IS built through glacier** (2026-10-06): `container.sh
|
||||
initramfs` copies glacier-ng to a root-owned `/opt/glacier-ng` (with
|
||||
the musl-built `.so`s; syspkg's trust checks need that) and runs
|
||||
`glacier-bootstrap -t -d /everest/pkgs templates/minimal.template
|
||||
/target` (root-owned: not under `/tmp`). T's links name RUNTIME paths,
|
||||
so anything reading through them from the build host must resolve
|
||||
inside T (vmlinuz is read with `chroot T cat`). Only `root` exists in
|
||||
the shipped `/etc/passwd`.
|
||||
- gbuild ends with the package's contents list. Its "To install with
|
||||
gpkg: gpkg -l ..." hint was removed 2026-10-06 at the user's request.
|
||||
|
||||
## libglacier-ng build
|
||||
|
||||
- **Separate `.so` per module** (`libglacier_log`, `_pkg`, `_istoreutils`,
|
||||
`_transaction`, `_dag`, `_verify`). This split is deliberate: a
|
||||
`_transaction`, `_verify`; `_dag` was removed 2026-10-06, unused). This split is deliberate: a
|
||||
third-party program links only what it needs.
|
||||
- **The combined `libglacier-ng.so`** (`make libglacier-ng`) exists only
|
||||
for the FFI tests.
|
||||
- **`libglacier_verify.so`** is private: no public header.
|
||||
- **Tools** in `build/bin`: `gsha256sum`, `gsign`, `grepo` (gsign and
|
||||
grepo compile the sources they need in, no rpath). Maintainer tools,
|
||||
later "glacier-tools": Everest's libglacier-ng package drops
|
||||
`usr/bin`.
|
||||
- **Warnings:** `config.mk` builds with `-Werror`. `-Wno-format-truncation`
|
||||
is set because that warning is noise; `-Wstringop-truncation` stays
|
||||
fatal.
|
||||
@@ -340,7 +791,7 @@ backups and `gstore -b`) archives symlinks.
|
||||
Run everything in throwaway containers. Several suites write to `/usr`,
|
||||
`/etc` and `/glacier`.
|
||||
|
||||
**FFI suite** (`glacier-ng/tests/*.lua`; 8 files, all passing):
|
||||
**FFI suite** (`glacier-ng/tests/*.lua`; 11 files, all passing):
|
||||
```sh
|
||||
docker run --rm --cap-add SYS_ADMIN -v ~/Projects/glacier-ng:/src:ro \
|
||||
-v ~/Projects/libglacier-ng:/libsrc:ro alpine:latest sh -c '
|
||||
@@ -355,8 +806,9 @@ GNU `tar` is required: the hostile-archive fixtures use `tar --append`.
|
||||
|
||||
| Suite | Image |
|
||||
|---|---|
|
||||
| `sign_alpine.sh` | `alpine:latest` with `/libsrc` and `/t.sh` mounted |
|
||||
| `syspkg_alpine.sh`, `overwrite_alpine.sh` | `alpine:latest` with `/src`, `/libsrc` and `/t.sh` mounted (see each header); `overwrite_alpine.sh` runs both as-is and with `--tmpfs /glacier:exec` |
|
||||
| `verify_alpine.sh`, `user_layout_alpine.sh`, `upgrade_alpine.sh` | `glacier-alpine_root` |
|
||||
| `verify_alpine.sh`, `user_layout_alpine.sh`, `upgrade_alpine.sh`, `sys_perms_alpine.sh`, `deps_alpine.sh`, `sysroot_alpine.sh`, `root_alpine.sh`, `bootstrap_alpine.sh`, `installed_alpine.sh`, `keyring_alpine.sh`, `repo_alpine.sh`, `audit_alpine.sh`, `fetch_alpine.sh` | `glacier-alpine_root` |
|
||||
|
||||
Build the `glacier-alpine_root` image with:
|
||||
```sh
|
||||
@@ -365,6 +817,46 @@ cd ~/Projects/glacier-ng && docker build -f Dockerfile.alpine_root \
|
||||
docker run --rm -v "$PWD/tests/NAME.sh:/t.sh:ro" glacier-alpine_root sh /t.sh
|
||||
```
|
||||
|
||||
`glacier-ng/lib/glacier/` does not exist in the tree (the `.so`s are
|
||||
built per image); syspkg_alpine/overwrite_alpine `mkdir -p` it
|
||||
(2026-10-07). If Docker fails with `veth ... operation not supported`,
|
||||
the host kernel was upgraded without a reboot (modules missing): use
|
||||
`docker build --network host` and `docker run --network=host` (or
|
||||
`--network none` where nothing is fetched) until the user reboots.
|
||||
Mutating a Lua file syspkg loads: mount it elsewhere and `cp` it over
|
||||
`/app/lib/...` inside the container (a bind mount is user-owned, so
|
||||
syspkg's trust check refuses it).
|
||||
|
||||
**Demo packages:** `glacier-ng/tests/mkpkg [-o DIR] [-k KEY.sec]
|
||||
[-d NAME:VER]... [-c NAME:VER]... NAME VERSION REPO` writes a one-script
|
||||
`.gpkg` plus its sidecar (`/app/tests/mkpkg` in the images); a base one
|
||||
needs `-k /root/glacier-test.sec` (the images' test key) for syspkg. The user plays with changes
|
||||
in `./run_docker.sh Dockerfile.alpine_root` using it; for an image,
|
||||
`mkdir -p /mnt/img && ./syspkg -R /mnt/img -n`.
|
||||
|
||||
**Mutation checks, how:** `cp -a` libglacier-ng (and glacier-ng, for Lua
|
||||
changes) into the scratchpad, break the defence there, then
|
||||
`docker build -f Dockerfile.alpine_root --build-context libglacier=COPY
|
||||
-t glacier-mut .` from the (copied) glacier-ng and run the suite against
|
||||
`glacier-mut`; delete the image after. Docker tags must be lowercase. A
|
||||
mutation that doesn't compile under `-Werror` proves nothing: rewrite it
|
||||
so it compiles. Watch for checks that pass for the wrong reason (e.g. a
|
||||
refusal caused by a missing directory instead of the defence). In shell
|
||||
suites, `sh -c '... $OUT ...'` sees an EMPTY `$OUT` (not exported), so a
|
||||
negative check written that way passes vacuously; use a function
|
||||
(`lacks` in bootstrap_alpine.sh). Found 2026-10-06 by a surviving
|
||||
mutation. The host shell is zsh: an unquoted `$VAR` holding `-v a:b`
|
||||
is NOT word-split, so a docker command built that way silently runs
|
||||
without the mount (a mutant "survived" that way, 2026-10-06). To
|
||||
mutate a Lua CLI without rebuilding, bind-mount the mutated file over
|
||||
`/app/NAME` (`-v $PWD/mut:/app/glacier-bootstrap:ro`).
|
||||
|
||||
**Full regression** = the FFI suite + `syspkg_alpine.sh` +
|
||||
`overwrite_alpine.sh` (both ways) + `sign_alpine.sh` + the thirteen `glacier-alpine_root`
|
||||
suites, all in parallel; the two output-based suites (`syspkg_alpine`,
|
||||
`user_layout_alpine`) have no PASS lines, so grep their output for
|
||||
`Unknown|traceback|failed|GL_` instead (only `--bogus` is expected).
|
||||
|
||||
**Interactive images:** `./run_docker.sh Dockerfile.{alpine,alpine_root,arch,debian}`.
|
||||
`run_docker.sh` passes the `libglacier` build context. `alpine`, `arch`
|
||||
and `debian` default to user `glacier` (uid 1000, in wheel); `alpine_root`
|
||||
@@ -372,10 +864,24 @@ defaults to root. All of them have:
|
||||
- root-owned `/app`;
|
||||
- no `/glacier` until `gstore -n` / `syspkg -n`;
|
||||
- the doas rule plus `permit nopass glacier`;
|
||||
- `/etc/profile.d/glacier.sh` putting `/glacier/usr/<uid>/links/bin` then
|
||||
`/glacier/sys/links/bin` on `PATH` (and `lib` on `LD_LIBRARY_PATH`)
|
||||
unconditionally, since `/glacier` only appears after login (fixed
|
||||
2026-10-05: it had no system links and checked `-d` at login);
|
||||
- `glacier.conf` installed.
|
||||
|
||||
**Boot test:** `cd ~/Projects/everest && ./everest.sh boot` boots QEMU
|
||||
(KVM). Press Enter for a root shell; `poweroff -f` to leave.
|
||||
(KVM). Press Enter for a root shell; `poweroff -f` to leave. To test as
|
||||
a non-root user, `adduser -D -H -h /tmp tester` at runtime, then
|
||||
`su - tester -c ...` (nothing shipped has a user besides root).
|
||||
Everest's `/etc/profile` sets `PATH` to per-user links, then
|
||||
`/glacier/sys/links/bin`, then `/usr/bin` (2026-10-05). Backups of
|
||||
everest-bootscripts files go NEXT TO `etc/`, never inside it: the package
|
||||
copies `etc/` wholesale. After changing a package:
|
||||
`./everest.sh build PKG && ./everest.sh initramfs`, then boot.
|
||||
To drive the boot non-interactively, fork QEMU on a pty from Python
|
||||
(`pty.fork()`), wait for `Please press Enter`, send lines and match the
|
||||
`# `/`$ ` prompt (no `expect` on the host).
|
||||
|
||||
## Bug history: so these don't get reintroduced
|
||||
|
||||
@@ -430,6 +936,99 @@ All fixed in the current tree.
|
||||
context every retry reported `ALREADY_INSTALLED`.
|
||||
23. Backups written in the same second were pruned in the wrong order
|
||||
(whole-second mtimes).
|
||||
24. System scope was root 0700 (index, store, links, every version dir),
|
||||
so no non-root user could run anything syspkg installed, through
|
||||
`/usr` or `/glacier/sys/links`. Found 2026-10-05. Also fixed: every
|
||||
mode glacier creates was filtered by the caller's umask (`gl_mkdirp`,
|
||||
seeding, extracted files), so syspkg under umask 077 made even a
|
||||
0755 store private. `gl_mkdirp`/`gl_mkdir_exact` now chmod what they
|
||||
create, and extracted entries get `archive mode & 07755` explicitly
|
||||
(never group/world-writable). Test: `tests/sys_perms_alpine.sh`
|
||||
(mutation-checked). No migration: a store made before the fix keeps
|
||||
its 0700 dirs until reinstalled.
|
||||
25. `gl_rebuild_index` `fopen(name, "w")`'d the new index file. Index
|
||||
names have one-second resolution and seeding HARDLINKS live's index
|
||||
files into the stage, so a staged install in the same second as the
|
||||
last commit rewrote the LIVE index in place; an aborted transaction
|
||||
then left live listing the stage's packages (and history files were
|
||||
overwritten). It now writes to `tmpfile()` and only `rename()`s onto
|
||||
the name. Found 2026-10-05 by `deps_alpine.sh`; regression test
|
||||
`tests/test_index_abort.lua` (mutation-checked). Rule: never open a
|
||||
seeded file for writing; replace it by rename.
|
||||
26. Live-context removal (`gl_unlink_pkg`) `unlink()`ed the destination
|
||||
of every regular file in the package WITHOUT `is_glacier_symlink`:
|
||||
a foreign file the package had been refused to replace was deleted,
|
||||
and the package's own symlink entries were never unlinked. The CLIs
|
||||
always stage, so only library callers were exposed. Removed;
|
||||
live removal now prunes like a commit (`gl_relink_store`). Found
|
||||
2026-10-05; `tests/test_live_remove_foreign.lua` (fails on the old
|
||||
code).
|
||||
27. gpkg's `handle_flag` had no `else`: an unknown flag (`-R`) was
|
||||
silently ignored and the NEXT word reported as the unknown option.
|
||||
Now `[x] Unknown option: -R` (capitalized, as syspkg). 2026-10-05.
|
||||
28. gbuild's `clone_source` printed its "Shallow clone failed" warning
|
||||
(and the full clone's/checkout's output) to stdout, which
|
||||
`SOURCE_DIR="$(clone_source)"` captured, so building from a commit
|
||||
hash (`-r <sha>`) always failed. Those now go to stderr. Found
|
||||
2026-10-06 building luajit.
|
||||
29. syspkg's and gstore's trust checks walked only the ancestors of the
|
||||
script's own realpath. Installed, `lib/glacier -> /usr/lib/glacier`
|
||||
and the `.so`s link into libglacier-ng's store dir, so a non-root-
|
||||
owned directory there was never checked: root loaded `.so`s from it.
|
||||
Both now also check every ancestor of each loaded file's realpath.
|
||||
Found 2026-10-06 by `tests/installed_alpine.sh` (both mutations
|
||||
caught).
|
||||
|
||||
30. A package could ship top-level `files/.glacier-*` entries. As a
|
||||
symlink, `.glacier-fresh` made syspkg truncate its target as root
|
||||
(an unsigned extra package emptied `/root/victim`, reproduced);
|
||||
`.glacier-overwrite` self-granted `-O`. Reserved names refused,
|
||||
marker writes `O_NOFOLLOW`. 2026-10-06 audit.
|
||||
31. `gl_rmdir_recursive` (lstat, then opendir by path) could be steered
|
||||
into a symlink's target when root deleted a user's tree (`gstore
|
||||
-x`). Now fd-relative. Race test in audit_alpine.sh.
|
||||
32. `add_to_archive` (same class, `gstore -b`). Now fd-relative; no test
|
||||
fails on the old code (window too narrow; review only).
|
||||
33. Manifests could `@include` any file (read as root); a dependency
|
||||
entry without a name cut the list short. Both refused.
|
||||
34. gpkg_common's `read_manifest_fields` (tar + pattern match) could
|
||||
disagree with the library: a decoy comment hid a replacement. Now
|
||||
`manifest_of` (`gl_read_pkg_manifest`).
|
||||
35. Format strings: gstore's `argv[0]`/UID and gpkg `-u`'s operand
|
||||
reached `lg_printf` as the format; `%d` was given Lua numbers;
|
||||
gstore UIDs wrapped past uid_t; `gl_abort_transaction` wiped a stage
|
||||
without holding the lock. All fixed (`lg_printf` now has a printf
|
||||
format attribute).
|
||||
|
||||
## Known gaps (not bugs, not yet scheduled)
|
||||
|
||||
- No migration for stores made before bug 24's fix (0700 dirs stay).
|
||||
- Extracted entries keep setuid/setgid bits (`& 07755`), as before.
|
||||
- No size limit on extraction (a decompression bomb fills the disk).
|
||||
- Bug 32's fix has no test that fails on the old code.
|
||||
- Hash verification is still CLI-only (ROADMAP 3.2); everything else
|
||||
added 2026-10-05 is enforced in the library.
|
||||
- A dependency refusal for `-x` of a name installed in several repos
|
||||
comes at commit, after the prompt (its repo is chosen after it).
|
||||
|
||||
## Backups of glacier-ng / everest files (no git there)
|
||||
|
||||
The rule stands (back up before editing: `file.old`, then `.old2`...).
|
||||
All of glacier-ng's backups were deleted 2026-10-06 at the user's
|
||||
request. The 2026-10-07 session made new ones: `lib/glacier_cdef.lua.old`,
|
||||
`lib/gpkg_common.lua.old`, `gpkg.old`, `syspkg.old`,
|
||||
`tests/{audit,overwrite,syspkg,repo}_alpine.sh.old`,
|
||||
`lib/gpkg_common.lua.old2`/`.old3`, `tests/repo_alpine.sh.old2`/`.old3`,
|
||||
`tests/fetch_alpine.sh.old`, `lib/gpkg_common.lua.old4`,
|
||||
`glacier-bootstrap.old`, `tests/bootstrap_alpine.sh.old`,
|
||||
`tests/repo_alpine.sh.old4`, `tests/fetch_alpine.sh.old2`; Everest:
|
||||
`scripts/container.sh.old4`, `everest.sh.old3`, `README.md.old3`,
|
||||
`templates/minimal.template.old2`, `pkgsrc/everest-bootscripts/glacier.old`.
|
||||
Everest still has its earlier ones (`everest.sh.old`/`.old2`,
|
||||
`scripts/container.sh.old`/`.old2`/`.old3`, `README.md.old`/`.old2`,
|
||||
`Dockerfile.build.old`, `templates/minimal.template.old`,
|
||||
`pkgsrc/everest-bootscripts/profile.old`). Leave them unless the user
|
||||
says to clean up.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
Reference in new issue
Block a user