libglacier-ng/CLAUDE.md

# CLAUDE.md

Context for Claude Code sessions on this repo (libglacier-ng, the package
manager for the Everest/Glacier Linux distribution). This file exists
because there's no built-in way to hand off a claude.ai planning
conversation into Claude Code — this is that handoff, written down.

For the symlink-routing / package-scope design specifically, see
`PACKAGE_SCOPES.txt` at the repo root — that doc is the source of truth
for that topic and isn't duplicated here.

## What this project is

A from-scratch Linux distribution (Everest, built on the Glacier
package-management model) with its own package manager
(`libglacier-ng` + `gpkg`/`gstore` CLI tools) and its own build
tooling (`gbuild`/`gworld`) for bootstrapping a set of packages up to
a minimal bootable system.

## Core architecture

**`gl_context_t`** (transaction.h) is the central abstraction: `uid`,
`root_path`, `index_path`, `store_path`, `links_path`, `mode`
(`GL_CTX_LIVE` / `GL_CTX_STAGE`), `scope` (`GL_SCOPE_USR` /
`GL_SCOPE_SYS`), `lock_fd`.

**Transaction lifecycle**: `gl_init_live_context` / `gl_init_stage_context`
→ `gl_commit_transaction` or `gl_abort_transaction`. Staged installs
hardlink-seed `index`/`store` from live at stage-init time (copy-on-write,
so removing a live package inside a transaction doesn't require special
casing). The `links` tree is deliberately **not** seeded — see below.

**Commit** does atomic rename-based promotion (stage → live, with the
previous live moved to `old/` as a rollback point), fsyncs the parent
dirs, then calls `gl_relink_store` to regenerate symlinks and
`gl_rebuild_index` to regenerate the index — both from a fresh
`GL_CTX_LIVE` context, not the just-committed stage context.

**Scopes**:
- `GL_SCOPE_USR`: per-uid, `/glacier/usr/{index,store,links}/<uid>`.
- `GL_SCOPE_SYS`: system-wide, `/glacier/sys/{index,store}`. Symlink
  destination depends on the package's **repo** — see `PACKAGE_SCOPES.txt`.
  Short version: `repo == "base"` → `/usr`; anything else →
  `/glacier/sys/links`. This is decided per-package (in `gl_link_pkg` /
  `gl_relink_store`), not baked into the context.

**Repos**: `base` (minimal working system, boot-critical by definition —
this is a repo-level classification, not a per-package flag), `extra`
(important but not required), `community` (everything else). `extra`
and `community` currently behave identically — see open items.

## Safety mechanisms (read before touching links/symlink code)

`ctx->links_path` is **not always glacier-exclusive territory** —
for system-scope `base` packages it's literally `/usr`, which holds
plenty of content glacier has no business touching. Everything that
reads or writes a links destination has to treat it as "possibly
shared":

- `is_glacier_symlink(path, store_prefix)` (istoreutils.c) is the one
  source of truth for "did glacier create this." True only if `path`
  is a symlink whose target lives under `store_prefix`. Never touch a
  path this returns false for.
- `gl_link_pkg` will not overwrite an existing path unless
  `is_glacier_symlink` says it's safe to replace.
- `gl_relink_store` prunes stale symlinks (`prune_stale_links`) rather
  than wiping and rebuilding the whole links directory — a full
  `rm -rf` of `ctx->links_path` would be catastrophic once that path
  can be `/usr`. It prunes **both** possible system-scope destinations
  (`/glacier/sys/links` and `/usr`) since a single pass can't know in
  advance whether any base-repo packages are involved.
- The stage links tree is never hardlink-seeded — it's genuinely
  unused (`gl_link_pkg` only ever runs against `GL_CTX_LIVE`;
  `gl_relink_store` always builds its own fresh live context). Seeding
  it would mean hardlinking all of `/usr` on every staged system
  transaction, and risking `EXDEV` if `/usr` and the stage area are on
  different filesystems.

## Bug history (so it doesn't get relitigated or reintroduced)

Roughly chronological, all already fixed in the current tree:

1. **`gl_abort_transaction` never actually released the lock file** — a
   copy-paste bug wrote into the wrong variable. Fixed by moving off
   `O_EXCL`-based locking entirely: `gl_begin_transaction` now just
   `open(O_CREAT)` + `flock()`, and lock files are never `unlink()`'d
   (the kernel releases `flock()` automatically on process death, even
   `SIGKILL`; explicit unlinking has its own TOCTOU race with a second
   process creating a new inode at the same path).
2. **`EXDEV` wasn't checked before renaming** — `gl_commit_transaction`
   now `stat()`s both sides and fails cleanly with `GL_TXN_ERR_XDEV`
   before attempting any renames if stage and live aren't on the same
   filesystem.
3. **Segfault on install**, root cause: the Makefile's `transaction`
   target didn't link `libglacier_transaction.so` against
   `libglacier_istoreutils.so`/`libglacier_log.so`, even though
   `transaction.c` calls `gl_relink_store()` and `lg_printf()`. Fixed
   by adding those `-l` flags and an explicit `transaction: log
   istoreutils` prerequisite (build order matters now).
4. **Symlinks going stale after commit** — `gl_link_pkg` used to run
   during staged installs too, embedding the *stage* path as the
   symlink target. Fixed: `gl_link_pkg` only runs for `GL_CTX_LIVE`;
   links are fully regenerated post-commit by `gl_relink_store`.
5. **Dangling symlinks never cleaned up on removal** — see "Safety
   mechanisms" above; this is what `prune_stale_links` fixes.
6. **`GL_TXN_ERR_RENAME` on every commit after the first** — leftover
   `old/` directory from a previous commit made the next commit's
   `rename()` fail with `ENOTEMPTY`. Fixed: `gl_commit_transaction`
   wipes `old_base` if present before using it.
7. **`-Wstringop-truncation` errors** (real bug, not noise) — four
   `strncpy(dst, ctx->foo_path, PATH_MAX - 1)` calls that could leave
   `dst` unterminated. Fixed by switching to `snprintf(dst,
   sizeof(dst), "%s", ...)`, which always terminates.
   (`-Wformat-truncation`, separately, is suppressed via
   `-Wno-error=format-truncation` in `config.mk` — those warnings are
   about genuinely-safe `snprintf` truncation, not a real bug.)
8. **`gbuild` build-system autodetection** checked for a bare
   `Makefile` before `configure`/`configure.ac`. Broke musl (and any
   project with a hand-rolled, non-autotools build system that ships
   both) since its `Makefile` is non-functional without `configure`
   generating `config.mak` first. Fixed by reordering the checks.
9. **Index only stored `repo::pkgname`, no version.** Now
   `repo::pkgname::version` (`gl_rebuild_index` looks up the
   `<pkgname>-<version>` store subdirectory). `struct gpkg_entry` in
   `istoreutils.h` needs a `char *ver;` field — this was a manual
   header edit, double check it's actually present.
10. **`gworld` recipe parser silently dropped `SYS_PROFILE`** — the
    key-matching regex only matched uppercase letters, not
    underscores. Fixed to `^([%u_][%u%d_]*)=(.*)$`. Caught by an
    actual precedence test, not code review — worth remembering that
    class of bug (a field silently never parsing, falling back to a
    default that happens to look right) is easy to miss by inspection
    alone.

## CLI additions

- `gpkg -s` / `--system`: operate against `GL_SCOPE_SYS` instead of a
  uid's tree. Requires root (checked via `geteuid()`). Must appear
  *before* `-l`/`-x` in a grouped flag string (e.g. `-sl`, not `-ls`) —
  flags are handled in parse order, same constraint `-V`/`-S` already
  had.
- `gstore -N` / `--new-system`: calls `gl_init_sys()`. Requires root.
- `gbuild -P` / `--profile PROFILE`: system profile, `ARCH-LIBC[-FEATURE...]`
  (e.g. `x86_64-musl`, `x86_64-glibc-multilib`). Derives a cross-compile
  prefix (translating `glibc`→`gnu` per real GNU triplet convention),
  detects native-vs-cross against the build host, and only pre-fills
  `-t`/`-x` defaults for whichever the user didn't already set
  explicitly. Features export as `GL_FEATURE_<NAME>=1` env vars for
  custom `-b`/`-i` commands to branch on — gbuild itself doesn't
  hardcode per-feature behavior, that's intentional.

## Build system

`Makefile` builds separate `.so`s per module
(`libglacier_log.so`, `libglacier_pkg.so`, `libglacier_istoreutils.so`,
`libglacier_transaction.so`, `libglacier_dag.so`) — **this split is
deliberate Unix-philosophy design**, not an accident: a third-party
glacier-compatible program should be able to link only
`libglacier_log.so` without pulling in everything else. `gpkg`/`gstore`
load the specific per-module `.so`s they need directly, matching this.

There's also a `libglacier-ng` Makefile target producing a **combined**
`.so` — this exists *solely* for the Lua FFI test harness's
convenience (one thing to `ffi.load()` instead of juggling rpath
chains in a throwaway test process). It is **not** in `all` and **not**
installed. Don't reach for it outside `tests/` — that's a sign
something should be split-loaded instead.

`config.mk` holds `CFLAGS`, including `-Wno-error=format-truncation`
(see bug #7 above for why that specific warning is suppressed and
others aren't).

## Lua tooling

- `gpkg`, `gstore`: LuaJIT + FFI, calling directly into the `.so`s.
- `lib/glacier_cdef.lua`: single shared source of truth for every
  `ffi.cdef()` declaration, used by `gpkg`, `gstore`, and the test
  suite. This used to be duplicated three ways, which is exactly what
  let a stale signature drift silently into `gpkg` at one point — if
  you're editing a C function signature, this file needs the matching
  update, and it's the *only* place that needs it now.
- `tests/*.lua`: LuaJIT FFI test suite exercising real filesystem/process
  behavior (`fork`+`SIGKILL` for lock crash-recovery, real `mount()`
  for the `EXDEV` check, etc.) — not mocked. Run via:
  ```sh
  cd tests
  make -C .. libglacier-ng   # builds the test-only combined .so
  GLACIER_TEST_CONFIRM=yes LIBGLACIER_SO=$(pwd)/../build/lib/shared/libglacier-ng.so luajit run_all.lua
  ```
  `test_exdev.lua` needs `--cap-add=SYS_ADMIN` in Docker to actually
  exercise the mount path; it skips cleanly (not a failure) without it.

## Build automation (bootstrap toolchain)

- `gbuild` (bash): single-package builder. Autodetects build system
  (cmake/meson/cargo/go/autotools/make), supports cross-compilation
  (`-t`/`-x`, now also `-P` for profiles).
- `gworld` (Lua): batch orchestrator. Reads `recipes/*.recipe`,
  topologically sorts by `DEPS`, invokes `gbuild` once per package in
  order. `-s SYSROOT` gives every package in the batch a shared
  staging dir — relies on `gbuild -p` never wiping its target, so
  headers/libs accumulate across the batch for free.
- Recipe format: `NAME VER REPO URL/REF-or-SRC DEPS BUILD INSTALL
  FLAGS SYS_PROFILE` (plain `key=value`, see `recipes/musl.recipe` for
  a real example).
- Real multi-pass toolchain bootstraps (LFS-style — gcc built twice,
  etc.) should use separate recipes per pass (`gcc-pass1`,
  `gcc-pass2`), publishing only the final pass under the real
  name/repo. The DAG models build steps, not abstract packages.
- **`grootstrap` needs a rebuild, not yet done.** The old design (raw
  `tar` extraction into a target dir + chroot + register everything
  under an arbitrary bootstrap uid) predates the `GL_SCOPE_SYS` +
  base-repo design and is now more complicated than necessary. Once
  `gpkg`/`gstore` themselves exist on the target (still an unsolved
  bootstrapping-the-bootstrapper problem — how do they get there
  before a working package manager exists to install them with),
  every package including `base` ones can go through the same `gpkg
  -s -l` path; the repo alone routes symlinks correctly. No more
  special-casing base packages during bootstrap.

## Known open items / stubs

- `gpkg -f` / `-u` (merge/update) are unimplemented stubs. `-f`
  references an undefined `uid` variable (should be `uidn`) — will
  nil-concatenation-error the moment it's actually implemented.
- `gpkg -x`'s confirmation summary shows placeholder `unknown::pkg
  0.0.0 (unknown)` since it has no local manifest to read version/repo
  from for a bare package name. Would need `resolve_repo` called
  before building the summary, not just before the removal itself.
- `gpkg -x` doesn't print a completion message on success (`-l` does:
  "Completed with no errors."). Cosmetic inconsistency.
- Reinstalling an up-to-date package double-logs ("Installed X" +
  "staged from X" for the same event).
- `extra` vs `community` repos don't currently diverge in behavior —
  both just mean "not base" for symlink-routing purposes.
- `gl_relink_store`/`gl_rebuild_index` are full rescans of the store on
  every commit — fine at current scale, worth revisiting if large
  transactions become common (a dirty-set of touched packages instead
  of full rescans).
- Package manifests don't record which `SYS_PROFILE` they were built
  under — deliberately not added yet (would need a `pkg.c` /
  `gl_gpm2gpkg` change), flagged as a future option, not a gap that
  needs fixing now.
- A stray nested `lib/glacier/lib/glacier/` directory was spotted once
  in a `tree` listing — looked like a leftover `make install`
  artifact, not in any load path, harmless but worth a `rm -rf`
  eventually.

## Conventions worth knowing before editing

- C source generally uses tabs for indentation, though the codebase is
  inconsistent about it in places (some blocks use spaces). ALWAYS USE TABS WHEN POSSIBLE.
- `PATH_MAX`-sized stack buffers + `snprintf` is the standard pattern
  for path construction throughout; prefer it over `strncpy` for
  anything copying into a fixed buffer (see bug #7).
- Don't add convenience wrappers that silently touch a broader
  filesystem scope than what's asked — this codebase has already been
  bitten twice by exactly that class of mistake (`gl_relink_store`'s
  wipe, `gl_link_pkg`'s blind `unlink`), both only becoming dangerous
  once a path assumption (`links_path` is glacier-exclusive) quietly
  stopped holding.