libglacier-ng/REPO_DESIGN.txt

+--------------------------------------------+
| Remote repositories and signing            |
+--------------------------------------------+

STATUS: DESIGN, 2026-10-06. ROADMAP 3.1. Steps 1 (Ed25519, signify
formats, gsign), 2 (keyring, the local base rule, gbuild -k,
bootstrap carrying .sig, Everest's key) and 3 (grepo, the index,
refresh with -y) of section 9 DONE 2026-10-06, step 4 (-f, -u,
dependency closure, the cache) DONE 2026-10-07; sections 7, 5 and 6
say how they came out. Step 5 next. See CLAUDE.md.
Sections 1 and 10 are decided; the only open item is 10a's TLS library.

Adds AUTHENTICITY to glacier: SECURITY_DESIGN.txt section 1 deferred it
to "a future phase, tied to remote-repo support". This is that phase.
It also unblocks gpkg/syspkg -f (ROADMAP 2.2), -u (2.1), automatic
dependency closure (2.3) and glacier-bootstrap -r (GLACIER_BOOTSTRAP
section 5), and closes Finding 1 of SECURITY_REVIEW_2026-07-16.md (a
manifest that says PKG_REPO = "base" gets linked into /usr).

Companion documents: SECURITY_DESIGN.txt (integrity, refusal matrix),
PACKAGE_SCOPES.txt (repo routing), GLACIER_BOOTSTRAP.txt (pins).


+--------------------------------------------+
| 1. Decisions (the user, 2026-10-06)        |
+--------------------------------------------+

  a. Signatures: Ed25519, vendored (Monocypher's Ed25519 variant, which
     brings its own SHA-512), compiled into the private
     libglacier_verify.so next to sha256. Key and signature files are
     signify-compatible, so OpenBSD's signify can make and check them
     too. No new base dependency.
  b. What is signed: one INDEX per repository (repo.idx + repo.idx.sig),
     listing every package it serves with its sha256. Packages are
     verified through the hash in the signed index, i.e. through the
     existing sha256 path. The index has a serial; a client refuses an
     index older than the last one it accepted (rollback).
  c. Local base packages: once signing exists, `syspkg -l FILE` of a
     package whose manifest says base requires a signature by a key
     trusted for base. extra/community local installs stay
     sidecar-only (integrity). Manual installation keeps working: a
     builder makes a key and root adds it to the keyring.
  d. Users may add their own repositories and keys, for their own scope
     only (gpkg). Never read by syspkg.


+--------------------------------------------+
| 2. Keys and signatures (proposal)          |
+--------------------------------------------+

signify's formats, byte for byte (two lines: "untrusted comment: ..."
then base64):

    public key   "Ed" | keynum[8] | pubkey[32]
    signature    "Ed" | keynum[8] | sig[64]
    secret key   "Ed" | "BK" | rounds[4] | salt[16] | checksum[8] |
                 keynum[8] | seckey[64]

  - keynum is a random 8-byte key id; a signature names the key that
    made it, so verification picks the key by keynum and never tries
    every key in the keyring.
  - Signing is pure Ed25519 over the WHOLE message, as signify does. So
    we never sign a .gpkg directly (it would have to be read into memory
    whole). What gets signed is small:
      - a repository's repo.idx;
      - a local package's SIDECAR (NAME-VER.gpkg.sha256, the one line
        "sha256:<hex>"), as NAME-VER.gpkg.sha256.sig. Checking the
        sidecar against the file is the existing path; the signature
        makes the sidecar authentic. `signify -V -p KEY -m X.sha256`
        checks the same file.
  - Secret keys: glacier's tools read and write only UNENCRYPTED secret
    keys (rounds = 0, what `signify -G -n` makes): passphrase support
    needs bcrypt_pbkdf, which is more vendored crypto for a build-host
    convenience. A passphrase-protected key works by signing with
    signify itself (10b).

New tool `gsign` (C, built with gsha256sum; a build/maintainer tool,
later in "glacier-tools", never in base):

    gsign -G -p KEY.pub -s KEY.sec [-c COMMENT]   make a keypair
    gsign -S -s KEY.sec -m FILE [-x SIGFILE]       sign (default FILE.sig)
    gsign -V -p KEY.pub -m FILE [-x SIGFILE]       verify

gbuild gains `-k KEY.sec`: sign the sidecar it writes.


+--------------------------------------------+
| 3. Keyring and repository configuration    |
+--------------------------------------------+

System (root-owned; syspkg applies its trust checks to these files and
every ancestor, like its own lib/):

    /etc/glacier/keys/NAME.pub
    /etc/glacier/repos.conf

repos.conf (libconfig, like glacier.conf):

    repos = (
        { name = "base";  url = "https://pkg.example.org/base";
          keys = [ "everest-2026" ]; },
        { name = "extra"; url = "https://pkg.example.org/extra";
          keys = [ "everest-2026" ]; }
    );

  - A key is trusted ONLY for the repos that list it. That binding is
    what makes "base" mean something: the base index must be signed by a
    key listed for base, and a local base package's sidecar likewise.
    Several keys per repo allow rotation (old and new side by side).
  - Order = priority, when one name is served by several repos.
  - name follows the gl_valid_pkg_ident rules. url is https:// or
    file:// (local mirrors, tests); http:// is refused.
  - url is optional: an entry without one only says which keys are
    trusted for that repo, for a system installed and kept up by hand
    from signed local files (section 7).
  - Missing file = no remote repositories; -f/-u then refuse in one
    line. No defaults, as with KEEP_OLD.

User (gpkg only):

    ~/.config/glacier/repos.conf     ($XDG_CONFIG_HOME)
    ~/.config/glacier/keys/NAME.pub

  - Merged AFTER the system list (lower priority).
  - A user repo named "base", or named like a system repo, is refused.
  - System keys are trusted for system repos only; a user's keys for the
    user's repos only.


+--------------------------------------------+
| 4. The index                               |
+--------------------------------------------+

URL/repo.idx and URL/repo.idx.sig. libconfig, parsed only AFTER its
signature checks:

    REPO = "base";
    SERIAL = 2026100601L;          # strictly increases with each publish
    VALID_UNTIL = 1761955200L;     # unix time; required (10c)
    PKGS = (
        { NAME = "musl"; VER = "1.2.5"; FILE = "musl-1.2.5.gpkg";
          SHA256 = "sha256:3a5f..."; SIZE = 812345L;
          DEPS = (); CONFS = (); },
        ...
    );

  - REPO must equal the configured name the index was fetched for.
  - Every version a repo serves is listed (so a pinned or older version
    can still be fetched by -f NAME=VER later).
  - DEPS/CONFS repeat the manifest's, so dependency closure can be
    planned before downloading anything. After download, the manifest
    must match the entry (NAME, VER, PKG_REPO = REPO, DEPS, CONFS):
    a mismatch refuses the package. That is the Finding 1 rule for
    remote packages: a package is "base" because the base index lists
    it AND it says so, never because it says so.
  - FILE is a bare file name (gl_valid_pkg_ident rules plus ".gpkg"):
    no paths, no URLs.

New tool `grepo` (maintainer side, glacier-tools): `grepo -s KEY.sec
-r REPO DIR` reads every NAME-VER.gpkg in DIR (manifests, hashes),
writes DIR/repo.idx with SERIAL = the previous one + 1 (or the date
form), and signs it. The server is then any static file host.


+--------------------------------------------+
| 5. Client state and fetching               |
+--------------------------------------------+

Inside each scope's base directory (one more entry beside index, store,
...; same ownership and modes as the rest):

    <base>/repos/REPO.idx, REPO.idx.sig   the last ACCEPTED index
    <base>/cache/REPO/FILE                downloaded packages

Refresh (every -f / -u; indexes are small; 10d):
  1. curl --proto =https --proto-redir =https --fail (or a file:// copy)
     URL/repo.idx and .sig into a temporary file in <base>/repos/.
  2. Verify the signature with a key listed for that repo (by keynum).
  3. Parse. REPO must match. SERIAL must be > the stored one, or equal
     with identical bytes. VALID_UNTIL must be in the future.
  4. rename() over the stored copy (never written in place: bug 25).
Any failure keeps the old index and refuses the operation in one line:
  [x] Index for extra is older than the one already accepted.

Fetch: download FILE into the cache under a temporary name, hash it
while downloading, rename into place only if the hash matches the
index, then write the sidecar FROM THE INDEX and hand the cached file
to the existing install path (which re-hashes it, as today). The cache
is a download optimization, not a trusted store: a cache hit is hashed
again.

  As implemented (2026-10-06, step 3; the user chose a refresh flag,
  the date-form SERIAL and grepo in C):
  - grepo is C (src/grepo.c, libglacier-ng; built from source like
    gsign, installed to bin; not in base): `grepo -s KEY.sec -r REPO
    [-n SERIAL] [-e DAYS] DIR`. SERIAL = max(previous + 1, today (UTC)
    * 100 + 1), e.g. 2026100601; -n must still be above the previous
    one. VALID_UNTIL = now + DAYS (default 30, 1..3650). Packages are
    sorted by file name; each must be a regular file, its manifest must
    say REPO and its name must be NAME-VER.gpkg (VER as the store names
    versions, "%d.%d.%d"). An existing DIR/repo.idx must be a
    well-formed index of REPO (its SERIAL is the previous one). It is
    written to a mkstemp file, signed, and both renamed into place, the
    .sig first.
  - The index format is section 4's, with DEPS and CONFS as lists of
    { NAME = "x"; VER = "1.0.0"; } (the manifest's PKG_DEPS shape).
    gl_index_parse (pkg.c) takes the BYTES the signature was checked
    over, never a path, and is strict: exactly REPO, SERIAL > 0,
    VALID_UNTIL > 0, PKGS; each package exactly NAME, VER, FILE, SHA256,
    SIZE >= 0, DEPS, CONFS; FILE must be exactly "NAME-VER.gpkg"; SHA256
    "sha256:" + 64 lowercase hex; no NAME + VER twice. A NUL or an '@'
    anywhere is refused before libconfig sees it ('@include').
    GL_INDEX_ERR_REPO when REPO is not the repo it was fetched for.
  - `syspkg -y` / `gpkg -y` (--refresh) refresh every repos.conf entry
    with a url and stop (decision 10d still holds for -f/-u). gpkg
    reads the SYSTEM repos.conf and keyring until step 6 (user repos),
    and stores into the user's base. No repos.conf, or no entry with a
    url: `[x] No remote repositories are configured.` (exit 1).
  - <base>/repos is made by gl_scope_state_dir (transaction.c): the
    base must be the scope's; repos is created with the scope's mode
    (0755 root / 0700 uid) and refused unless it is a real directory
    with exactly that owner and mode. Files are 0644 / 0600. -y holds
    the scope's lock (gl_begin_transaction on a live context); under
    -R the indexes go under the install root, checked with the install
    root's OWN keyring (DIR/etc/glacier; changed 2026-10-07, see
    "Keyrings under -R" below).
  - file:// is read directly (absolute paths only); https:// goes
    through `curl -sf --proto =https --proto-redir =https`. 64 MiB at
    most (GL_SIGN_MAX_MSG). Every repo is tried; exit 1 if any failed.
  - One line per repo:
      [✓] Index for base: serial 2026100601, 11 packages.
      [i] Index for base is up to date: serial 2026100601.
      [x] Cannot fetch the index for base.         (a missing .sig too)
      [x] Index for base is not signed by a key trusted for base.
      [x] Malformed index for base.
      [x] Index fetched for base is the index of another repository.
      [x] Index for base is older than the one already accepted.
      [x] Index for base has the serial of the one already accepted, but other contents.
      [x] Index for base has expired.
      [x] Malformed /glacier/sys/repos/base.idx.   (left for the admin)
  - The accepted copy is trusted by where it lives (root's or the
    user's private repos dir): it is parsed for its SERIAL, not
    re-verified.
  - Test: glacier-ng/tests/repo_alpine.sh (103 checks; 18 mutants, 12 C
    and 6 Lua, all caught).


+--------------------------------------------+
| 6. -f, -u and dependency closure           |
+--------------------------------------------+

  -f NAME... (ROADMAP 2.2): like -l, with the file found by name in the
     accepted indexes (highest version, first repo by priority). Missing
     dependencies are added automatically (ROADMAP 2.3's closure): the
     highest version satisfying the minimum, recorded `auto`. The
     summary lists them, marked as dependencies. Conflicts and every
     2.3 check run as today, before the prompt.
  -u (ROADMAP 2.1): every installed package whose repo is configured and
     whose index has a higher version is replaced, as -lO does (backups,
     KEEP_OLD). Packages from repos not configured are left alone.
     `-u NAME...` limits it to those names.
  syspkg -f/-u fetch only from system repos; gpkg -f/-u from system and
  the user's repos. Base is still refused in user scope (library rule).

  As implemented (2026-10-07, step 4; M.fetch / M.upgrade in
  glacier-ng/lib/gpkg_common.lua). The user decided three details:
  - Which repo: the FIRST repo by repos.conf order that serves NAME, at
    its highest version. A lower-priority repo never wins by publishing
    a bigger version. -u stays within the installed package's repo.
  - A dependency installed at too old a version is UPGRADED in the same
    transaction from its own repo (backed up per KEEP_OLD, like -lO),
    keeping its auto/explicit mark; the summary shows
    "old -> new (dependency)".
  - Cache: downloads stay in <base>/cache/REPO while a run is declined
    or refused; the files a commit installed are deleted after it.
  The rest:
  - Every -f/-u runs -y's refresh first; if any repo fails, the
    operation is refused (decision 10d). The accepted indexes are then
    read from <base>/repos (gl_index_parse again; trusted by location).
  - Closure: for each planned package, each DEPS entry of its INDEX
    entry is met by a planned package, an installed one at >= VER
    (this scope; for a user, the system's index too), else upgraded (an
    installed one too old) or added (pick(): first repo serving it,
    recorded auto). What can't be met is left to gl_check_plan, which
    prints the 2.3 refusal. The summary marks additions
    "(dependency)".
  - Download: gl_scope_cache_dir (transaction.c) makes and checks
    <base>/cache and <base>/cache/REPO as gl_scope_state_dir checks
    repos (owner, exact mode, not a symlink). A cached file whose hash
    matches the index is reused; otherwise file:// is read, https://
    goes through `curl -sf --proto =https --proto-redir =https` on a
    pipe; at most SIZE + 1 bytes are ever read, hashed on the way, into
    a dot-named temporary, renamed into place only if size and hash
    match. 0644 / 0600.
  - No sidecar is written: the index's sha256 is handed to
    verify_and_copy directly (its `vouched` argument), which re-hashes
    while copying into the private dir, as for -l. A base package needs
    no .sig of its own: the base index vouches for it.
  - The copy's manifest (library-read) must say exactly what the index
    entry says: NAME, VER, PKG_REPO = the repo, DEPS and CONFS (same
    order, "NAME MAJ.MIN.PAT"). Else, and for a bad hash or size:
      [x] extra::evil 1.0.0 does not match its index.
    and the cached file is deleted. Nothing to download:
      [x] Cannot fetch extra::foo 1.0.0.
    No repo serves a name:
      [x] foo is not in any repository.
  - -f of a name installed at another version needs -O (as -l);
    dependency upgrades and -u don't (meta.may_replace). -O still also
    means displacing foreign files.
  - -u [NAME...]: a named package not installed, or whose repo has no
    url, is refused (`[x] foo is not installed.` / `[x] community::foo
    is not in any configured repository.`); plain -u skips the latter.
    Nothing newer: `[✓] Nothing to do.`, exit 0.
  - gpkg: user scope never fetches from base (base indexes are not even
    considered, so `[x] core is not in any repository.`); the system's
    repos.conf until step 6; indexes and cache in the user's base.
  - syspkg -R DIR -f/-u: indexes and cache under DIR, DIR's keyring.

  Keyrings under -R (decided 2026-10-07, for glacier-bootstrap
  configuring the system's repositories): -y, -f and -u read
  DIR/etc/glacier/repos.conf and DIR/etc/glacier/keys, with the same
  trust checks, plus: each file's real path must lie inside DIR
  (`[x] Refusing P: it resolves outside DIR.`), so a symlink in DIR
  can't make the host's file stand in for DIR's. The indexes are DIR's
  state and DIR checks them with its own keys from then on, and a host
  with no glacier configuration can still bootstrap. A local `-R -l`
  base package's .sig is still checked with the HOST's keyring (section
  7). Tests: repo_alpine.sh / fetch_alpine.sh give the image its own
  keyring; bootstrap_alpine.sh sections 14-16.
  - Test: glacier-ng/tests/fetch_alpine.sh (84 checks; 16 Lua mutants
    and 1 C mutant, all caught except the download-level hash check,
    which is EQUIVALENT: the private copy's re-hash refuses the same
    package and the mismatch path deletes the cached file. Kept as
    defence in depth: a bad download never sits in the cache under its
    final name).
  - Known gap: a run killed mid-download leaves its dot-named temporary
    in the cache (never read; the next fetch writes its own).


+--------------------------------------------+
| 7. Local installs (decision 1c)            |
+--------------------------------------------+

  syspkg -l FILE, manifest says base:
      FILE.sha256 must match (as today) AND FILE.sha256.sig must verify
      with a key listed for base in repos.conf. Otherwise:
        [x] base::musl is not signed by a key trusted for base.
  syspkg -l, other repos; gpkg -l: unchanged (sidecar only).
  glacier-bootstrap -d: carries each package's .sig into its private
      copies, so syspkg -R checks it. The keyring checked is the HOST's
      /etc/glacier (the build host's trust, as KEEP_OLD is today).
  Consequence for ~/Projects/everest: it needs a key; container.sh signs
  every base package (gbuild -k); the build host's /etc/glacier/keys and
  repos.conf list it. The image ships the PUBLIC key and a repos.conf so
  the installed system trusts its own base (since 2026-10-07 in the
  everest-repos package, split out of everest-bootscripts).

  As implemented (2026-10-06, step 2):
  - The repo is the one the LIBRARY reads from the verified copy
    (gl_read_pkg_manifest, as gl_install_pkg), never gpkg_common's
    pattern-matching read_manifest_fields: a manifest comment
    `# PKG_REPO = "extra"` would make that one say extra.
  - The sidecar is read ONCE; those bytes are both what the copy's hash
    is checked against and what the signature is checked over. So
    glacier-bootstrap copies the sidecar's bytes as read (it used to
    write a new one from the hash) next to the .sig.
  - The key is the one, among those listed for base, whose keynum the
    signature names. Any other problem with the signature (missing,
    malformed, other bytes, other key) is the one-line refusal above,
    before the prompt; one refused package refuses the batch.
  - repos.conf is read by gl_conf_repo_keys (pkg.c), strictly: `repos`
    is the only setting; each entry has name, keys and optionally url,
    nothing else; names and key names follow gl_valid_pkg_ident; a
    duplicate name, or a url other than https://X or file://X, is
    refused. Any of that: `[x] Malformed /etc/glacier/repos.conf.`
  - repos.conf and each listed key, every ancestor of them and of their
    real paths, must be root-owned and not group/world-writable:
    `[x] Refusing F: P is not root-owned, or is group/world-writable.`
    A listed key that is missing (`Cannot read F.`) or not a public key
    (`Malformed F.`) stops the run: the admin's configuration is at
    fault, not the package. Read only if some package is base.
  - gbuild -k KEY.sec signs the sidecar (gsign from $GSIGN_BIN,
    build/bin, or PATH), trying the key before building; a build
    without -k removes a stale .sig. tests/mkpkg -k likewise.
  - Test images (Dockerfile.*) carry a TEST key, glacier-test, listed for
    base, its secret at /root/glacier-test.sec. tests/keyring_alpine.sh
    covers this section (17 mutants, all caught).
  - KEEP_OLD backups are re-packed from the store, so a base backup has
    no signature: `syspkg -lO` refuses it until the restore interface
    exists (decision 10f).


+--------------------------------------------+
| 8. Where the checks live                   |
+--------------------------------------------+

Signature verification is C (libglacier_verify.so). The POLICY (which
key for which repo, the base rule, rollback state) lives in the CLIs'
shared module, like hash verification today, so a third-party caller of
gl_install_pkg still isn't checked. ROADMAP 3.2 (move verification into
the library) now covers signatures too.


+--------------------------------------------+
| 9. Order of work                           |
+--------------------------------------------+

  1. Vendored Ed25519 + signify formats in libglacier_verify.so; gsign;
     tests including interop with signify (Alpine's `signify` package)
     and hostile inputs (bad base64, wrong keynum, truncated files).
  2. Keyring + repos.conf + the local base rule (section 7); gbuild -k;
     glacier-bootstrap carries .sig; Everest gets a key.
  3. Index: grepo; refresh, verify, rollback state. DONE 2026-10-06
     (section 5, "As implemented"; syspkg/gpkg -y).
  4. -f with closure; then -u. DONE 2026-10-07 (section 6, "As
     implemented").
  5. glacier-bootstrap -r against an index (pins stay optional extra
     assurance).
  6. User repos.
Each step is tested in Docker (file:// repos need no network) and
mutation-checked, as usual.


+--------------------------------------------+
| 10. Decisions on the details (2026-10-06)  |
+--------------------------------------------+

  a. Fetching on Everest: curl plus one TLS library in base. Signatures
     make packages authentic regardless; TLS also hides WHAT is fetched
     and stops a network attacker serving a stale (still valid) index.
     STILL OPEN: which TLS library (BearSSL or LibreSSL); decide when
     packaging it. Until then, everything is built and tested with
     file:// repos and Alpine's curl.
  b. No encrypted secret keys in gsign (no bcrypt_pbkdf). A
     passphrase-protected key signs with signify itself.
  c. VALID_UNTIL is REQUIRED; an expired index is refused. The
     maintainer sets the lifetime; grepo defaults to 30 days.
  d. Indexes are refreshed on every -f / -u. Offline, they refuse.
  e. Everest's SECRET key lives outside the project tree:
     ~/.config/everest/keys/everest-2026.sec (EVEREST_KEYS), made by
     `everest.sh keygen`, mounted read-only into the build container.
     The public key ships in everest-bootscripts (/etc/glacier/keys).
  f. Base backups (KEEP_OLD) are refused by `syspkg -lO` for want of a
     signature; accepted for now. The restore interface (KEEP_OLD's
     "later") can trust backups glacier made in its root-owned backup
     dir.