Previous Home Next

4.1. Sources

Source archives should be staged in a dedicated directory using a repeatable POSIX shell fetch workflow.

Goal: place source archives in LBI_SOURCES (or another deliberate directory) before package builds begin.

This section introduces a POSIX sh helper that uses curl for source archives and git archive for pinned source-control snapshots. It is intended for repeatable source staging, not for package management.

All command examples in this section are intended for a normal user shell, not a root shell.

Provided Files

The source staging files are distributed with this website and can be downloaded here:

The helper accepts URLs directly, or a manifest file containing one source per line. The patch zip contains the book's local patch files; extract it into LBI_SOURCES before building packages that list a distributed patch in their input assumptions.

Direct URL Usage

After downloading the script into your current working directory, run it with one or more source URLs:

sh ./fetch-sources.sh "https://mirror.example.org/sources/pkg-a-1.0.tar.xz" "https://mirror.example.org/sources/pkg-b-2.3.tar.gz"

When a source needs an explicit local filename, place the output name immediately after that URL:

sh ./fetch-sources.sh \
    "git+https://git.example.org/pkg#0123456789abcdef" \
    "pkg-0123456789ab.tar.gz"

By default, downloads are written to LBI_SOURCES when that variable is set; otherwise they are written to ./sources.

Manifest Usage

A manifest is a plain text file with one entry per line:

# URL [OUTPUT_NAME]
https://mirror.example.org/sources/pkg-a-1.0.tar.xz
https://mirror.example.org/sources/pkg-b-2.3.tar.gz pkg-b.tar.gz
git+https://git.example.org/pkg#0123456789abcdef pkg-0123456789ab.tar.gz

The provided starter manifest currently includes:

musl-1.2.6.tar.gz
llvm-project-22.1.3.src.tar.xz
mimalloc-v3.3.0.tar.gz (downloaded from the upstream v3.3.0.tar.gz tag URL but saved with this explicit output name)
linux-7.0.tar.xz
om4-6.7.tar.gz
ncurses-6.6-20260418.tgz
libedit-20251016-3.1.tar.gz
bheaded-0.0.1-mk2.tar.gz
sbase-c1341583c963.tar.gz (generated from the pinned upstream git commit in sources.manifest)
BSD-Diffutils-main.zip
file-5.47.tar.gz
zstd-1.5.7.tar.gz
bsdgrep-master.zip
zlib-ng-2.3.3.tar.gz
byacc-20260126.tgz
bfs-4.1.tar.gz
bmake-20260406.tar.gz
awk-20251225.tar.gz
samurai-1.3.tar.gz
make-4.4.1.tar.gz
bsdpatch-v0.99.1.tar.gz
mandoc-1.14.6.tar.gz
bsdsed-v0.99.2.tar.gz
libarchive-3.8.7.tar.xz
xz-5.8.3.tar.xz
gettext-tiny-0.3.3.tar.xz
ubase-e8249b49ca3e.tar.gz (generated from the pinned upstream git commit in sources.manifest)
Python-3.14.4.tar.xz
pigz-2.8.tar.gz
dash-0.5.13.3.tar.gz
oksh-7.8.tar.gz
bc-7.0.3.tar.xz
pkgconf-2.5.1.tar.xz
iana-etc-20260409.tar.gz
libressl-4.2.1.tar.gz
cacert-2026-03-19.pem
curl-8.19.0.tar.xz
cmake-4.3.2.tar.gz
flex-2.6.4.tar.gz
sqlite-autoconf-3530000.tar.gz
flit_core-3.12.0.tar.gz
rustc-1.95.0-src.tar.xz
coreutils-0.8.0.tar.gz
red-v1.0.2.tar.gz
dinit-0.21.0.tar.xz
limine-5be26a73d7b7.tar.gz (generated from the pinned upstream git commit in sources.manifest)

Then fetch everything listed in that file:

sh ./fetch-sources.sh --manifest ./sources.manifest

Checksum File

The provided sources.b2sums file records BLAKE2b checksums for the files named in sources.manifest, including explicit output names such as mimalloc-v3.3.0.tar.gz and the generated git snapshot archives. It also includes the checksum for linux-by-intent-patches.zip.

Keeping the manifest and checksum file in sync makes source staging auditable and repeatable as this chapter grows.

Script Behavior

The helper uses curl --fail --location --retry 3 --continue-at - so downloads follow redirects, retry on transient failures, and resume partial files when possible.

Entries beginning with git+ must include a pinned ref after #. The helper clones that repository, checks out the ref, and writes a deterministic gzip archive using git archive and gzip -n.

Existing non-empty files are skipped by default. Use --force to re-download existing archives.

Once source archives are staged, later build steps can consume them from LBI_SOURCES without mixing fetch policy into each package section.

Previous Home Next