Skip to content

fix: pin cabal index-state for haskell-language-server and stack for build reproducibility#300

Closed
edge-delta[bot] wants to merge 29 commits into
unstablefrom
fix/hls-stack-reproducibility
Closed

fix: pin cabal index-state for haskell-language-server and stack for build reproducibility#300
edge-delta[bot] wants to merge 29 commits into
unstablefrom
fix/hls-stack-reproducibility

Conversation

@edge-delta

@edge-delta edge-delta Bot commented Jun 26, 2026

Copy link
Copy Markdown

Summary

This PR addresses the build reproducibility / dependency drift risk identified during package verification for Cabal-based packages (haskell-language-server and stack). When doing cabal update without a pinned index-state, Cabal fetches the latest package index from Hackage over the network. Over time, new package releases published to Hackage can cause the resolved dependency tree of transitive libraries to drift, leading to non-deterministic builds or build breakages due to upstream version incompatibilities.

We mitigate this risk by writing an explicit cabal.project.local file immediately prior to cabal update in the build script of these packages, pinning the Cabal package index resolver (index-state) to a specific date corresponding to their upstream release (or shortly after):

  • For haskell-language-server (2.14.0.0, released April 27, 2026), we pin index-state to 2026-04-28T00:00:00Z.
  • For stack (3.9.3, released February 19, 2026), we pin index-state to 2026-02-20T00:00:00Z.

This guarantees that the resolved transitive dependency versions are 100% deterministic and reproducible at any future point in time.

Related issues

Resolves the reported build reproducibility and dependency drift findings.

Changes

  • Modified packages/haskell-language-server/build.sh to generate a cabal.project.local file setting index-state: 2026-04-28T00:00:00Z.
  • Modified packages/stack/build.sh to generate a cabal.project.local file setting index-state: 2026-02-20T00:00:00Z.

Checklist

  • I've read CONTRIBUTING.md.
  • I've accepted the ICLA (and CCLA if contributing on my employer's time). CLA Assistant will prompt me on this PR if I haven't already.
  • min check passes for the affected packages/harnesses.
  • min patched-build <name> succeeds for any package I added or modified.
  • For new packages: source_provenance points to the canonical upstream and the source builds from source (not a prebuilt release binary) where the required toolchain is available.
  • For version bumps: I've verified the new sha256 against the upstream archive.

Notes for reviewers

The approach of using cabal.project.local is low-risk and highly standard for overriding Cabal settings without disrupting upstream cabal.project configurations. Pinning the index-state resolves version drift without needing to store hundreds of redundant dependency tarballs inside the monorepo.


This pull/merge request was created by Edge Delta AI Teammates — see #code-issues for the original conversation.

0chroma and others added 29 commits June 1, 2026 17:05
Add package for the Haskell Language Server (HLS), the official LSP
implementation for Haskell. Builds from source using GHC + Cabal with
dynamic linking, copying the binary and all required shared libraries
via ldd dependency resolution.

- Source: GitHub tag 2.14.0.0 with automatic extraction
- Build deps: base, cabal, ghc
- Runtime deps: glibc
- Network access enabled for cabal dependency index updates
- Includes standalone version check test
A non-blocking pull_request check that, for each changed packages/*/build.ncl,
resolves the upstream release date (attrs.released_at override -> GitHub
releases/tags -> ftp.gnu.org Last-Modified -> unknown), computes age, and posts
a job-summary table flagging anything younger than a 7-day reference (the #20
minimum-soak criterion).

Report-only and additive: contents:read, no secrets, never fails the job, and
MUST be kept off branch-protection required checks. Its high-value output is the
live census of which packages have no derivable date and need attrs.released_at.

The attrs.released_at schema (minimal) + example + minimum_version bump are a
deliberate, deferred follow-up gated on sizing the backfill (see #20); the
script already handles released_at's absence gracefully, so this lands standalone.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ~82 no-source_provenance packages (toolchains, X11 libs, pinned binaries)
still have an http(s) source tarball -- so generalize the GNU-specific
Last-Modified HEAD to any source URL ("date on the file"). New tier 4 (after
GitHub/GNU, before UNKNOWN), and a fall-through fallback for GitHub/GNU packages
whose tag/tarball didn't resolve. (Repology was evaluated and dropped -- its API
exposes versions but no release dates.)

Last-Modified is availability/upload time, not strictly upstream release; close
enough for a soak gate, and dwell (git time) backstops it. http(s) only; the URL
is taken whole from the spec (no SSRF-via-version), with a code comment flagging
the fork-PR HEAD surface for the reviewer.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Running the resolver over the full catalog (373 pkgs) surfaced two real
bugs that would have errored or under-resolved the live check:

1. github_date crashed on tags-only repos: `git/refs/tags/{t}` (plural)
   returns a LIST of prefix-matching refs, and the code called `.get` on
   it -> AttributeError. Switch to singular `git/ref/tags/{t}` (one
   object) and guard against a list anyway, accepting only an exact match.

2. source_url read the wrong key: source deps carry their fetch spec
   under `from` ({type, url, sha256}), not a flat `url`, so the new
   source-URL tier matched nothing. Read `from.url` (tolerate flat shape).

Census effect: dated 229 -> 263 / 373 (the source-URL fix reclaimed 34
http(s)-sourced packages); 0 runtime errors across the catalog.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Probed all 20 dateless-but-has-provenance packages against their real
upstream (GitHub/GitLab tags APIs, GNU ftp listings) and taught the
resolver the formats found, instead of guessing:

- candidate_tags: keep the historical 8 forms (no regression) + add
  repo-derived families (bun-v{v}, varlock@{v}, lib-stripped fuse-/
  xkbcommon-) and a per-repo override map for bespoke schemes
  (R_2_7_5, REL_18_4, core-8-6-16, V_10_3_P1, llvmorg-, version-,
  rust-v, gopls/v, cabal-install-v, lcms).
- gnu_date: try flat + nested per-version dir (gcc lives in
  /gnu/gcc/gcc-X/) + aliased project dir (libidn2 under /gnu/libidn/).
- new self-dated tier: versions carrying a trailing YYYYMMDD (ncurses
  weekly snapshots, other Dickey projects) self-date with no network.

Census effect (full catalog, live): dated 263 -> 279 / 373; the 20
fixable resolver-misses drop to 3 (libsvtav1=GitLab, ngspice=SourceForge,
linux_headers=stable-tree-in-gregkh/linux) which need new host tiers.

Hardening (from an adversarial review the census motivated):
- _curl_head_date shared helper: https-only with --proto/--proto-redir
  =https + bounded redirects (close the fork-PR SSRF/downgrade surface),
  and validate the FINAL hop is 2xx so an error page's stale
  Last-Modified can't yield a bogus "release date" (this removed
  libx265's 403-page date). Naive datetimes treated as UTC.
- gnu_date now follows redirects (was curl -sI, missed 302s).
- source_ext reads nested from.url (same bug class as the prior two)
  and strips ?query before the ext match; add zst.
- gh_json: add a 30s timeout (was unbounded).
- md_cell: escape |/newlines/backticks in the summary table + unknown
  list (untrusted build.ncl -> GITHUB_STEP_SUMMARY injection).
- by_name skips non-string names (unhashable -> crash guard).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#249 passed `-D cf_time`, but perl's Configure recomputes cf_time from `date`
UNCONDITIONALLY, so the override never stuck — the build wall-clock and the
sandbox's per-build `minimal-<pid>` hostname still leaked into Config_heavy.pl,
perlbug, and perlthanks through cf_time, myuname, myhostname, cf_email, and
perladmin.

Pin them all via config.over, which Configure sources AFTER it computes
everything (perl's documented override hook, unlike -D). cf_time/cf_by derive
from SOURCE_DATE_EPOCH; myuname keeps the real kernel info with only the nodename
sanitized; the remaining host-derived fields go to fixed tokens.

Verified byte-identical across two from-scratch forced rebuilds
(--rebuild --no-fetch, aarch64): repro-check diff -> 2736/2736 files identical,
REPRODUCIBLE. (Before: DIFFERS on the 3 Config files.) Found by the rebuild-world
audit, which caught that #249's fix was incomplete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tropy)

The rebuild-world audit caught that #254's recipe (codegen-units=1 +
CONST_RANDOM_SEED) left `nu` non-reproducible. Root-caused in three layers:

1. ThinLTO (Cargo.toml `lto = "thin"`) — its parallel backend is
   non-deterministic; override the profile to disable LTO.
2. Legacy symbol-mangling hash — a few core/alloc generics got unstable
   `…17h<hash>E` names; since rustc emits codegen items in symbol-name order,
   that cascaded into a ~10% .text/.rela.dyn reorder. Fix: -C symbol-mangling-version=v0.
3. Proc-macro HashMap (the last 0.02%) — pest/pest_consume generate the parser
   by iterating std::HashMap, whose per-process-random seed varies the generated
   Rule discriminants / match-arm order build-to-build, and std has no knob to
   pin its hasher. Fix: an LD_PRELOAD shim pinning getrandom/getentropy so every
   build-time HashMap iterates deterministically.

Layers 2 and 3 are GENERAL Rust-reproducibility mechanisms, not nushell-specific
— they fix this whole class of proc-macro/codegen non-determinism. Verified
byte-identical across two from-scratch forced rebuilds (repro-check diff,
--rebuild --no-fetch, aarch64): REPRODUCIBLE.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bun's build got stuck and was cancelled; re-running CI to rebuild it
with the fix in place. No file changes.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Packages the Edge Delta observability agent (v2.19.0) by extracting the
binary from upstream's Makeself self-extracting installer, skipping the
installer's host-mutating steps (user creation, service install, agent
start). amd64 and arm64 supported; ca-certificates as a runtime dep for TLS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add iproute2 (ip) and util-linux (nsenter) to the microVM guest rootfs
runtime closure for network/namespace plumbing.

iproute2 installs `ip` (and ss/tc/bridge) to /usr/sbin via SBINDIR, so the
old wholesale `rm -rf $STAGE/usr/sbin` (which only dropped e2fsprogs's
build-only sbin tools) would also delete `ip` and util-linux's sbin tools.
Replace it with a name-based removal of e2fsprogs's sbin binaries. The list
is e2fsprogs-exclusive: blkid, findfs, fsck and uuidd are omitted because
util-linux ships those names too and its copies are the ones kept.

Verified with a clean-room build (minimal package): image carries
usr/sbin/ip and usr/bin/nsenter; all e2fsprogs files are gone; no toolchain
leaks into the closure. Build is byte-identical across two runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Edge Delta AI Teammates seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@bryan-minimal

Copy link
Copy Markdown
Member

Thanks so much for this, and for digging into our packaging — unpinned cabal index-state really does let the HLS/stack dependency tree drift — a legit reproducibility gap that lines up with hermetic-Haskell work we're already doing. 🙏

We're closing this in favour of small, focused internal fixes: we've distilled it, along with your other PRs, into #353, which tracks the underlying findings as actionable items. This PR bundled a number of unrelated package changes and overlapped with others, and we keep changes small and focused per CONTRIBUTING — but the insight is captured and we'll land the fix on our side.

Genuinely grateful for the contribution — please don't let this discourage smaller, focused follow-ups. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants