Skip to content

feat: autofit[nss] install extra (Phase 4 of nss_first_class_sampler)#1277

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/nss-install-extra
May 16, 2026
Merged

feat: autofit[nss] install extra (Phase 4 of nss_first_class_sampler)#1277
Jammy2211 merged 1 commit into
mainfrom
feature/nss-install-extra

Conversation

@Jammy2211
Copy link
Copy Markdown
Collaborator

Summary

Phase 4 of the nss_first_class_sampler roadmap — pip install autofit[nss] is now the single command that installs af.NSS and all its dependencies. Replaces the multi-step install saga documented in z_projects/profiling/FINDINGS_v3.md. Closes #1276.

pip install autofit[nss]
python -c "import autofit as af; af.NSS()"  # works

Approach (Option C from the prompt's matrix)

The original Phase 4 prompt recommended Option A — full vendoring of ~1300 LOC of nss + blackjax/ns into PyAutoFit. Investigation revealed modern pip 23+ handles URL-direct deps cleanly, so the lighter Option C path (pip extra with pinned git+ URLs) works without ongoing re-vendor maintenance burden. Verified empirically on a fresh venv.

The pin choice matters

Initial try with HEAD of handley-lab/blackjax (5dbf89a9) failed: HEAD adds numpy>=1.25 which conflicts with PyAutoFit's anesthetic==2.8.14 (numpy<2.0). Pinned to ef45acd2f instead — the May 2026 "Merge PR #60 — double_compile" commit, which is what was installed locally and validated through the entire Phase 1-3 work. The same anesthetic cap will release when Python 3.13 anesthetic>=2.9 takes over; revisit the blackjax pin in that bump.

What's in the extra

nss = [
    "fastprogress<1.1",
    "blackjax @ git+https://github.com/handley-lab/blackjax.git@ef45acd2f2fa0cca15adbdcd3ff7cb3a98987cb5",
    "nss @ git+https://github.com/yallup/nss.git@69159b0f4a3a53123b9eec7df91e4ed3885e4dc4",
]

The fastprogress<1.1 pin avoids fastprogress 1.1.5's transitive pull of python-fasthtml + starlette + uvicorn etc. The two URL deps win the resolver fight against mainline blackjax via pip's direct-URL precedence.

CI

New workflow at .github/workflows/nss_install_smoke.yml:

  • Triggers on pull_request (catches local pyproject.toml regressions before merge).
  • Runs on 0 3 * * 0 cron (Sundays 03:00 UTC — catches upstream drift past the pinned SHAs).
  • Fresh Ubuntu venv, pip install -e PyAutoFit[nss], then a Python smoke that imports autofit, nss.ns, blackjax.ns.adaptive (asserting the fork's init attribute exists), and instantiates af.NSS().

Test plan

  • pytest test_autofit1258 passed, 1 skipped (1252 baseline + 6 NSS unit tests from Phase 2-3; no regression).
  • Fresh python -m venv /tmp/nss_install_smoke && pip install -e PyAutoFit[nss] on Python 3.12 — completes in ~3 minutes, installs blackjax 0.1.0b1.dev85+gef45acd2f, nss 0.1.0, jax 0.7.1, numpy 1.26.4, and the rest of the transitive tree without dependency conflicts.
  • Fresh-venv import autofit as af; af.NSS(n_live=10, ...) succeeds end-to-end. blackjax.ns.adaptive.init exists (canonical check for the handley-lab fork).
  • NSS unit tests (15/15) still pass — the updated ImportError text still matches the existing pytest.raises(ImportError, match=...) regex.

API Changes

Pure additive at the install layer (autofit[nss] extra). The af.NSS ImportError message changes to reference the new install command. No code-API changes.
See full details below.

Full API Changes (for automation & release notes)

Added

  • autofit[nss] install extra in PyAutoFit/pyproject.toml. Pulls fastprogress<1.1 + handley-lab/blackjax fork at SHA ef45acd2f + yallup/nss at SHA 69159b0f.

Changed Behaviour

Migration

Users who previously followed the manual install recipe from FINDINGS_v3.md can switch to pip install autofit[nss]. No action required for existing installs — both paths produce a working af.NSS.

🤖 Generated with Claude Code

Adds a `[project.optional-dependencies] nss` extra so `pip install autofit[nss]`
becomes the single safe install command for af.NSS, replacing the
multi-step install saga documented in FINDINGS_v3.md.

The extra pins three deps in dependency order:
- `fastprogress<1.1` — fastprogress 1.1.5 transitively pulls
  python-fasthtml + starlette + uvicorn + httpx + websockets (none
  needed at runtime).
- `blackjax @ git+https://github.com/handley-lab/blackjax.git@ef45acd2f` —
  the nss-required fork pinned to the May 2026 "Merge PR #60 — double_compile"
  commit. Locally validated through the entire Phase 1-3 nss_first_class_sampler
  work. HEAD as of pin time (5dbf89a9) adds `numpy>=1.25` which conflicts
  with autofit's `anesthetic==2.8.14` (numpy<2.0); bump both pins together.
- `nss @ git+https://github.com/yallup/nss.git@69159b0f` — current HEAD;
  byte-identical to the locally-validated install.

Approach picked from the Phase 4 prompt's option matrix: Option C
(pip extra with pinned URL deps). The original prompt recommended
Option A (full vendoring, ~1300 LOC). Investigation revealed modern
pip 23+ handles URL-direct deps cleanly, so the lighter Option C path
works without ongoing re-vendor maintenance burden.

Updates the af.NSS ImportError message to reference the new install
command. Adds a CI workflow that runs a fresh-venv install + import
smoke on every PR + a weekly cron — catches both local pyproject.toml
regressions and upstream drift past the pinned SHAs.

Closes #1276

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label May 16, 2026
@Jammy2211 Jammy2211 merged commit 9376708 into main May 16, 2026
3 of 5 checks passed
@Jammy2211 Jammy2211 deleted the feature/nss-install-extra branch May 16, 2026 09:48
Jammy2211 pushed a commit that referenced this pull request May 17, 2026
…inline)

The Tests workflow has been red on `main` since 2026-05-16 09:48 UTC,
the moment PR #1277 merged the `autofit[nss]` install extra. The 12
tests under `test_autofit/non_linear/search/nest/nss/` hit the script's
own ImportError guard on `af.NSS()` because the CI install step only
installs `[optional]`.

A naïve fix (combine `[optional,nss]` into one pip install) hit a real
dependency conflict — both extras pin `blackjax` but to different
versions:

  - [optional] pins `blackjax>=1.2.0` (mainline, PyPI)
  - [nss]      pins handley-lab fork @ ef45acd2 (~0.1.0b1.dev85+)

The fork carries the `blackjax.ns.adaptive.init` entrypoint that
mainline 1.2.x lacks, so it's not a "use the older one" merge — they're
genuinely incompatible. pip rightly refuses `[optional,nss]` with
`ResolutionImpossible`.

Resolution: split into parallel jobs.

- `unittest`: installs `[optional]` and runs the full test suite
  EXCLUDING `test_autofit/non_linear/search/nest/nss/`. This keeps
  `test_blackjax_nuts.py` (which requires mainline blackjax 1.2+) green.
- `unittest_nss` (new): installs `[nss]` alone in a fresh env and runs
  ONLY the NSS test suite. Matrix is python 3.12 / 3.13 to match
  the main `unittest` job.

Both jobs need to pass for the `Tests` workflow to be green. The
existing `nss_install_smoke.yml` workflow stays as the weekly-cron
upstream-drift canary on the [nss] git pins; this new job is the
PR/push-time test gate.

Fixes the failing Tests check observed on runs 25958838630 (#1277
merge), 25959492149 (#1278 merge), 25998822676 (#1280 revert merge),
and 25999018708 (the first attempt at this fix that revealed the
extras conflict).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: autofit[nss] install extra (Phase 4 of nss_first_class_sampler)

1 participant