v0.6.1

Fixed

• M4-T02 / M11-T01: complete the Hugging Face, Transformers, vLLM,
  OpenAI-compatible and PEFT integration interfaces and consume them in
  detection, lock resolution and metadata capture. Reuse the bounded AST scan,
  resolve adapters once, and avoid HTTP client construction during offline lock.
  Detect literal OpenRouter base_url arguments without retaining URLs or keys.
• M9-T01: align the local composite Action with the published Action's
  numeric audit exit code; validate all four CLI exits with successful and
  failed annotation steps.
• M9-T02: emit comma-separated GitHub workflow-command properties and keep
  the file and line from the same evidence item, preserving escaping.
• M7-T04: keep the invalid-candidate retry available after an endpoint rejects
  JSON mode; retain the final failed response and cap all requests at three.
• M11-T01: connect evaluation framework detection and task extraction to init;
  pre-fill sorted task candidates, including lm-eval YAMLs with custom tags, without
  executing tags or importing the framework. Authors still confirm selected tasks and
  metric implementations; secret-bearing and machine-specific task names are excluded.
• M8-T02: refresh CLI documentation for GitHub output and checklist templates, repair
  macOS temporary-directory aliases and Windows path separators in quickstart generation,
  fix captured terminal styling and width under GitHub Actions and Windows,
  render CLI documentation with a portable program name and console, show freshness diffs,
  and enforce all generated-document checks in ordinary CI and nightly again.
• M7-T04: include explicitly declared built-in manifest fields in discover's duplicate
  exclusion context, recognize source-snippet evidence, and discard evidence outside the
  sent input files while retaining the candidate's stable ID and lowering confidence.
  Clean returned candidates, failed raw responses and errors before persistence,
  remove unsafe fields/bindings and exact credential echoes, and preserve real
  filenames containing #L while normalizing generated snippet locators.
  Preserve valid relative config bindings containing ./ without allowing
  normalization to hide original credential, identity or traversal content.

Validation

• Local quality gate: 1,437 tests passed; redaction branch coverage remains
  100%; schemas and generated documentation are current. Export snapshots
  change only their two package-version markers.
• The maintainer deferred remote/GPU and unavailable exact judge-input
  validation for this patch; these resource checks are not fresh passes.
  Five-project full Gate A and available command/metadata matrices have zero
  new gold deltas; one real DeepSeek discover request passed independent
  candidate review and acceptance/audit checks. See val.md §8.7 and the
  patch session report.

v0.6.0

Adoption-sprint release: everything in the M9–M11 track plus the Beta
baseline from M1–M8.

M9: GitHub Actions integration

• audit --format github — workflow-command output for any CI
• reprollm-action@v1 — published composite Action, bootstrapped in CI

M10: Paper checklist alignment

• export --template neurips|acl|acm — venue-specific checklist mappings
• docs/checklists.md — 45-row mapping across three conference checklists
• All five P1 rules confirmed implemented

M11: Evaluation framework integrations

• lm_eval, lighteval, inspect_ai — detect usage, extract task names
  for init pre-filling, report versions
• docs/integrations/ — manifest examples and run snippets per framework
• docs/integrations/card-snippets.md — HF model/dataset card snippets

Since 0.5.0 (Beta)

• Windows golden comparison closed at the root (byte-exact canonical form)
• Performance budgets (20k-file audit <10s) in nightly CI
• py.typed, CITATION.cff, twine check clean
• examples/ with real lockfiles, CI-validated
• Documentation suite: quickstart (CI-regenerated), concepts, FAQ, why,
  CLI reference, diff reading guide
• lock --offline constructs no HTTP client (SOCKS-proxy environments)

v0.5.3

Added (M11)

• Evaluation framework integrations (integrations/): lm_eval,
  lighteval, inspect_ai — each detects usage (imports, CLI invocations),
  extracts task names from framework config files (lm-eval task YAMLs,
  inspect-ai @task decorators) for init pre-filling, and reports
  installed versions. They never import the target library.
• docs/integrations/ — manifest examples and recorded-run snippets for
  lm-evaluation-harness, lighteval, and inspect-ai.
• docs/integrations/card-snippets.md — copy-paste reproducibility snippets
  for Hugging Face model cards, dataset cards, and paper artifact READMEs.

v0.5.2

Added (M10)

• export --template neurips|acl|acm: venue-specific checklist mapping
  appended to REPRODUCIBILITY.md. ReproLLM provides evidence (field values,
  hashes, run IDs) for conference reproducibility items — never Yes/No
  answers. Items outside ReproLLM's scope are listed honestly as not
  covered.
• docs/checklists.md: full mapping table across three conference
  checklists (45 rows, versions reviewed 2026-09-29).
• Nine demonstration exports (3 examples × 3 venues) committed under
  examples/.
• All five P1 rules confirmed implemented (M3/M4): code.remote_recorded,
  model.trust_remote_code_declared, dataset.subset_declared,
  gen.stop_declared, prompt.few_shot_declared.

Since 0.5.1

• reprollm-action composite Action published at
  EnumaElish123/reprollm-action@v1 and bootstrapped in ReproLLM's own CI.

v0.5.1

Added (M9)

• audit --format github: workflow-command output for any CI that speaks
  GitHub Actions annotations — ::error/::warning/::notice with
  file/line backlinks from finding evidence, spec-compliant escaping, and a
  summary notice. No Action installation required.
• action/ — the complete composite Action (action.yml + Marketplace
  README), ready for the maintainer to push to the reprollm/action repo.
  Inputs: path, fail-on, level, version, check-lock; outputs:
  critical-count, warning-count; steps include a Step Summary table and an
  artifact upload.
• CI now self-audits ReproLLM with --format github (bootstrapped
  dogfooding in the Checks tab).

v0.5.0

The complete Beta workflow: audit (Level 0/1/2), init, lock, run with
redaction, semantic diff, export, project rules, and experimental discover.

Added (M8 stabilization)

• examples/ — three runnable example repositories with real mirror-resolved
  lockfiles, refreshed by scripts/refresh_examples.py and audited in CI.
• Documentation suite: CI-regenerated quickstart.md (real output, cannot
  rot) and cli.md; new concepts.md, why.md, and a 12-entry faq.md;
  README rewritten for the Beta; docs/diff.md gained a line-by-line
  output-reading guide.
• Boundary guarantees as regression tests: empty manifest, corrupt lock,
  future schema_version (clear upgrade message), manifest-only repo,
  non-UTF-8 filenames — all actionable errors, never tracebacks.
• Performance budgets (20k-file audit < 10 s, init < 15 s) run nightly;
  per-PR runs stay fast.
• Packaging: py.typed ships; twine check passes; CITATION.cff;
  scripts/check_no_leaks.py for post-hoc artifact scans.
• Community: roadmap and good-first-issue catalog under docs/community/;
  CONTRIBUTING/SECURITY finalized.

Fixed

• The Windows golden mismatch is closed at the root: test-written fixtures
  now force LF endings everywhere (the last straggler of the platform-newline
  family), and golden comparisons use a byte-exact canonical form whose
  failure messages carry hex dumps.
• lock --offline constructs no HTTP client at all.

v0.5.0a1

Added (M7)

• reprollm export: deterministic REPRODUCIBILITY.md from manifest + lock
  • run — identity tables with revisions/pinnability/digests, code,
    environment, hardware, execution (explicit no-run degradation), an audit
    run embedded verbatim, known limitations, and version footer; whole
    document passes secret redaction.
• Project rules end-to-end: reprollm rules add/list write
  .reprollm/project-rules.yaml; accepted declarations execute as audit
  rules (project.*, severity_origin project_rule); their bindings join
  run-time observations (consistency.custom_fields has real data; the
  project_rules_sha256 lock-freshness branch triggers for real).
• Experimental reprollm discover (opt-in; time-boxed, delivered):
  offline collector with redaction gating (files whose redaction triggers
  are dropped whole), one temperature-0 JSON request to a user-configured
  OpenAI-compatible endpoint with a confirmed file list, deterministic
  candidate ids, evidence-path demotion, and rules accept/ignore/pending
  listing. --paper is rejected in Beta.
• Discover candidate schema (§20.4) completes the placeholder; schemas
  re-exported. docs/export.md, docs/project-rules.md,
  docs/discover.md, and docs/plan/backlog.md.

Fixed (M7 dogfooding)

• lock --offline no longer constructs an HTTP client at all — a SOCKS
  proxy environment used to crash offline locking at client construction.

Validation

• Five-repository full flow (init → audit L1 → lock --offline → audit L2 →
  export) completed in disposable worktrees; discover dry-runs verified.
  Real GPU re-pairs (lm-eval clean re-run, HarmBench) remain gated on GPU
  availability; the DeepSeek judge and DistilGPT2 DP pairs are the formal
  standards per val.md §8.5.

Changed (validation baseline)

• val.md §8.5 (reviewed 2026-09-28): the DeepSeek judge pair and the
  DistilGPT2 privacy fine-tune are promoted from supplements to the formal
  Gate B M5 standards for the FastChat-judge and llm-dp scenarios; the
  credential-gated GPT-4 and Llama-2 originals are retired with their
  residual differences recorded. No pinned commit or other gold changed.

v0.4.0

Changed (M6, second half)

• Record the second-half validation evidence and prepare gated 0.4.0 release
  notes (M6-T08 preparation). Keep the package version at 0.1.1 until real paired
  model acceptance and the outstanding resource gates are complete.

• Document diff inputs, table v1, version components, verdicts, overrides and
  independent display/exit thresholds; add tested fixture excerpts to the README
  (M6-T07, spec §§6, 17–18). Real paired-model acceptance and release remain pending.

• Add four JSON/text diff goldens: captured prompt/temperature changes, a locked
  model revision, identical inputs and a torch patch update (M6-T06, spec §18).
  The capture scenario commits its changed inputs before measuring commit drift;
  identical inputs have no changes, and torch patch drift is MEDIUM under table v1.

• Migrate Level 2 generation, model identity, custom-field, environment and file
  consistency checks to State leaves and their alternatives (M6-T05, spec §§11,
  12.12, 17). Audit projects current declarations and working files alongside
  runtime observations; standalone diff continues to use historical snapshots.
  Existing audit snapshots and lock/run schemas remain unchanged.
  Invalid lock paths, including empty paths, remain explicit consistency findings
  instead of aborting State projection.

Added (M6, first half)

• Add reprollm diff for run IDs/prefixes, run files/directories and lock files,
  with text/JSON reports, schema validation, display filtering and independent
  exit thresholds (M6-T04, spec §§1, 6, 17–18, 23). Snapshots are contained and
  mandatory when referenced; report values are sanitized after comparison.
  The v1 report adds nullable filtered_below; summary always remains complete.

• Compare effective State values with normalized equality, stable severity/section
  ordering, complete counts and source-conflict notes (M6-T03, spec §18).
  The diff_report v1 placeholder now requires input references, summary and
  changes. No diff writer was previously released; consumers of the placeholder
  schema must regenerate it. Lock/run schemas are unchanged.

• Ship the normative drift table and resolve ordered profile overrides, dirty-code
  notes and package version components (M6-T02, spec §§6, 18.1–18.2). Patch drift
  lowers severity by one level: torch MEDIUM_HIGH → MEDIUM, following the
  specification rather than the conflicting LOW example in the sprint plan.

• Project manifest, lock and run snapshots into a deterministic experiment State;
  retain every source with provenance and runtime precedence (M6-T01, spec §17).
  Three new State goldens include declared/resolved/observed values; two synthetic
  run fixtures complete projection coverage. Existing lock/run schemas and audit
  snapshots are unchanged. YAML dates in free-form fields project to stable ISO
  strings before deterministic source ordering.

Added (M5, second half)

• Document runtime capture, structured conflicts, environment policies, raw hashes
  versus redacted snapshots, child-owned artifacts, SLURM placement and private
  reporting; add run/inspection commands to the quick start (M5-T08, spec §§1, 5, 15–16).

• Add a real-child, dirty-repository leakage test covering both environment modes,
  argv, forbidden files, snapshots, patches and captured logs. Run the security
  gate separately in CI and before building release distributions (M5-T07,
  spec §§5.1, 16, 22 T-06); any failure blocks publication.

• Audit the latest valid run for file, generation, model, environment and accepted
  custom-field conflicts; preserve all structured evidence and project severities.
  Skip corrupt records with visible warnings and remove the final four runtime
  placeholders (M5-T06, spec §§5, 11, 12.12, 15). Reviewed Level 2 snapshots now
  explain missing evidence; a new post-run snapshot demonstrates a CRITICAL
  temperature conflict and exec.run_recorded PASS. Schemas are unchanged.

Validation (M5-T09 preparation)

• Record unchanged five-project gold results, byte-identical initialization and
  full Level 1 comparisons, and independently specified runtime conflicts/latest
  matching runs. Real inference/training/judge gates remain blocked on provisioned
  resources; see the validation report.

Release note (M5, shipped in 0.4.0)

Planned for 0.3.0: the first release of the run record schema v1 writer
and the documented redaction policy. Draft notes
include sanitized inspection/conflict examples and an isolated packaging preflight
(M5-T10 preparation). Task commits are prepared; resource validation, maintainer review and CI are
still pending; no version bump, tag or publication is claimed.

Added (M5, first half)

• Expose run, runs list, and unique-prefix runs show, with config defaults,
  child argument/exit preservation, JSON inspection, and a reviewed locked-fixture
  run snapshot showing CLI/config temperature disagreement (M5-T05, spec §§1, 5, 15).
• Execute commands without a shell, write crash evidence before launch and an
  atomic final record, forward signals, tee opt-in redacted logs, and preserve
  child exit codes through metadata-capture failures. Persist portable paths
  and hashed machine identity (M5-T04, spec §5.1 R-01/R-02/R-05/R-06/R-10–R-12).
• Hash repository-contained input and output files, snapshot bounded redacted
  text, and observe declared CLI/config/environment bindings with deterministic
  normalization and complete repeated-flag history (M5-T03, spec §§5, 15–16).
• Capture CPU/GPU metadata through bounded nvidia-smi calls, hash GPU UUIDs,
  and capture SLURM/PBS/LSF namespaces with secret names omitted and values
  redacted; malformed or unavailable GPU data is explicit (M5-T02, spec §5.1 R-07/R-08).
• Implement the nine ordered text-redaction classes, segment-based environment
  name checks, allowlisted environment capture, and streaming PEM suppression;
  execute the synthetic positive/negative/name corpus and enforce 100% branch
  coverage for the redaction module in CI (M5-T01, spec §16).
• Accept complete JWT padding variants under the maintainer-approved
  spec correction #3, with
  all 27 segment-padding combinations covered. No schema or rule ID changes.

Release note (M4, shipped in 0.4.0)

Planned for 0.2.0: the first release of the lock schema v1 writer
(reprollm.lock, schema_version: 1). Existing manifest schema v1 remains
compatible; no schema migration is required.

Clarified

• Keep only the four runtime consistency placeholders through M4; require an
  empty stub set after M5-T06, with a CI assertion for the current boundary.
  This maintainer-approved clarification changes no schema or rule selection
  (#2, spec §§6.1, 12.12).

Added (M4, second half)

• Record five-project metadata validation with independently checked HF
  revisions and file hashes, offline/online stability, and authenticated
  gated-file denial without credential leakage; retain Linux H2 as a pending
  manual gate (M4-T08, spec §§4, 12, 22; validation record).

• Reuse the bounded HF client in doctor --check-network, including mirror,
  retry, metadata-validation, and sanitized authentication failure behavior;
  document lock provenance, pinnability, offline resolution and freshness
  (M4-T07, spec §§1, 4, 22 T-04).

• Add reviewed lock and Level 2 audit snapshots for all three complete fixtures,
  with exported-schema validation, an intentionally unavailable vLLM backend,
  API judge pinnability, and a gated-model denial (M4-T06, spec §§4, 10, 22).

Fixed (M4 dogfooding)

• Recognize supported Apple Git version strings in doctor instead of reporting
  a parsing warning for their vendor suffix (M4-T08, spec §1).

Documentation (M4-T09 preparation)

• Prepare draft release notes with the real
  FastChat judge lock fragment and isolated wheel/sdist packaging evidence.
  Version bump, tag and publication await the manual Linux validation gate.

Added (M4, first half)

• Add a bounded Hugging Face Hub HTTP client with mirror support, environment-based
  authentication, sanitized errors, and deterministic retry behavior, backed by a
  hand-written offline response corpus (M4-T01, spec §4.3 and §22 T-04).
• Resolve Hugging Face, API, and local model and dataset identities; hash prompts,
  declared files, metrics, local metadata, adapters, and bounded weights; and capture
  backend and environment provenance in online and offline modes (M4-T02, spec §4).
• Add reprollm lock with atomic deterministic YAML writes, offline and optional API
  verification modes, large-weight control, provenance summaries, and a network-free
  --check freshness path with exit code 1 for missing or stale locks (M4-T03, spec §§1, 4).
• Load and validate locks during audit, auto-detect Level 2, report manifest and accepted
  project-rule staleness independently, and compare locked prompt/config hashes with the
  current working tree using repository-contained paths (M4-T04, spec §§9–12.12).
• Enforce resolved model, tokenizer, chat-template, dataset, backend, prompt, and judge
  lock metadata with provider-specific severities and provenance evidence; API snapshot
  aliases remain informational while mutable API aliases warn (M4-T05, spec §§12.4–12.9).

v0.1.1

M3 completes all 38 Level 1 LLM experiment rules and finalizes the seven built-in
profiles: core, inference, evaluation, llm_judge, finetuning, privacy,
and safety.

An intentionally incomplete OpenAI judge manifest now reports the missing judge
sampling contract directly:

CRITICAL (1)
  X judge.params_declared      Missing fields: evaluation.judge.params.max_tokens, evaluation.judge.params.temperature
      evaluation.judge.params.max_tokens (absent)
      evaluation.judge.params.temperature (absent)
      fix: Set evaluation.judge.params.max_tokens, evaluation.judge.params.temperature in reprollm.yaml.

28 passed · 0 suppressed · 11 skipped
Result: FAIL (1 critical, 2 warning)

Added (M3, second half)

• Generated rule and profile reference pages with CI freshness checks, a
  documented Level 1 coverage overview, and an enforced 85% CI coverage floor
  (M3-T09, spec §§6, 12, 22–23).
• Complete and intentionally incomplete Level 1 manifests for the HF/vLLM,
  OpenAI judge, and privacy fixtures, with six reviewed JSON snapshots and
  variant-scoped dependency/secret overlays (M3-T08, spec §§3, 6.1, 12, 22).
• Apply audit.ignore by canonical rule ID or alias, preserving the original
  severity and mandatory reason in INFO findings. Honor config defaults for
  fail_on and show_passed, with explicit CLI flags taking precedence;
  suppressed results remain visible and do not affect exit thresholds
  (M3-T07, spec §§1.1, 8, 11, 21).
• Complete built-in profile rule selections and severity overrides, with
  most-derived override origins in audit findings and profile inspection.
  Add 15 explicitly skipped Level 2 placeholders and preserve project-owned
  severity metadata without implementing lock or runtime checks (M3-T06,
  spec §§6.1, 11–12, 18.2).

Fixed (M3, second half)

• Treat either explicit boolean value for models.<role>.trust_remote_code as
  satisfying the repository-wide detection rule, matching the selected
  execution path without encouraging an inaccurate true declaration
  (#1, M3-T10, spec §12.4).
• Compare detected profiles with the resolved inheritance closure, so a profile
  inherited by the selected experiment is not reported as undeclared. Clarify
  that unrelated repository capabilities do not need to be added to the
  manifest (M3-T10, spec §§6, 12.3).
• Allow reprollm profiles show core while continuing to reject explicitly
  selecting core in a manifest or audit --profiles (M3-T06).

Documentation (M3-T10)

• Mark statically detected model IDs as candidates whose experiment role must
  be verified, add a practical manifest guide, and record Project A/B Level 1
  dogfooding findings and their disposition.

Added (M3, first half)

• Five training and four privacy rules for training hyperparameters, complete
  LoRA adapters, threat models, nonempty mechanism parameters, privacy metrics
  and attack budgets (M3-T05, spec §§12.10–12.11).
• Six evaluation and four judge rules for metrics and implementation references,
  aggregation, repetitions, safety definitions/budgets, and independently declared
  judge roles and sampling settings (M3-T04, spec §§12.8–12.9).
• Five generation/backend rules and three prompt rules, including conditional
  sampling seeds, vLLM fields, explicit stop/few-shot declarations and prompt
  file existence without reading prompt contents (M3-T03, spec §§12.6–12.7).
• Five Level 1 dataset rules for role declarations, Hugging Face split/subset,
  preprocessing and sampling seeds, with zero-valued seeds preserved (M3-T02,
  spec §12.5).
• Six Level 1 model declaration rules, including per-role API exclusions,
  inference dtype/quantization fallback evidence, adapter detection, and explicit
  remote-code trust declarations (M3-T01, spec §12.4). The trust rule is selected
  by core per the maintainer-approved correction to spec §6.1.

Fixed (M2-fix corrective sprint, 2026-09-10)

• Audit JSON no longer persists the raw invocation path (absolute paths
  leaked /home/...); the target is normalized to . or a repository-
  relative POSIX path, and generated_at is second-precision UTC (F-01,
  F-02).
• CLI error contract enforced: output-write failures are exit 2 with an
  actionable message; unexpected exceptions exit 3 without a traceback
  (traceback only with -v) (F-03).
• A pyproject.toml without [project]/[tool.poetry] no longer counts
  as a dependency manifest (F-04).
• Dependency evidence points at physical declaration lines instead of
  array/mapping indexes; no more line: 0 (F-05).
• Separator-equivalent keyword spellings (red team/red-team,
  dp-sgd/dp_sgd, …) count as one detection concept (F-06).
• Exact eval/evaluation directory segments are an evaluation signal
  (F-07).
• Scan diagnostics (truncation, syntax errors, unparsed declarations)
  surface on stderr under -v; stdout stays one JSON document (F-08).

Added

• Normative secret fixture corpus (§16.5): positive/negative/env/file
  cases for the M5 redaction classifiers (F-09).

Changed

• init/doctor domain logic extracted into core.manifest_scaffold and
  core.diagnostics; CLI commands only parse/prompt/render/exit (F-10).
• Detection signals now come from the profile YAML definitions (user
  overrides honored); only schema-inexpressible semantics stay in Python
  (F-11).

Added

• Authoritative five-project gold-answer validation matrix (val.md), milestone-gated real
  training/inference/API checks, and the M2 corrective sprint plan (M2F-T12).