attacklm-dataset v0.4.3 — LiRA workflow closure

LiRA workflow closure

What's new

• scripts/score_shadow.py (NEW): Closes the LiRA workflow Step 2. The
  docstring of shadow_train.py referenced a non-existent script that users
  had to write themselves. Now it ships in-repo: load a HF model, score a
  JSONL of records, write shadow_{K}.json. Atomic write, --dry-run,
  duplicate-id handling, missing-assistant-turn handling.
• --mia-method offline: White-box MIA baseline that requires NO shadow
  models. Uses sample mean/std of audit-set NLL as the OUT distribution.
  For each record: z = (nll - μ_out) / σ_out. Lower z = more likely member.
  Default threshold: -1.5 (configurable). Field names: offline_z,
  offline_mu_out, offline_sigma_out, offline_flagged.
• compute_offline_z() added to inversion.scoring (reuses
  zscore_normalize() for the σ=0 guard).
• 15 new tests (10 in test_score_shadow.py, 5 in test_offline_mia.py).
  Total: 475 passing, 0 skipped.

Bug fix

• pyproject.toml drift fixed: was 0.4.0, everything else was 0.4.2.
  Caught by bumpversion --check. The every-commit-equals-a-new-tag rule
  paid for itself again.

Workflow recipes

Full LiRA (K=16 shadows, ~16h on RTX 4080 for 3B):

for k in $(seq 0 15); do
    attacklm train --output models/shadow_$k
    python scripts/score_shadow.py \
        --model models/shadow_$k \
        --records data/audit_set.jsonl \
        --output-dir losses/ \
        --shadow-index $k
done
python -m inversion.shadow_train \
    --loss-dir losses/ --output shadow_params.json
python scripts/inversion_audit.py \
    --model target \
    --mia-method lira \
    --lira-params shadow_params.json \
    --mia-threshold-mode lrt

Quick baseline (1 minute, no shadows):

python scripts/inversion_audit.py \
    --model target \
    --mia-method offline

Prior art

Carlini 2022 §3.2 (reference attack with z-normalization).
This is NOT Sablayrolles (early planning mis-claimed the provenance);
it's the loss-z baseline with sample std.

Files

• data tarball (3.8 MB, 18 sources, 24,652 records, 13.1x compression)
• See docs/LIRA.md for the full workflow