Repository navigation
You fine-tune on customer data with SFTTrainer. Can you prove it didn't memorize? memaudit runs that audit inside your training job. #7082
ansh200516
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
If you SFT on private or customer data, at some point legal, security, or a customer asks: "Show me the model didn't memorize any of it." Most teams have nothing — no test, no report, a shrug.
memaudit turns your existing TRL job into that evidence. Two lines:
inject()plants calibrated decoy secrets in your raw dataset,MemorizationAuditCallbackdoes the rest. When training ends you get a local JSON report you can forward — versioned, threat-model table mapped to EDPB Opinion 28/2024, checksums, no phone-home. Apache-2.0.It's built for how SFTTrainer actually works. Your trainer tokenizes, masks, truncates, and packs before callbacks fire — which is exactly how naive memorization tests silently break: the probe lands in the prompt with
completion_only_loss=True, or truncation eats it, or LoRA never touches the layer it lives in. Result: a test that says "clean" about a model that never saw the test. memaudit verifies at train start that every decoy actually receives loss under your real config — masking, packing,max_length, chat templates, LoRA/QLoRA, tied embeddings — and fails the run instead of printing a fake all-clear.What the report answers — two questions regulators treat separately, never blended into one score: can an attacker with model access tell which records were trained on (membership, at a 1% false-alarm point with confidence intervals), and does the model complete a training record from its prefix (regurgitation, under a stated protocol). It refuses to print headlines when your canary/control counts are too thin, and anything that didn't run says "not run" — never 0.
Public proof run (reproduces on a laptop): TinyLlama-1.1B-Chat + 20k Alpaca, LoRA r=8, 1 epoch, 100 decoys / 200 controls. Membership: 10/100 at 1% false alarm, CI [0.049, 0.176], AUC 0.837. The leakage curve is the useful part: decoys seen once — 0/34; seen 16× — 9/33. Duplication is what leaks, and the report shows that curve for your dataset. Regurgitation: 0/100 under the exact-match protocol. Full tables: case study.
pip install "memaudit[peft,trl]"
Put it on one training job this week — if the pre-flight flags something about your masking/packing config, that alone is worth the run. I'd also genuinely value TRL-expert eyes on the pre-flight edge cases (packing strategies,
assistant_only_loss, Liger). Guided run for early design partners: ansh.singh.160305@gmail.com(I'm the author.)
All reactions