feat(MODEL-MM-indextts2): convert the checkpoints OFFLINE, and refuse to read pickle in the engine (#634) - #742
Merged
Conversation
… to read pickle in the engine (#634) Upstream ships `.pth`: a ZIP around a Python pickle. Loading it needs a reader this tree does not have, and the answer is not to write one. Pickle executes arbitrary code by construction, so a reader in the engine would run an attacker-controllable program inside the process that serves users, and every other lane here already loads safetensors or GGUF. The conversion therefore happens offline, once, and the engine loads the result through the reader it already has. `scripts/convert-indextts2-checkpoint.py` flattens the nested state dicts with '.', which is exactly the naming `indextts2_pth_manifest.json` records, so the converted names ARE the manifest's names and the manifest can check the conversion. Run against the real checkpoint on the NAS, all three files matched their recorded tensor counts: 456, 972, 284. It also measured something the manifest could only hint at. `codec.pth` is **75% optimizer state** -- 729 of its 972 tensors are training residue -- and dropping it takes that file from 579.16 MiB to 192.99 MiB. The drop is reported with a count, never silent, and the prefix is gated from BOTH sides: every optimizer key in the manifest must match it, and no weight in `gpt.pth` or `s2mel.pth` may. A prefix widened from "optimizer." to "opt" would quietly eat model weights, which is exactly the mutation that proves the second check. The conversion needs torch and 4 GiB of weights, so CI cannot run it. `tests/scripts/test_indextts2_convert.py` holds the part where a silent mistake is unrecoverable -- which tensors survive, under which names -- using fakes, with neither torch nor the checkpoint, because a dropped weight looks exactly like a weight that was never there. It also asserts the flattener survives the ints, strings and Nones that sit beside weights in a `.pth`. 8 cases. Five mutations, five caught, script verified byte-identical after: the drop prefix widened to "opt", the drop prefix emptied, the flattener joining with '/' instead of '.', the flattener crashing on non-tensor leaves, and one source silently omitted. This unblocks the checkpoint loader; it is not the loader, and there is still no render. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(MODEL-MM-indextts2): convert the checkpoints OFFLINE, and refuse to read pickle in the engine (#634)
Upstream ships
.pth: a ZIP around a Python pickle. Loading it needs a readerthis tree does not have, and the answer is not to write one. Pickle executes
arbitrary code by construction, so a reader in the engine would run an
attacker-controllable program inside the process that serves users, and every
other lane here already loads safetensors or GGUF. The conversion therefore
happens offline, once, and the engine loads the result through the reader it
already has.
scripts/convert-indextts2-checkpoint.pyflattens the nested state dicts with'.', which is exactly the naming
indextts2_pth_manifest.jsonrecords, so theconverted names ARE the manifest's names and the manifest can check the
conversion. Run against the real checkpoint on the NAS, all three files matched
their recorded tensor counts: 456, 972, 284.
It also measured something the manifest could only hint at.
codec.pthis75% optimizer state -- 729 of its 972 tensors are training residue -- and
dropping it takes that file from 579.16 MiB to 192.99 MiB. The drop is reported
with a count, never silent, and the prefix is gated from BOTH sides: every
optimizer key in the manifest must match it, and no weight in
gpt.pthors2mel.pthmay. A prefix widened from "optimizer." to "opt" would quietly eatmodel weights, which is exactly the mutation that proves the second check.
The conversion needs torch and 4 GiB of weights, so CI cannot run it.
tests/scripts/test_indextts2_convert.pyholds the part where a silent mistakeis unrecoverable -- which tensors survive, under which names -- using fakes,
with neither torch nor the checkpoint, because a dropped weight looks exactly
like a weight that was never there. It also asserts the flattener survives the
ints, strings and Nones that sit beside weights in a
.pth.8 cases. Five mutations, five caught, script verified byte-identical after:
the drop prefix widened to "opt", the drop prefix emptied, the flattener joining
with '/' instead of '.', the flattener crashing on non-tensor leaves, and one
source silently omitted.
This unblocks the checkpoint loader; it is not the loader, and there is still no
render.
Issue: #634.
🤖 Generated with Claude Code