feat: verify safetensors integrity after Civitai/HF LoRA download#2206
Merged
Conversation
…A download
Truncated or right-size-but-wrong-bytes LoRA downloads previously installed
silently and rendered garbage ("mosaic") output at generate time — the leading
non-Metal cause of corrupt output. PortOS already verified HF-cached weights
(hfCache.js, #1324) but the direct Civitai/HF LoRA install path was uncovered.
- Export verifySafetensorsStructure from hfCache.js (barrel via export *; README row added)
- installFromCivitai/installFromHuggingface now run a structural header/size
check after download and, for Civitai, a deep sha256 compare against
file.hashes.SHA256; on truncation/mismatch the file is deleted and a
ServerError (CIVITAI_LORA_CORRUPT / HF_LORA_CORRUPT) is thrown so the user
re-downloads instead of training/rendering against corrupt weights
- Tests: corrupt-download deletion, sha256 mismatch/match (case-insensitive),
plus updated existing fixtures to emit structurally-valid safetensors bytes
Closes #2199
Claude-Session: https://claude.ai/code/session_012wmkuPi75gYu6rrfhG8jxA
# Conflicts: # .changelog/NEXT.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A truncated or right-size-but-wrong-bytes LoRA download from Civitai or HuggingFace previously installed silently and then rendered garbage ("mosaic") output at generate time — the leading non-Metal cause of corrupt output. PortOS already verified HF-cached weights (
hfCache.js, #1324) but the direct Civitai/HF LoRA install path was not covered.What changed
verifySafetensorsStructurefromserver/lib/hfCache.js(reachable through theexport *barrel; added a README row per the module-organization rule) instead of re-implementing the safetensors header parse.installFromCivitai/installFromHuggingfacenow run a newverifyDownloadedLorahelper afterdownloadToFile(...)and before writing the sidecar:sha256File(...)againstfile.hashes.SHA256(frompickPrimaryFile), case-insensitively (Civitai reports uppercase;sha256Filereturns lowercase), and only when the digest is a well-formed sha256.ServerError(CIVITAI_LORA_CORRUPT/HF_LORA_CORRUPT) so the user re-downloads instead of training/rendering against corrupt weights.pickHfLoraFile, so the structural check is its integrity guard (no deep sha256 compare available there).Tests
CIVITAI_LORA_CORRUPTandHF_LORA_CORRUPT); SHA-256 mismatch → deletes + throws; SHA-256 match (case-insensitive) → installs.validSafetensors()helper) now that downloads are verified.Closes #2199
https://claude.ai/code/session_012wmkuPi75gYu6rrfhG8jxA