Main -> release: releasing version 3.0.1 with only vulnerabilities fixes - #399
Merged
Conversation
* fix(security): load dict assets as JSON instead of pickle (CWE-502) Several models loaded small mapping dictionaries via pickle.load, which allows arbitrary code execution during unpickling (CWE-502, semgrep python.lang.security.deserialization.pickle.avoid-pickle). Each of these assets is a plain str -> (str|int|float) mapping, so it can be serialized as JSON, which has no code-execution surface -- retiring the finding outright rather than accepting the risk. Assets migrated (hosted on S3 under a new .json key alongside the untouched .pkl, so the change is fully reversible): - geneformer: gene_median_dictionary, token_dictionary, ensembl_mapping_dict - uce: species_offsets Changes: - geneformer_tokenizer.py / uce_utils.py: pickle.load(rb) -> json.load(r), drop the now-unused `import pickle` - geneformer_config.py / uce_config.py / uce/model.py: point paths at the .json keys; rename uce offset_pkl_path -> offset_json_path - ci/download_all.py + geneformer tokenizer test: use the .json filenames Tahoe's GeneVocab.from_file always receives vocab.json (downloaded from the tahoebio HuggingFace repo), so its .pkl branch was dead code and is removed. Verified: each .json reloads equal to its original pickle, and the geneformer tokenizer / uce offset loaders read the JSON through the real code paths with byte-identical results (token IDs load as Python int rather than np.int16, which torch handles identically).
…395) HyenaDNA was the only model still using torch.load(weights_only=False), which allows arbitrary code execution during unpickling (CWE-502). The hosted .ckpt is a full PyTorch-Lightning checkpoint whose non-tensor objects (OmegaConf configs, optimizer/scheduler state) block weights_only=True; only ["state_dict"] is ever consumed. Point HyenaDNAConfig at a slimmed, tensor-only checkpoint hosted under a new S3 key hyena_dna/<model>.weights.ckpt (originals left untouched) and flip the loader to weights_only=True. Update ci/download_all.py and the config test for the new filename.
bputzeys
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.