Skip to content

csubst download

Jump to bottom
Kenji Fukushima edited this page Sep 10, 2026 · 5 revisions

csubst download

csubst download prepares model resources used by optional CSUBST workflows. Downloading once lets later jobs reuse local files. VESM files are always SHA-256 verified; ProstT5 availability is checked by loading the local model and tokenizer. These are different validation contracts.

Available resources

  • vesm-35m: model files for VESM-35M variant-effect scoring in csubst sites
  • prostt5: ProstT5 model files used by 3Di workflows
  • prostt5-cnn: ProstT5 encoder and the published 3Di CNN head
  • esm3di-35m (default): complete ESM3Di-35M checkpoint and base ESM2 configuration/tokenizer, matching the default AA-to-3Di predictor
  • all: all four resources

Examples

# Download the default ESM3Di-35M resource
csubst download

# Download a specific resource
csubst download --resource prostt5

# Prepare all resources
csubst download --resource all

Use --no_download yes to check that a resource is already available without network access:

csubst download --resource esm3di-35m --no_download yes

The model revisions are pinned by CSUBST. Existing valid files are reused rather than downloaded again.

Integrity and compatibility

  • VESM: every preparation checks file sizes and SHA-256 hashes, including --no_download yes. Missing or corrupt local files fail that offline check.
  • ESM3Di and the ProstT5 CNN head: the same mandatory size/SHA-256 checks apply. The ESM3Di checkpoint contains the entire trained backbone; separate base ESM2 weights are not downloaded.
  • ProstT5: the tokenizer/model must load with local-only Transformers calls for an offline check to succeed. CSUBST does not provide a SHA-256 verifier for this resource; successful loading is not a checksum guarantee.
  • --verify is a deprecated compatibility option shown only in advanced help. VESM accepts both values with a warning; --verify no never disables its mandatory verification. --verify yes with prostt5, prostt5-cnn or all fails before preparing any resource because that checksum check is unsupported. For new commands, omit --verify and use --no_download yes for local checks.

Cache locations

Resource Model/data location
VESM checkpoint and base model <CSUBST cache>/models/vesm-35m/v1/
ESM3Di-35M checkpoint/config/tokenizer <CSUBST cache>/models/3di/esm3di-35m/v1/
ProstT5 CNN head <CSUBST cache>/models/3di/prostt5-cnn/v1/
Downloaded protein structures <CSUBST cache>/structures/
ProstT5 weights and tokenizer Hugging Face cache, or --prostt5_local_dir when specified

The CSUBST cache is selected by --resource_cache_dir, then CSUBST_CACHE_DIR, then ~/.cache/csubst. This also controls CSUBST's shared resource locks, but does not relocate ProstT5 weights. Hugging Face normally uses ~/.cache/huggingface/hub; its HF_HOME / HF_HUB_CACHE configuration and Transformers settings apply. See the Hugging Face cache guide.

CSUBST publishes validated VESM downloads and downloaded structures atomically. Resource-specific interprocess locks coordinate downloads. The same lock implementation protects shared ProstT5 sequence/3Di cache writes; these derived caches have their own --prostt5_cache_file and --sa_state_cache_file options and are separate from model weights.

Offline preparation

For VESM, run the download once on a connected host, then copy the complete managed model directory, including its manifest, under the same relative path in the offline host's CSUBST cache. Check it with:

csubst download --resource vesm-35m --no_download yes

For ProstT5, a dedicated local directory makes the files to copy explicit:

# Connected host: the directory must exist before it is selected.
mkdir -p models/prostt5
csubst download --resource prostt5 --prostt5_local_dir models/prostt5

# After copying models/prostt5 to the offline host:
csubst download --resource prostt5 --prostt5_local_dir models/prostt5 --no_download yes

Use the same --prostt5_local_dir in the consuming 3Di command and add --prostt5_no_download yes there. Copying only the CSUBST cache does not copy ProstT5 weights from Hugging Face. All offline checks still require the model's Python dependencies to be installed.

For ESM3Di, copy its complete CSUBST model directory including the manifest. For ProstT5-CNN, copy both the CNN directory in the CSUBST cache and the ProstT5 encoder/tokenizer files in the Hugging Face cache or explicit local directory. Check the selected resource using --no_download yes.

Optional dependencies

The downloader and the consuming workflow need their corresponding optional Python dependencies. For VESM-35M:

python -m pip install "csubst[vep] @ git+https://github.com/kfuku52/csubst"
csubst download --resource vesm-35m

For the 3Di predictors, install csubst[3di] and see Structural alphabet predictors.

Run csubst download -h for common options or csubst download --help-advanced to include cache-directory and lock controls.

Related pages

Clone this wiki locally