Annotate a VCF

Annotate a VCF and download it back

Upload a VCF, wait for VariantGrid to import + annotate any novel variants, then download the cohort-level annotated export (all samples, single-sample VCFs included).

The quick way

The whole flow is a one-liner with annotate_vcf():

from variantgrid_api.api_client import VariantGridAPI

api = VariantGridAPI(server="https://variantgrid.com", api_token="YOUR_API_TOKEN")

# export_type is "vcf" (gzipped *.vcf.gz) or "csv" (zipped *.csv.zip)
path = api.annotate_vcf("input.vcf", export_type="vcf", dest_path="/data/results/")
print(f"Annotated VCF written to {path}")

annotate_vcf() blocks until annotation is finished, which can take hours (it uploads, then polls until complete, then downloads). Fine for a quick interactive job, but for large VCFs or batches you usually don't want to hold a process open that long - use the submit now, download later pattern instead. annotate_vcf() raises TimeoutError if annotation isn't done within timeout (default 3600s), so bump timeout if you do block on a slow job.

Drive each step yourself

annotate_vcf() just chains upload → wait → download. Do it by hand when you want control over each step:

# For ad-hoc uploads pass path=None so the upload isn't treated as a SeqAuto backend-link hint
upload = api.upload_file("input.vcf", path=None)
uploaded_file_id = upload["uploaded_file_id"]           # or upload["sha256_hash"]
api.wait_for_annotation(uploaded_file_id, timeout=3600, poll_interval=10)  # raises on error/timeout
path = api.download_annotated(uploaded_file_id, export_type="csv", dest_path="/data/results/")

Inspecting status directly

poll_upload_status(uploaded_file_id) returns the raw status dict (including annotation_complete, progress_percent, error, vcf_id, samples, ...) if you want to inspect progress directly. All of these accept sha256=<hash> instead of uploaded_file_id - the server dedups on the content hash, so it is stable across machines and useful if you didn't retain the id.

Submit now, download later

Annotation can take hours, so for anything non-trivial you'll want to fire the jobs off and come back later rather than block. The server keys each upload on the SHA-256 of the file content, so the VCF file itself is your receipt: you don't need to persist upload ids or any pending.json - just re-hash the same files later and ask for them. This works from any machine that has the VCFs.

Here it is over a whole directory of VCFs:

import hashlib
from pathlib import Path
from variantgrid_api.api_client import VariantGridAPI

api = VariantGridAPI(server="https://variantgrid.com", api_token="YOUR_API_TOKEN")
vcf_dir = Path("/data/vcfs")


def sha256(path):
    """Content hash the server dedups on - same value as `sha256sum <file>`."""
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()


# --- Submit: fire off every VCF in the directory, then walk away. Store nothing. ---
for vcf in sorted(vcf_dir.glob("*.vcf.gz")):
    api.upload_file(str(vcf), path=None)
    print(f"submitted {vcf.name}")

# --- Later (even on another machine): re-hash the same files, fetch whatever's ready ---
for vcf in sorted(vcf_dir.glob("*.vcf.gz")):
    status = api.poll_upload_status(sha256=sha256(vcf))
    if status.get("error"):
        print(f"{vcf.name}: ERROR - {status['error']}")
    elif status["annotation_complete"]:
        api.download_annotated(sha256=sha256(vcf), export_type="vcf", dest_path="/data/results/")
        print(f"{vcf.name}: downloaded")
    else:
        print(f"{vcf.name}: not ready yet ({status.get('progress_percent')}%)")

Re-run the second loop whenever you like - already-downloaded files just get fetched again, and anything still annotating reports its progress. Because the hash comes from the file bytes, sha256sum *.vcf.gz on the command line gives you the very same handles if you'd rather look them up by hand.

wait_for_annotation() / annotate_vcf() are the blocking convenience wrappers - they poll this same status in a loop until it's complete (or timeout elapses). Reach for the manual poll_upload_status check above when you don't want to hold a process open.

From the command line

The same stateless flow is available as a command line tool, vg_api annotate_vcf. The first call uploads the VCF; running the same command again downloads the annotated result once it's ready, or tells you it isn't ready yet - no local state, since it's keyed on the file's SHA-256:

$ export VARIANTGRID_API_TOKEN=YOUR_API_TOKEN         # or pass --token
$ vg_api annotate_vcf input.vcf.gz -o results/
Uploaded input.vcf.gz (id=13256). Annotating - run the same command again later to download.

$ vg_api annotate_vcf input.vcf.gz -o results/        # a while later
input.vcf.gz: not ready yet (progress 40.0%) - run the same command again later.

$ vg_api annotate_vcf input.vcf.gz -o results/        # once it's done
Annotated vcf written to results/input.vcf_annotated_v254_GRCh38.vcf.gz

Exit codes make it scriptable - 0 downloaded, 3 pending, 1 error - so you can poll in a loop:

while true; do
    vg_api annotate_vcf input.vcf.gz -o results/
    [ $? -eq 3 ] || break        # stop on success (0) or error (1); keep going while pending (3)
    sleep 300
done

Options: --export-type vcf|csv, -o/--dest, --server (or $VARIANTGRID_API_SERVER), --token (or $VARIANTGRID_API_TOKEN), and --wait to block until it's done (can take hours) instead of polling by hand.

Notes

• The annotate_vcf wrapper already uploads with path=None. Only pass a path to upload_file for SeqAuto uploads that link to a registered JointCalledVCF / SingleSampleVCF (that is what path is for - it is ignored on non-SeqAuto deployments).
• Include a full ##contig header in the VCF so the server can detect the genome build - a partial contig set can leave the build undetectable.
• Downloads require the target deployment to have the cohort export analysis templates configured (ANALYSIS_TEMPLATES_AUTO_COHORT_EXPORT) - otherwise a clear error is returned.