Use local copy of VEP configuration

[nawatts]

Currently, VEP configuration is downloaded from Hail's US bucket.

gnomad_methods/gnomad/utils/vep.py

Lines 65 to 74 in 2090cbb

	VEP_REFERENCE_DATA = {
	"GRCh37": {
	"vep_config": "gs://hail-us-vep/vep85-loftee-gcloud.json",
	"all_possible": "gs://gnomad-public/papers/2019-flagship-lof/v1.0/context/Homo_sapiens_assembly19.fasta.snps_only.vep_20181129.ht",
	},
	"GRCh38": {
	"vep_config": "gs://hail-us-vep/vep95-GRCh38-loftee-gcloud.json",
	"all_possible": "gs://gnomad-public/resources/context/grch38_context_vep_annotated.ht",
	},
	}

This bucket is requestor pays, so reading from it requires that either --requester-pays-allow-all or --requester-pays-allow-buckets hail-us-vep is specified when starting the cluster. If someone isn't aware of that, it's easy to start a cluster (which takes a fair amount of time with VEP) that can't run the gnomAD VEP utilities.

hailctl dataproc's VEP init scripts download these configuration files and link them to /vep_data/vep-gcloud.json. If VEP configuration was loaded from the local copy downloaded by the init scripts instead of the hail-us-vep bucket, then the requestor pays arguments would not be necessary.

https://github.com/hail-is/hail/blob/498f73704368ea548dfcd4acc469c6fbcd61f83a/hail/python/hailtop/hailctl/dataproc/resources/vep-GRCh37.sh#L26-L28

https://github.com/hail-is/hail/blob/498f73704368ea548dfcd4acc469c6fbcd61f83a/hail/python/hailtop/hailctl/dataproc/resources/vep-GRCh38.sh#L26-L28

[lfrancioli]

Makes total sense to me.