Add a varcode-powered worker for annotating VCF files

[armish]

This new worker annotates each variant in a VCF file with the top priority effect properties:

• Gene name (Official gene symbol)
• Transcript ID (Ensembl ID for the transcript)
• Protein-level change (e.g. V600E)
• Type of the variant (.e.g Substitution, Splice Site, Intronic, ...)

This worker is run when user submits a new VCF file (i.e. when a new run is created)

Once the worker completes its annotation, new information on variants are shown under Annotations Column Group with the Varcode prefix:

[ihodes]

Did something change to obsolete this?

[armish]

No, actually not -- this is just an artifact of the diff. These lines are still present a little bit above, but somehow this diff view thinks that they were deleted and added back.

[ihodes]

Awesome to have this coming in! Thanks for the work, @armish.

I do think we can drop the gene_annotator; there's no reason to have both now, AFAIK. With that, I'd drop the varcode_ prefix to the column names as well.

Could you also add the migration code required to get the database up to date with the new schema in the PR text? We do this to make deployment slightly easier. It's definitely not the right solution… but it's something :)

Ah, and this will also require updating the test database used for generating screenshots, and generating new screenshots for the examine page. Cf. the tests/pdifftests directory for more info.

[ihodes]

I can't tell if it's just some funky GitHub formatting, but some of these lines around here may be longer than 80 charts; mind chopping them down to size?

[armish]

Yeah, will definitely do.

[armish]

@ihodes: and here is the migration code:

ALTER TABLE genotypes
    DROP COLUMN "annotations:gene_names",
    ADD "annotations:gene_name" TEXT,
    ADD "annotations:transcript" TEXT,
    ADD "annotations:effect_notation" TEXT,
    ADD "annotations:effect_type" TEXT;

that is, if Travis gives us the green light :)

[ihodes]

If there are multiple genes, do they still get displayed? (If so, this should probably still be "annotations:gene_names")

[armish]

Nope -- this annotation only picks an effect in a single gene. So we won't be having multiple genes annotations any more.

[ihodes]

Hmm—is there a way to get that back? That seems like a useful feature /cc @iskandr @tavinathanson

[iskandr]

@armish You only have a single gene if you're picking the top priority annotation for each variant.

You could do this to get the top effect for each distinct gene:

for gene_name, gene_effects in variant.effects().groupby_gene_name().items():
    top_gene_effect = gene_effects.top_priority_effect()

but you'll have to then deal with all sorts of weird non-coding gene types.

[armish]

From a programmatic point of view, there is no reason not to include multiple gene names; but putting my biologist hat on, I think we should only show a single gene to the users. This might be a bold step forward, but I opt for single gene/variant route.

Happy to take this discussion off-line ;)

[ihodes]

I don't have a very strong opinion about this, but could you file this in an issue to resolve out of band with this PR?

[ihodes]

This is looking good! We still need these to show up in our screenshot tests as well, cf.

this will also require updating the test database used for generating screenshots, and generating new screenshots for the examine page. Cf. the tests/pdifftests directory for more info.

[armish]

phew! It took me a few iterations to get it right, but here we are with the updated screenshots and a successful build \o/

[ihodes]

This looks great! Thanks for putting it together—will be a great addition to Cycledash.