Repository navigation
unspecified_caller vs. very_sensitive_caller vs. vcf_candidate_importer #433
Description
Hello, I was hoping you could explain the differences between each of the options you can use for the --variant_caller option. How do each of these differ in terms of the type of model produced and interpretation of results? I'm having trouble finding documentation for this. Thank you!
Activity
pichuan commentedon Mar 25, 2021
Hi @maryawood , thanks for the question.
As you probably noticed in the code,
very_sensitive_calleris the default that we use for candidate generation. In all our released models, this is the default so there is no need to change it.vcf_candidate_importeris a more advanced / experimental implementation , where we allow the users to provide a list of candidates that they want DeepVariant to call. You can see an example use case here: https://gist.github.com/pichuan/baba6ee9bd9890be2a45076a4934dd38Another important use case for
vcf_candidate_importeris PEPPER-DeepVariant which works on Oxford Nanopore long-read data.We haven't written an official documentation for this, because there are many subtle details when providing the VCF file for
--proposed_variants, otherwise it might confuse DeepVariant. You're welcome to give it a try, and feel free to ask questions here.Our team might be a bit slower to answer specific questions on this, but we can try our best to support.
Thanks!
maryawood commentedon Mar 25, 2021
Hi @pichuan, thanks for the explanation! I've been working on using DeepVariant to train my own sequencing-type specific model, and I'm getting to the point of wanting to mess with some of the different options available to improve performance. It sounds like
vcf_candidate_importerwould be less relevant for this purpose, but what aboutunspecified_caller? How does that option differ fromvery_sensitive_caller?pichuan commentedon Mar 25, 2021
unspecified_calleris an invalid option. I've never actually specified it, but I think the behavior should be that it should crash earlier (hopefully with a meaningful error message).To train your own sequencing-type specific model, generally if the sequencer's base error rate is not too high, the default
very_sensitive_callershould just work. But, if it's tricker cases like ONT reads, having a separate candidate generation process which is smarter about proposing candidates, and then usingvcf_candidate_importeris the right way to go.@maryawood Can you say a bit more about why the default
vcf_candidate_importerdoesn't work for you? Do you get very imbalanced training data?maryawood commentedon Mar 30, 2021
Hi @pichuan, sorry for the slow response! I've only tried
very_sensitive_caller, and while it seems to be working alright, I'm getting lower-than-desired precision and quite low recall, even when adding additional training data or adjusting the evaluation metrics. This may be fixable by tuning parameters more, so I'm just trying to get a handle on all the different options available to customize the training process!One other question - I see that including a VCF with population-level allele frequencies is an option. Is there a specific format that the frequencies need to be in for compatibility with DeepVariant? (e.g. do they need to be in a specific INFO field?) Thanks!
pichuan commentedon Apr 7, 2021
Hi @maryawood , given that you're doing training, the issue might not be which candidate generation approach you use.
For your training data, it would be useful to know how many total examples you're training with.
And actually, it'll be helpful if also know the breakdown of class0, class1, class2.
(You can use this version of shuffle_tfrecords_beam.py which will report the number of each class in addition to the total number of examples.
All the topics you're touching here are more advanced topics, which we can't easily provide support for. If it's possible to share more on what you're trying to do, it will be helpful for us to point to the tools that are more relevant to you.
And, if you're curious about the version that uses population-level allele frequencies, please refer to this biorxiv paper and this example on how to use it. But without knowing what you're trying to do, I can't say this will be useful for you or not.