<*> allele vs <NON_REF>

[lbergelson]

In the VCF 4.3 spec the <*> is specified as the unspecified alt allele.

GATK uses the <NON_REF> allele instead for this purpose and treats <*> as a deprecated version of the * allele. This is obviously not matching the spec. Are there other software suites that generate GVCFs that use <*>?

@pd3 I'm wondering why <*> was selected as the canonical unspecified alt. Do other common callers produce gVCFs with <*>?

[pd3]

Historically, samtools was the first caller to start using the unspecified allele and originally it was *. After this was codified in the specification as <*>, samtools/bcftools started producing <*> instead. I don't know why GATK uses <NON_REF> instead of <*>.

[lbergelson]

Interesting, the history seems complicated. I did bit of digging to try and understand it.

• It looks like samtools used X and then <X> as well before moving to <*>.

• According to git history samtools adopted <*> in 2015 (samtools/samtools@78c2343)

• bcftools supported it in 2014 (samtools/bcftools@802ff30)

• The spec included both <X> and <NON_REF> up until 2014, when it changed to standardize on <*> (4a91745)

• GATK started using<NON_REF> since 2013 so it's pretty old. (broadgsa/gatk-protected@7c3cd48)

Unfortunately GATK never got the message to change to <*>, and it doesn't seem likely to happen due to how many <NON_REF> gvcfs there are out there now.

It might be worth considering adding back <NON_REF> to the spec since practically it's used in petabytes of gvcf files and it's unlikely to go away.

[yfarjoun]

Given that we have consistently said that gVCFs are NOT deliverable, I think that we have an opportunity to change the <NON_REF> allele at EVERY GATK version...

…

On Tue, Oct 16, 2018 at 4:53 AM Petr Danecek ***@***.***> wrote: Historically, samtools was the first caller to start using the unspecified allele and originally it was *. After this was codified in the specification as <*>, samtools/bcftools started producing <*> instead. I don't know why GATK uses <NON_REF> instead of <*>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#352 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACnk0klF-KcFTAOygOlmd8GnUpOMeFYfks5ulZ6IgaJpZM4XdDxd> .

[nh13]

@yfarjoun my best google-fu could not find a list of supported or unsupported deliverables for GATK, but searching for ‘<NON_REF>’ did yield a non-trivial number of tools that rely or leverage the allele. So likely there are gVCFs that are stored that have this allele, waiting to be combined in joint calling mid-project, or otherwise. I just don’t buy your argument.

I’d argue that GATK switch over to the spec and perhaps add an option to output the old allele for a while.

[yfarjoun]

@nh13 I agree that GATK hasn't public about what is or isn't a deliverable, but we have said that there should be a consistency between the version that created the gVCF and the one that combines/genotypes them.

Regardless, we agree on the conclusion, that GATK can switch to the spec, rather than the spec changing.

[pd3]

Yes, that would be great if GATK could switch to the spec.

@lbergelson You are right, completely forgot about the X and <X>.

[tfenne]

@yfarjoun & @lbergelson I would think it's probably feasible to have GATK recognize <NON_REF> and <*> as the unspecified non-ref allele, in perpetuity, while switching to producing <*> at some point? There's no reason GATK can't read both of those alleles and treat them the same, no?

[yfarjoun]

It is, of course, feasible, but I'm actually concerned about * vs. <*>. since we will have a hard time distinguishing (in speech) between "star allele" meaning "spanning deletion" and "star allele" meaning "unspecified allele"...

[tfenne]

Ah, yes, good point.

[lbergelson]

To make it even more confusing, it seems like at some point we used <*> to mean spanning deletion, and some code accepts either <*> or * as spanning deletions.

[pd3]

@yfarjoun It can't be that hard, you just did it! ;-)

[trutane]

I must admit, having <*> and * mean different things is a degree of subtlety that makes me nervous. There is already mounting FUD about it, with someone claiming it indicates homozygous reference sites.

I think there is ample opportunity for ambiguity in the v4.3 spec regarding how to interpret *'s in various contexts, comparing sections 1.6.1 vs 5.5:

1.6.1, Fixed fields, ALT:

...an angle-bracketed ID String (“<ID>”) or a breakend replacement string as described in the section on breakends. The ‘*’ allele is reserved to indicate that the allele is missing due to an overlapping deletion.

5.5, Representing unspecified alleles and REF-only blocks (gVCF):

A symbolic alternate allele <*> is used to represent this unspecified alternate allele.

If the spec docs were more clear about this distinction, and GATK et al. tools can be updated to be spec-consistent, I guess I could live with it, though it still might make me a bit queasy...