bcftools norm error

[Fazulur]

Dear bcftools team,

I got the below error when I tried to normalize vcf (generated using gatk jointcalling for genome build 37) using bcftools 1.9 version.
Error: wrong number of fields in FMT/MMQ at 1:10146, expected 14474, found 21711

Command I used
bcftools view --threads 16 -r 1 sample.vcf.gz |bcftools norm --threads 16 -m -any -O b -o test.bcf.gz

Could you please help me to resolve this issue.

Thanks In Advance
Fazulur Rehaman

[pd3]

This looks like a big VCF. The error message suggests that the FORMAT annotation MMQ across all samples has different number of values than expected. Can you share the output of

bcftools view sample.vcf.gz -r 1:10146 -Oz -o test.vcf.gz

eg send it to my email listed on the profile page?

[pd3]

Thank you for the test case. It seems you are using an older version of bcftools, as otherwise it should have failed with the following error message

[E::bcf_hdr_add_sample_len] Empty sample name: trailing spaces/tabs in the header line?

After removing the trailing blank spaces, I can run

$ bcftools query -f '[%REF,%ALT  %MMQ\n]' test.vcf |  awk '$2!="."' 
AC,A,*  40,0
AC,A,*  36,0
AC,A,*  40,0
AC,A,*  40,0
AC,A,*  30,0
AC,A,*  32,0
AC,A,*  25,0
AC,A,*  36,0
AC,A,*  35,0
AC,A,*  27,0
AC,A,*  39,0
AC,A,*  37,0
AC,A,*  0,25,0
AC,A,*  34,0

and see that although the MMQ tag is defined as Number=A in the header, only one of the samples has the correct count of three values but the rest has only two. This should be fixed in the program that produced the VCF, ideally a bug report should be issued. If you don't need the tag, a quick work around for you is to remove the offending tag by running bcftools annotate -x FMT/MMQ.

[Fazulur]

Dear Petr Danecek,

Thanks a lot for your quick resposne & resolving the issue.
I will submit bug report to GATK regarding this issue. For now, i will remove MMQ tag from my VCF file.

Thanks & Regards
Fazulur Rehaman

[zofieLin]

Hi,
I also suffered a similar error when using bcftools norm.
I was using bcftools/1.11,
bcftools norm -m-both -o test.norm.vcf test.vcf
and the error message is
Error: wrong number of fields in INFO/PG at chr6:44095266, expected 6, found 10

below is the INFO/PG in the vcf file

chr6 44095266 . T C,* . . AC=1,1;AF=0.056,0.056;AN=18;BaseQRankSum=-0.556;ClippingRankSum=0;DP=121;ExcessHet=4.8688;FS=196.232;InbreedingCoeff=0.2413;MQ=53.06;MQRankSum=0;PG=0,16,36,19,38,41,16,35,38,36;QD=13.53;ReadPosRankSum=-1.614;SOR=5.416

It could be solved by removing the PG tag ##INFO=<ID=PG in the header of vcf, but I guess it may not be a good solution, it there any good suggestion for this problem?

Many thanks!

[pd3]

@zofieLin you can redefine the tag as Number=. and reheader

bcftools view -h file.vcf > hdr.txt
# edit hdr.txt, make PG tag Number=.
bcftools reheader -h hdr.txt -o fixed.vcf file.vcf

[ManuelMoradiellos]

Hi sorry for opening up this issue again but I have a question regarding the fix of changing the tag Number=. .

In my case I'm working with a raw TCGA germline data VCFv4.2 and using bcftools 1.16 .

I first filtered my .vcf to only keep the variants with 'FILTER="PASS"' to then split the .vcf due to the multivariants, but I the following error pretty early:

Error: wrong number of fields in FMT/AD at 1:15053, expected 31167, found 20778

I've tried to take a look at what may be missing but I don't understand why it's returning such as big number for AD and its something that I must keep.

If I try the Number=. would it affect further steps of my annotation pipeline? Because I'm not sure that working with a .vcf file with different number of fields across variants could lead to further errors...

Thanks in advance and sorry if this is a basic question~!

[pd3]

@ManuelMoradiellos please do not append to existing and closed issues, as such will be easily overlooked