Can't read zipped vcf from stdin #1502
Comments
|
Hi @barslmn, Your input file sample/Test.vcf.gz is already compressed. Is there a specific reason why you are compressing it again? |
|
Hi Diana,
bcftools view outputs plain/text by default so it doesn’t compress an already compressed file.
I got the same error when compressed with bgzip and gzip.
Best, B
```
***@***.***:~$ bcftools –version
bcftools 1.18-14-g2c81697c
Using htslib 1.18-20-g30211d8
Copyright (C) 2023 Genome Research Ltd.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
***@***.***:~$ bgzip –version
bgzip (htslib) 1.17
Copyright (C) 2023 Genome Research Ltd.
***@***.***:~$ gzip –version
gzip 1.12
Copyright (C) 2018 Free Software Foundation, Inc.
Copyright (C) 1993 Jean-loup Gailly.
This is free software. You may redistribute copies of it under the terms of
the GNU General Public License <https://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.
Written by Jean-loup Gailly.
```
```
***@***.***:~$ file sample/Test.vcf
sample/Test.vcf: Variant Call Format (VCF) version 4.2, ASCII text
***@***.***:~$ bgzip sample/Test.vcf
***@***.***:~$ file sample/Test.vcf.gz
sample/Test.vcf.gz: Blocked GNU Zip Format (BGZF; gzip compatible), block length 8258
***@***.***:~$ cat sample/Test.vcf.gz | /home/bar/ensembl-vep/vep –everything –force_overwrite –vcf –pick –format vcf –output_file STDOUT –compress bgzip –fork 1 –database
Smartmatch is experimental at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/AnnotationSource/File.pm line 472.
Use of uninitialized value $ref in join or string at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 282.
Use of uninitialized value in split at /home/bar/ensembl-vep/Bio/EnsEMBL/IO/Parser/BaseVCF4.pm line 446.
Use of uninitialized value $ref in subtraction (-) at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 299.
Argument “d�*3M-^Xt�M-^Ye�E�iM-\n3zκ���1�M-\0M-\f]dfM-^H�0�2~M-^Q…” isn’t numeric in addition (+) at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 299.
Use of uninitialized value in string eq at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 304.
Use of uninitialized value $ref in concatenation (.) or string at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 384.
Argument “d�*3M-^Xt�M-^Ye�E�iM-\n3zκ���1�M-\0M-\f]dfM-^H�0�2~M-^Q…” isn’t numeric in sprintf at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser.pm line 499.
Use of uninitialized value $ref in join or string at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 282.
.
.
.
```
```
***@***.***:~$ gzip -d sample/Test.vcf.gz
***@***.***:~$ file sample/Test.vcf
sample/Test.vcf: Variant Call Format (VCF) version 4.2, ASCII text
***@***.***:~$ gzip sample/Test.vcf
***@***.***:~$ cat sample/Test.vcf.gz | /home/bar/ensembl-vep/vep –everything –force_overwrite –vcf –pick –format vcf –output_file STDOUT –compress bgzip –fork 1 –database
Smartmatch is experimental at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/AnnotationSource/File.pm line 472.
Use of uninitialized value $ref in join or string at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 282.
Use of uninitialized value in split at /home/bar/ensembl-vep/Bio/EnsEMBL/IO/Parser/BaseVCF4.pm line 446.
Use of uninitialized value $ref in subtraction (-) at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 299.
Argument “^Y]d�J��$e�QF��M-^Ss^N��”M-^SG�f!)�6M-^Y�M-^\M-^I^Z=�o2a…“ isn’t numeric in addition (+) at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 299.
Use of uninitialized value in string eq at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 304.
Use of uninitialized value $ref in concatenation (.) or string at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser/VCF.pm line 384.
Argument ”^Y]d�J��$e�QF��M-^Ss^N��“M-^SG�f!)�6M-^Y�M-^\M-^I^Z=�o2a…” isn’t numeric in sprintf at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/Parser.pm line 499.
.
.
.
```
|
|
Can you please send the file Test.vcf.gz to helpdesk@ebi.ac.uk? |
|
I guess mail attachments don't show up on github. I send to helpdesk also attaching here. |
|
Thank you for sending your file. We are investigating why VEP does not read a file compressed with However, your file works in vep: |
|
Hi Diana, thanks for looking into it.
I recommend you input this file directly into VEP.
I am just piping in as plain text.
I opened the issue because docs say it can read compressed from the stdin.
Best, B
|
|
That's correct, VEP is supposed to read the compressed file. |
|
Yes this works as expected since zcat decompresses.
┌────
│ ***@***.***:~$ bcftools view sample/Test.vcf.gz -Oz | zcat | /home/bar/ensembl-vep/vep --everything --force_overwrite --vcf --pick --format vcf --output_file STDOUT --fork 1 --cache | sed 5q
│ Smartmatch is experimental at /home/bar/ensembl-vep/modules/Bio/EnsEMBL/VEP/AnnotationSource/File.pm line 472.
│ ##fileformat=VCFv4.2
│ ##FILTER=<ID=PASS,Description="All filters passed">
│ ##bcftoolsVersion=1.18-14-g2c81697c+htslib-1.18-20-g30211d8
│ ##bcftoolsCommand=mpileup --threads 1 -Ou -A -d 10000 -L 10000 -a FORMAT/AD,FORMAT/DP -f /home/bar/reference/GRCh38/Homo_sapiens_assembly38.fasta -
│ ##reference=file:///home/bar/reference/GRCh38/Homo_sapiens_assembly38.fasta
│
└────
Bcftools can output directly as plain text too.
┌────
│ bcftools view Test.vcf.gz -Ov | perl vep --everything --force_overwrite --vcf --pick --format vcf --output_file STDOUT --compress bgzip --fork 1 --cache
└────
|
|
For the input file (option |
|
I'm going to close this issue. If you have more questions please feel free to open a new issue. Best wishes, |
Describe the issue
Can't read zipped vcf from stdin.
System
Full VEP command line
This works:
This gives error (difference is -Oz in bcftools):
Full error message
The text was updated successfully, but these errors were encountered: