The height of the input image is not 100 (standard in DeepVariant) or 300 (standard in DeepTrio).

[Suke-fudan]

I ran DeepVariant twice based on "https://github.com/google/deepvariant/blob/r1.1/docs/deepvariant-pacbio-model-case-study.md".
deepvariant1- whatshap phase- whatshap haplotag-deepvariant2
Now I also want to use DeepTrio.I used the haplotagged.bam(generated from whatshap haplotag) as input bam.When I ran DeepTrio,the output.vcf.gz was generated normally.However,the log file showed the following warning message:

I0926 14:26:35.659228 47028170803008 call_variants.py:336] Shape of input examples: [140, 221, 9]
W0926 14:26:35.665323 47028170803008 call_variants.py:353] The height of the input image is not 100 (standard in DeepVariant) or 300 (standard in DeepTrio). Please double-check that the model is trained with the same parameters and version of DeepVariant as you generated the examples with. An error will not appear when these are mismatched because of how InceptionV3 works. Note that if you set --pileup_image_height in DeepVariant, then you must use a model trained with that same parameter.
2021-09-26 14:26:35.668419: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-09-26 14:26:35.669638: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
2021-09-26 14:26:35.671197: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2600000000 Hz
WARNING:tensorflow:Using temporary folder as model directory: /TMP_DIR/tmpbptqemkc
W0926 14:26:35.690017 47028170803008 estimator.py:1846] Using temporary folder as model directory: /TMP_DIR/tmpbptqemkc

The version I used:
DeepVariant 1.1.0
glnexus v1.3.1
whatshap 1.0
DeepTrio 1.2.0

Does the warning message affect the results or can be ignored?
Should I use a higher version of DeepVariant(1.2.0)?

[danielecook]

@Suke-fudan did you retain a copy of the original commands that you ran?

Your pileup images appear to be 140 reads. This value is normally set by default to be 100 or 300. Did you possibly set this somewhere? Can you reply with the commands and flags you used?

[Suke-fudan]

Yes,the original commands are as follows.

#deepvariant1 version:1.1.0
/opt/deepvariant/bin/run_deepvariant
--model_type PACBIO
--ref ${ref}
--reads ${bam}
--output_vcf ${outdir1}/${sample}.dv1.vcf.gz
--output_gvcf ${outdir1}/${sample}.dv1.g.vcf.gz
--num_shards 8

#whatshap phase version:1.0
whatshap phase
--output ${outdir2}/${sample}_deepvariant1.phased.vcf.gz
--reference ${ref}
${outdir1}/${sample}.dv1.vcf.gz
${bam}
#whatshap haplotag version:1.0

tabix -p vcf ${outdir2}/${sample}_deepvariant1.phased.vcf.gz

whatshap haplotag \
--output ${outdir2}/${sample}_haplotagged.bam \ #generate "C1_haplotagged.bam"、"F1_haplotagged.bam"、"M1_haplotagged.bam"
--reference ${ref}
${outdir2}/${sample}_deepvariant1.phased.vcf.gz
${bam}

samtools index ${outdir2}/${sample}_haplotagged.bam

#DeepTrio version:1.2.0
/opt/deepvariant/bin/deeptrio/run_deeptrio
--model_type PACBIO
--ref ${ref}
--reads_child ${outdir2}/C1_haplotagged.bam
--reads_parent1 ${outdir2}/F1_haplotagged.bam
--reads_parent2 ${outdir2}/M1_haplotagged.bam
--output_vcf_child ${outdir4}/C1.output.vcf.gz \
--output_vcf_parent1 ${outdir4}/F1.output.vcf.gz
--output_vcf_parent2 ${outdir4}/M1.output.vcf.gz
--sample_name_child 'C1'
--sample_name_parent1 'F1'
--sample_name_parent2 'M1'
--num_shards 8
--output_gvcf_child ${outdir4}/C1.g.vcf.gz
--output_gvcf_parent1 ${outdir4}/F1.g.vcf.gz
--output_gvcf_parent2 ${outdir4}/M1.g.vcf.gz
--use_hp_information

However, I didn't remember I set that value.This thing is a little weird.

[pichuan]

Note that not all our DeepTrio models use the default height in the code (which is the 100+100+100=300 that @danielecook is referring to).

See here:
https://github.com/google/deepvariant/blob/r1.2/scripts/run_deeptrio.py#L181-L184

@Suke-fudan please considering using the one-step run_deeptrio.py script for this. See this section:
https://github.com/google/deepvariant/blob/r1.2/docs/deeptrio-quick-start.md#run-deeptrio-with-one-command

And, if you really want to break it down to multiple steps, you can still use run_deeptrio.py and add --dry_run=true to see the command breakdown.

So, basically for PACBIO DeepTrio, you'll want to set:
--pileup_image_height_child=60 and --pileup_image_height_parent=40
in your make_examples stage.

Does this make sense? Let me know if that works for you.

[pichuan]

Oh, btw, the warning message in call_variants.py about image height might be confusing.
It is just a warning, so you should still be able to run. But I agree that this is confusing. We'll think about how to make things more robust.

For now, I would still recommend the one-command approach: https://github.com/google/deepvariant/blob/r1.2/docs/deeptrio-quick-start.md#run-deeptrio-with-one-command
which is our current way of making things more robust.

[pichuan]

And to clarify - it is important that during calling, you have to create examples that have the same heights as when we trained the release models.

Which means you have to specify --pileup_image_height_child=60 and --pileup_image_height_parent=40 when applying WGS and PACBIO DeepTrio models.

(And you have to have them as 100 if you're applying WES DeepTrio models.)

If you run make_examples with dimensions that are different from how we train our release models, all steps might run through fine, but the accuracy will be completely off!

[Suke-fudan]

Thank you for your answer. The reference I ran the the original commands is https://github.com/google/deepvariant/blob/r1.2/docs/deeptrio-pacbio-case-study.md

The data I uesd is HiFi reads of a family of four.

The --use_hp_information arg makes use of a phased reads, thus allowing a further improvement of the accuracy. In order to use this feature input BAM files have to be phased. For the detailed description on how to do that please see DeepVariant PacBio case study.https://github.com/google/deepvariant/blob/r1.2/docs/deepvariant-pacbio-model-case-study.md

The "deepvariant1-whatshap phase-whatshap haplotag" was used to generate phased bam file.(reference:https://github.com/google/deepvariant/blob/r1.2/docs/deepvariant-pacbio-model-case-study.md)

Should I use a higher version of DeepVariant(1.2.0) or regenerate the phased bam file?

[danielecook]

@Suke-fudan sorry for the earlier confusion regarding pileup height.

You should use the latest version of DeepVariant (1.2.0), but this version is not used to perform phasing. Phasing is performed by whatshap. However, DeepVariant 1.2.0 is required in order to make use of phased bams when calling variants.

[Suke-fudan]

Thank you for your help!So,the correct commands may be:
#①
#deepvariant1 version:1.2.0
/opt/deepvariant/bin/run_deepvariant
--model_type PACBIO
--ref ${ref}
--reads ${bam}
--output_vcf ${outdir1}/${sample}.dv1.vcf.gz
--output_gvcf ${outdir1}/${sample}.dv1.g.vcf.gz
--num_shards 8
#②
#whatshap phase version:1.0
whatshap phase
--output ${outdir2}/${sample}_deepvariant1.phased.vcf.gz
--reference ${ref}
${outdir1}/${sample}.dv1.vcf.gz
${bam}

#③generate phased bam to use DeepTrio1.2.0
#whatshap haplotag version:1.0

tabix -p vcf ${outdir2}/${sample}_deepvariant1.phased.vcf.gz

whatshap haplotag
--output ${outdir2}/${sample}_haplotagged.bam \ #generate "C1_haplotagged.bam"、"F1_haplotagged.bam"、"M1_haplotagged.bam"
--reference ${ref}
${outdir2}/${sample}_deepvariant1.phased.vcf.gz
${bam}

samtools index ${outdir2}/${sample}_haplotagged.bam

④The final result of variant calling
#DeepTrio version:1.2.0
/opt/deepvariant/bin/deeptrio/run_deeptrio
--model_type PACBIO
--ref ${ref}
--reads_child ${outdir2}/C1_haplotagged.bam
--reads_parent1 ${outdir2}/F1_haplotagged.bam
--reads_parent2 ${outdir2}/M1_haplotagged.bam
--output_vcf_child ${outdir4}/C1.output.vcf.gz
--output_vcf_parent1 ${outdir4}/F1.output.vcf.gz
--output_vcf_parent2 ${outdir4}/M1.output.vcf.gz
--sample_name_child 'C1'
--sample_name_parent1 'F1'
--sample_name_parent2 'M1'
--num_shards 8
--output_gvcf_child ${outdir4}/C1.g.vcf.gz
--output_gvcf_parent1 ${outdir4}/F1.g.vcf.gz
--output_gvcf_parent2 ${outdir4}/M1.g.vcf.gz
--use_hp_information

[Suke-fudan]

I used the latest version of DeepVariant (1.2.0) and ran the commands：
#①
#deepvariant1 version:1.2.0
/opt/deepvariant/bin/run_deepvariant
--model_type PACBIO
--ref ${ref}
--reads ${bam}
--output_vcf ${outdir1}/${sample}.dv1.vcf.gz
--output_gvcf ${outdir1}/${sample}.dv1.g.vcf.gz
--num_shards 8
#②
#whatshap phase version:1.0
whatshap phase
--output ${outdir2}/${sample}_deepvariant1.phased.vcf.gz
--reference ${ref}
${outdir1}/${sample}.dv1.vcf.gz
${bam}

#③generate phased bam to use DeepTrio1.2.0
#whatshap haplotag version:1.0

tabix -p vcf ${outdir2}/${sample}_deepvariant1.phased.vcf.gz

whatshap haplotag
--output ${outdir2}/${sample}_haplotagged.bam \ #generate "C1_haplotagged.bam"、"F1_haplotagged.bam"、"M1_haplotagged.bam"
--reference ${ref}
${outdir2}/${sample}_deepvariant1.phased.vcf.gz
${bam}

samtools index ${outdir2}/${sample}_haplotagged.bam

④The final result of variant calling
#DeepTrio version:1.2.0
/opt/deepvariant/bin/deeptrio/run_deeptrio
--model_type PACBIO
--ref ${ref}
--reads_child ${outdir2}/C1_haplotagged.bam
--reads_parent1 ${outdir2}/F1_haplotagged.bam
--reads_parent2 ${outdir2}/M1_haplotagged.bam
--output_vcf_child ${outdir4}/C1.output.vcf.gz
--output_vcf_parent1 ${outdir4}/F1.output.vcf.gz
--output_vcf_parent2 ${outdir4}/M1.output.vcf.gz
--sample_name_child 'C1'
--sample_name_parent1 'F1'
--sample_name_parent2 'M1'
--num_shards 8
--output_gvcf_child ${outdir4}/C1.g.vcf.gz
--output_gvcf_parent1 ${outdir4}/F1.g.vcf.gz
--output_gvcf_parent2 ${outdir4}/M1.g.vcf.gz
--use_hp_information

However,the log file of DeepTrio still contains errors：(The result files have been generated)
cat log |grep -i error
W1008 21:26:50.592375 47245352568640 call_variants.py:353] The height of the input image is not 100 (standard in DeepVariant) or 300 (standard in DeepTrio). Please double-check that the model is trained with the same parameters and version of DeepVariant as you generated the examples with. An error will not appear when these are mismatched because of how InceptionV3 works. Note that if you set --pileup_image_height in DeepVariant, then you must use a model trained with that same parameter.
W1009 09:06:50.674110 47029842433856 call_variants.py:353] The height of the input image is not 100 (standard in DeepVariant) or 300 (standard in DeepTrio). Please double-check that the model is trained with the same parameters and version of DeepVariant as you generated the examples with. An error will not appear when these are mismatched because of how InceptionV3 works. Note that if you set --pileup_image_height in DeepVariant, then you must use a model trained with that same parameter.
W1009 18:31:59.770313 48004354086720 call_variants.py:353] The height of the input image is not 100 (standard in DeepVariant) or 300 (standard in DeepTrio). Please double-check that the model is trained with the same parameters and version of DeepVariant as you generated the examples with. An error will not appear when these are mismatched because of how InceptionV3 works. Note that if you set --pileup_image_height in DeepVariant, then you must use a model trained with that same parameter.
raise ValueError('call_variants_outputs did not pass sanity check.')
ValueError: call_variants_outputs did not pass sanity check.

tail -n 50 log
Traceback (most recent call last):
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/postprocess_variants.py", line 1249, in
tf.compat.v1.app.run()
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/absl_py/absl/app.py", line 299, in run
_run_main(main, args)
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/absl_py/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/postprocess_variants.py", line 1225, in main
merge_and_write_variants_and_nonvariants(variant_generator,
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/postprocess_variants.py", line 1012, in merge_and_write_variants_and_nonvariants
variant = next_or_none(variant_iterable)
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/postprocess_variants.py", line 998, in next_or_none
return next(iterable)
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/haplotypes.py", line 87, in maybe_resolve_conflicting_variants
for overlapping_candidates in _group_overlapping_variants(sorted_variants):
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/haplotypes.py", line 106, in _group_overlapping_variants
for variant in sorted_variants:
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/postprocess_variants.py", line 853, in _transform_call_variants_output_to_variants
canonical_variant, predictions = merge_predictions(
File "/TMP_DIR/Bazel.runfiles_eyn11z72/runfiles/com_google_deepvariant/deepvariant/postprocess_variants.py", line 717, in merge_predictions
raise ValueError('call_variants_outputs did not pass sanity check.')
ValueError: call_variants_outputs did not pass sanity check.

real 15m9.448s
user 14m3.211s
sys 0m20.620s
I1010 03:07:07.182588 47710997727040 postprocess_variants.py:1233] Finished writing VCF and gVCF in 36.08515272140503 minutes.
I1010 03:07:07.600154 47710997727040 genomics_reader.py:222] Reading /DeepTrio/C1/F1.output.vcf.gz with NativeVcfReader
I1010 03:09:40.323815 48001616553792 postprocess_variants.py:1233] Finished writing VCF and gVCF in 38.57903414567311 minutes.
I1010 03:09:40.659351 48001616553792 genomics_reader.py:222] Reading /DeepTrio/C1/C1.output.vcf.gz with NativeVcfReader

real 45m21.485s
user 43m7.954s
sys 1m1.029s

real 47m46.669s
user 45m31.656s
sys 1m0.352s
Traceback (most recent call last):
File "/opt/deepvariant/bin/deeptrio/run_deeptrio.py", line 659, in
app.run(main)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "/opt/deepvariant/bin/deeptrio/run_deeptrio.py", line 655, in main
raise Exception('One or more postprocess_variants failed.')
Exception: One or more postprocess_variants failed.

I am very confusing :Are the result files reliable?Or should I rerun DeepTrio?

[pichuan]

Hi @Suke-fudan , if your confusion is mainly about this line The height of the input image is not 100 (standard in DeepVariant) or 300 (standard in DeepTrio). , please ignore it for now.

Internally we plan to remove that warning because it is confusing.

To be clear:

• Our DeepTrio WGS and PACBIO models are trained with child height=60, and parents height=40. (Therefore 140 total).
• Our DeepTrio WES model was trained with child height and parents height=100, which is 300 total.

If you're running PACBIO or WGS, you will see the (incorrect) warning about 140 isn't standard. If that's the case, please feel free to ignore that warning. We will improve in the future.

[Suke-fudan]

Thanks for your help! I can ignore the warning about "height".
However,when I ran Deepvariant 1.1.0 and DeepTrio 1.2.0,the only error was about "height". When I ran Deepvariant 1.2.0 and DeepTrio 1.2.0,the errors were not only about "height",but also about "ValueError" like this:
**raise ValueError('call_variants_outputs did not pass sanity check.')
ValueError: call_variants_outputs did not pass sanity check.

raise Exception('One or more postprocess_variants failed.')
Exception: One or more postprocess_variants failed.**

How should I deal with these errors？

[danielecook]

@Suke-fudan - I am not entirely sure why this error has come up. Can you double check that you are using the correct reference genome?

Are you able to identify which region of the genome this error is occurring at?

Would you be comfortable sharing a file that allows us to reporduce the error?

[Suke-fudan]

Thank you for your kind help! In fact ,I reran the same commands and got the result files without any error message.

[pichuan]

Thanks @Suke-fudan for your update.
And thanks for reporting the confusing warning message.
In the next release, we plan to store the input shape for how each model is trained on, and we'll update our logging message to be more accurate.