Are non-human reference genomes supported?

[programmingprincess]

I am attempting to run the example script (https://github.com/raphael-group/hatchet/blob/master/script/runHATCHet.sh) on pig data. In the deBAF step, I am getting the error: The given reference cannot be used because the chromosome names are inconsistent!

The command being used is

python /home/jiaqiwu6/hatchet/utils/deBAF.py -N /scratch/data/oncopig/kidney_RG.dedup.bam -T /scratch/data/oncopig/tumor1_RG.dedup.bam /scratch/data/oncopig/tumor2_RG.dedup.bam /scratch/data/oncopig/tumor3_RG.dedup.bam /scratch/data/oncopig/tumor4_RG.dedup.bam /scratch/data/oncopig/tumor5_RG.dedup.bam /scratch/data/oncopig/cell_line_RG.dedup.bam -S Normal tumor1 tumor2 tumor3 tumor4 tumor5 tumor0 -r /scratch/data/oncopig/ref/sus11.1.fa -j 22 -q 20 -Q 20 -U 20 -c 4 -C 300 -O /scratch/data/oncopig/hatchet_script/baf/normal.baf -o /scratch/data/oncopig/hatchet_script/baf/bulk.baf -v

I wonder if this is because I used -g hg19 in the binBAM step. Are non-human reference genomes supported (if so, where can I specify this)? Alternatively, can we exclude certain chromosomes from the computation entirely?

Thanks in advance.

[simozacca]

HATCHet can be applied with any reference genome of a diploid organism, however the current implementation needs to be aware of some details of the reference: in particular, it needs to know the length of the chromosomes in the reference. This is why there is a list of supported references. If you could please provide the name and the details of your reference genome (especially the chromosome lengths), I will be very happy to integrate immediately that reference among those supported.

[programmingprincess]

Thanks! I could not find the list of supported references. The reference genome I am working with is sus scrofa 11.1

[simozacca]

We are gonna release a new version in the next couple of days which will automatically support any reference genome, without the need to specify the name.

I will update this post as soon as it will be released

[simozacca]

@programmingprincess The current version of HATCHet is now supporting any reference genome; in particular, now binBAM is provided directly with the reference genome itself (which is required to be properly indexed) and the size of the chromosomes are read from the corresponding dictionary (as default, same name of the indexed reference with .dict extension).

Please let us know if you can test the current version with your samples and reference genome and we keep the issue open for now.

[programmingprincess]

@simozacca Just tested the current version with the reference genome, it looks good! Thanks for the follow up 👍

[JohnUrban]

Hello- thanks for the tool.

I am having trouble using this tool with a non-human reference. All in the same dir are : genome.fasta, genome.fasta.fai, and genome.dict.

I tried running the whole with a hatchet.ini file as discussed here:
http://compbio.cs.brown.edu/hatchet/script/README.html

hatchet run hatchet.ini

And I tried just

hatchet count-reads -N ${NORMAL} -T ${PRIME} -b 25kb -g ${REF} -j 16 -O XX -o XpX -t total_read.counts --verbose

Both give this error:

WARNING: a chromosome named either 1 or a variant of CHR1 cannot be found in the normal BAM file
WARNING: a chromosome named either 2 or a variant of CHR2 cannot be found in the normal BAM file
WARNING: a chromosome named either 3 or a variant of CHR3 cannot be found in the normal BAM file
WARNING: a chromosome named either 4 or a variant of CHR4 cannot be found in the normal BAM file
WARNING: a chromosome named either 5 or a variant of CHR5 cannot be found in the normal BAM file
WARNING: a chromosome named either 6 or a variant of CHR6 cannot be found in the normal BAM file
WARNING: a chromosome named either 7 or a variant of CHR7 cannot be found in the normal BAM file
WARNING: a chromosome named either 8 or a variant of CHR8 cannot be found in the normal BAM file
WARNING: a chromosome named either 9 or a variant of CHR9 cannot be found in the normal BAM file
WARNING: a chromosome named either 10 or a variant of CHR10 cannot be found in the normal BAM file
WARNING: a chromosome named either 11 or a variant of CHR11 cannot be found in the normal BAM file
WARNING: a chromosome named either 12 or a variant of CHR12 cannot be found in the normal BAM file
WARNING: a chromosome named either 13 or a variant of CHR13 cannot be found in the normal BAM file
WARNING: a chromosome named either 14 or a variant of CHR14 cannot be found in the normal BAM file
WARNING: a chromosome named either 15 or a variant of CHR15 cannot be found in the normal BAM file
WARNING: a chromosome named either 16 or a variant of CHR16 cannot be found in the normal BAM file
WARNING: a chromosome named either 17 or a variant of CHR17 cannot be found in the normal BAM file
WARNING: a chromosome named either 18 or a variant of CHR18 cannot be found in the normal BAM file
WARNING: a chromosome named either 19 or a variant of CHR19 cannot be found in the normal BAM file
WARNING: a chromosome named either 20 or a variant of CHR20 cannot be found in the normal BAM file
WARNING: a chromosome named either 21 or a variant of CHR21 cannot be found in the normal BAM file
WARNING: a chromosome named either 22 or a variant of CHR22 cannot be found in the normal BAM file
Traceback (most recent call last):
  File "/mnt/sequence/jurban/software/conda/anaconda3/envs/hatchet/bin/hatchet", line 33, in <module>
    sys.exit(load_entry_point('hatchet==0.4.7', 'console_scripts', 'hatchet')())
  File "/mnt/sequence/jurban/software/conda/anaconda3/envs/hatchet/lib/python3.7/site-packages/hatchet/__main__.py", line 65, in main
    globals()[command](args)
  File "/mnt/sequence/jurban/software/conda/anaconda3/envs/hatchet/lib/python3.7/site-packages/hatchet/utils/count_reads.py", line 16, in main
    args = ap.parse_count_reads_arguments(args)
  File "/mnt/sequence/jurban/software/conda/anaconda3/envs/hatchet/lib/python3.7/site-packages/hatchet/utils/ArgParsing.py", line 277, in parse_count_reads_arguments
    chromosomes = extractChromosomes(samtools, normal, samples)
  File "/mnt/sequence/jurban/software/conda/anaconda3/envs/hatchet/lib/python3.7/site-packages/hatchet/utils/ArgParsing.py", line 520, in extractChromosomes
    if len(chrm) == 0 and len(no_chrm) == 0: raise ValueError("No chromosomes found in the normal BAM")
ValueError: No chromosomes found in the normal BAM

I installed hatchet the following way:

conda create -n hatchet -c bioconda hatchet
conda activate hatchet
conda install tabix picard mosdepth
conda install shapeit -c dranew
conda install -c gurobi gurobi

Tell Ben I say hello. I knew him when he was at Brown, and sat in on some of his lab meetings.

[JohnUrban]

p.s. Im guessing the offending code is this:

505 def extractChromosomes(samtools, normal, tumors, reference=None):
    506     # Read the names of sequences in normal BAM file
    507     normal_sq = getSQNames(samtools, normal[0])
    508 
    509     # Extract only the names of chromosomes in standard formats
    510     chrm = set()
    511     no_chrm = set()
    512     for i in range(1, 23):
    513         if str(i) in normal_sq:
    514             no_chrm.add(str(i))
    515         elif "chr" + str(i) in normal_sq:
    516             chrm.add("chr" + str(i))
    517         else:
    518             sys.stderr.write("WARNING: a chromosome named either {} or a variant of CHR{} cannot be found in the normal BAM file\n".format(i, i))
    519 
    520     if len(chrm) == 0 and len(no_chrm) == 0: raise ValueError("No chromosomes found in the normal BAM")

This appears to be hard-coded to expect human chromosomes.

I work with an insect - the chromosome names are: X, II, III, IV, and ....associated_contig_k... (for k in 1:N).

[simozacca]

Unfortunately the current version of HATCHet does not support genomes with different chromosome names than those in the human genome. Also, please note that HATCHet can only properly work with diploid genomes.

We will consider the extension in future developments, thank you!

[JohnUrban]

Thanks for letting me know. When you say "diploid genomes", are you referring to the ploidy of the organism or the assembly? The insect I work with is diploid (well it some cells polyploidize, but most somatic cells are diploid). The current assembly is collapsed into 1 haploid complement.

[simozacca]

I was referring to the ploidy of the organism; sorry I did not know that this insect was diploid. We think that we might be able to extend HATCHet to work with arbitrary chromosomes and we are working on it. Hopefully we might have new developments soon.