Generating embeddings after model training

[katiana22]

Hello,

Thank you for this great work!

I am interested in generating embeddings from a gpn model I've trained. Specifically I would like to re-create a plot similar to UMAP Fig.2 in the paper. I can see from the embedding_umap notebook that 2 parquet files are used, one with the windows and one with model embeddings. How can I generate the windows.parquet file for arabidopsis, or other species?

Also if I want to run this analysis on a different species/reference genomes, do I need to significantly adjust the Snakefile and any other file/folder to reproduce your analysis similarly to arabidopsis?

Thank you!

[gonzalobenegas]

Hello!

windows.parquet is produced by rules download_annotation, expand_annotation and define_embedding_windows. It needs two input files:

• Gene annotation (gff/gtf): we obtained it from Ensembl Plants, but there are many other sources, which might vary slightly in format.
• RepeatMasker file: can often be downloaded from the UCSC Genome Browser. We obtained from the Genome Browser from PlantRegMap, which also contains for many other plants. The official UCSC Genome Browser would contain for other species such as vertebrates.

I don't think you would encounter significant issues porting this to another species. Let me know!

[katiana22]

Thank you for your quick response - this is helpful!

When I run snakemake --cores all I get the following error:

AttributeError in file /root/repos/gpn-main/analysis/arabidopsis/Snakefile, line 150:
'str' object has no attribute 'id'

If I understand it correctly the code reads the genome.raw.fa.gz and id_mapping.tsv files and then creates the genome.fa.gz file, but I get an error from this line: genome[chrom].id = id_mapping[genome[chrom].id] in the Snakefile. The genome[chrom] is a str and does not have an id argument.

Any help would be greatly appreciated!

[gonzalobenegas]

Sorry for this... It probably worked with a previous version of the Genome class. I'd recommend just skip this step, it was unnecessary in the end. I'd just download the genome from Ensembl:
wget -O output/genome.fa.gz https://ftp.ensemblgenomes.ebi.ac.uk/pub/plants/release-57/fasta/arabidopsis_thaliana/dna/Arabidopsis_thaliana.TAIR10.dna_sm.toplevel.fa.gz

[katiana22]

This worked! Thank you very much!