Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Human vs mouse synteny demo #3444

Merged
merged 2 commits into from
Jan 9, 2023
Merged

Human vs mouse synteny demo #3444

merged 2 commits into from
Jan 9, 2023

Conversation

cmdcolin
Copy link
Collaborator

@cmdcolin cmdcolin commented Jan 9, 2023

This PR adds a human vs mouse synteny demo to our example data. I wanted to try this to test some of our scalability limits, which I elaborate on below

It uses the T2T-CHM13 (aka hs1) genome vs mm39, all using data from the UCSC genome browser

Scalability brainstorming

The hs1 vs mm39 liftover.chain.gz file that we use for the SyntenyTrack is 69MB of gzip data, which is 219MB ungzipped. The maximum size of ungzipped data we support is 512MB due to that being the maximum size of strings in chrome, so it comes pretty close to our limits. I can certainly imagine species (plant genomes, etc) that would exceed our limits.

An indexed file format could help us in some cases. We have not thus far focused on indexed file formats, because we were using somewhat small PAF files that could be loaded into memory but scalability concerns are referenced here #2788. But, with indexing, we may not need to download the entire file when accessing a local region on the LGV synteny track (currently, synteny track adapters generally download the entirety of the file. this is an adapter behavior that could be adjusted for)

The bigChain format from UCSC could possibly help as an example of an indexed file format, it is only indexed in "one dimension" e.g. for the query genome and not the target genome, so accessing the data from the target genome would be unindexed. A custom tabix-y style chain format can probably be made also, similar to mafviewer. "2-D" indexed formats would be cool, but may not be available. Also, "biologically", it may be better to have two tracks: "hs1 (query) vs mm39 (target)" and "mm39 (query) vs hs1 (target)", which would mean the 1D indexing is fine.

@github-actions github-actions bot added the needs label triage Needs a label to show in changelog (breaking, enhancement, bug, documentation, or internal) label Jan 9, 2023
@cmdcolin cmdcolin removed the needs label triage Needs a label to show in changelog (breaking, enhancement, bug, documentation, or internal) label Jan 9, 2023
@github-actions github-actions bot added the needs label triage Needs a label to show in changelog (breaking, enhancement, bug, documentation, or internal) label Jan 9, 2023
@cmdcolin cmdcolin added documentation and removed needs label triage Needs a label to show in changelog (breaking, enhancement, bug, documentation, or internal) labels Jan 9, 2023
@cmdcolin cmdcolin force-pushed the human_vs_mouse branch 2 times, most recently from 4c0a998 to 9ee2c94 Compare January 9, 2023 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant