Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large GFF3Tabix stresses machine #1195

Closed
cmdcolin opened this Issue Aug 27, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@cmdcolin
Copy link
Contributor

cmdcolin commented Aug 27, 2018

It appears that loading the NCBI annotation transformed into GFF3Tabix causes some stress on the system

Here is a profiling screenshot showing 50 seconds needed to display file loaded from localhost on browser. The data directory for a Chr1 from NCBI GRCh38 is attached here too

log

ncbi_human_gff.tar.gz

@cmdcolin

This comment has been minimized.

Copy link
Contributor Author

cmdcolin commented Aug 27, 2018

Possibly move to @gmod/gff-js

@cmdcolin

This comment has been minimized.

Copy link
Contributor Author

cmdcolin commented Aug 29, 2018

I think likely it is not a problem with gff-js itself but maybe it is due to redundant work being done e.g processing duplicate regions after redispatches, or the cache not being used, or something like this. Haven't exactly determined

@rbuels rbuels added this to the 1.15.3 milestone Aug 29, 2018

@rbuels

This comment has been minimized.

Copy link
Collaborator

rbuels commented Aug 29, 2018

I'll look into this, since I'm kind of redoing the tabix code now.

@garrettjstevens

This comment has been minimized.

Copy link
Contributor

garrettjstevens commented Sep 16, 2018

A bit of an update here. On the current dev branch, it takes ~75 seconds on my machine for one region of this data:
before
And on the 1147_npm_tabix_fixtests branch it only takes ~32 seconds:
after

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.