Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate assembly loading timeouts #195

Open
garrettjstevens opened this issue Mar 10, 2023 · 3 comments
Open

Investigate assembly loading timeouts #195

garrettjstevens opened this issue Mar 10, 2023 · 3 comments
Assignees
Milestone

Comments

@garrettjstevens
Copy link
Contributor

Currently in app.module.ts when registering the Mongoose module, we set a large maxTimeMS, since that seems to fix some issues people have had loading large data sets. We want to avoid this if we can, though. It's possible that the timeouts are coming from loading the chunks of the reference sequences. We need to:

  • Test loading some large assemblies with and without annotations and without setting maxTimeMS to see if we can determine where the timeouts are happening.
  • See if reducing the default chunk size (set in refSeq.schema.ts) makes it so those can be loaded without increasing maxTimeMS
@garrettjstevens garrettjstevens added this to the May 2023 milestone Mar 10, 2023
@kyostiebi kyostiebi added this to To Do in Apollo team board via automation Mar 10, 2023
@kyostiebi kyostiebi moved this from To Do to In Progress in Apollo team board Mar 10, 2023
@cmdcolin
Copy link

I don't recall if there were particular reasons for having the sequence in the database, but if not, could be worth considering storing externally. I believe I made a comment about this a long while back, but it could be worth re-considering potentially(?)

@garrettjstevens
Copy link
Contributor Author

As part of the requirements for bringing Artemis features to Apollo 3, the sequence has to be editable, which is why we're storing it in the database. There could potentially be a mode for those who don't care about sequence editing, though, that uses an external sequence to simplify things.

@cmdcolin
Copy link

cmdcolin commented Mar 10, 2023

It may not necessarily be the case that applying a sequence change requires storing in the database. WebApollo/Apollo2 had the concept of sequence changes also, stored just as patches in a database table.

I will also note, editing the sequence can be a very challenging feature to deliver correctly, and I would seriously consider what things can be done to try to simplify it or even define the problem away

I just mention the above because so much stuff is based on genome coordinates, and sequence changes will invalidate that, and if maintained separately in the WebApollo/Apollo2 way of things, it requires some hard to reason about coordinate math that to me seems almost impossible to get right

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

3 participants