Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support plaintext fasta on desktop by dynamically creating a FAI file on the fly #2443

Merged
merged 10 commits into from
Nov 2, 2021

Conversation

cmdcolin
Copy link
Collaborator

@cmdcolin cmdcolin commented Oct 23, 2021

Fixes #2428

Allows streaming a large FASTA file so the operation remains out of memory, creates a FAI file on the fly, stores it in userData/fai/filename+Date.now()+.fai and then dynamically creates an IndexedFastaAdapter config from this

Helps aid memory usage so that we don't have to load the whole genome into memory

Uses some of the concepts learned about streaming files from text-index in the process

@github-actions github-actions bot added the needs label triage Needs a label to show in changelog (breaking, enhancement, bug, documentation, or internal) label Oct 23, 2021
@cmdcolin cmdcolin added enhancement New feature or request and removed needs label triage Needs a label to show in changelog (breaking, enhancement, bug, documentation, or internal) labels Oct 25, 2021
@cmdcolin cmdcolin force-pushed the support_plaintext_fasta branch 2 times, most recently from c90e963 to 9b96f76 Compare October 27, 2021 14:59
@codecov
Copy link

codecov bot commented Oct 27, 2021

Codecov Report

Merging #2443 (d99b67f) into main (79e5fb1) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

❗ Current head d99b67f differs from pull request most recent head 61c48bd. Consider uploading reports for the commit 61c48bd to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2443      +/-   ##
==========================================
- Coverage   61.16%   61.11%   -0.06%     
==========================================
  Files         540      540              
  Lines       25147    25168      +21     
  Branches     5873     5881       +8     
==========================================
  Hits        15381    15381              
- Misses       9445     9466      +21     
  Partials      321      321              
Impacted Files Coverage Δ
...roducts/jbrowse-desktop/src/OpenSequenceDialog.tsx 3.84% <0.00%> (-1.42%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79e5fb1...61c48bd. Read the comment docs.

@rbuels
Copy link
Contributor

rbuels commented Nov 1, 2021

have you tested the error handling for when the index generator fails?

@cmdcolin
Copy link
Collaborator Author

cmdcolin commented Nov 1, 2021

have you tested the error handling for when the index generator fails?

should be ok, it has a little error display if no sequences were found in the file for example (staying at the OpenSequenceDialog screen)

@cmdcolin
Copy link
Collaborator Author

cmdcolin commented Nov 1, 2021

had a limitation before that only allowed fasta with \n and not \r\n but now allows both

@cmdcolin cmdcolin merged commit 3c3c8e2 into main Nov 2, 2021
@cmdcolin cmdcolin deleted the support_plaintext_fasta branch November 2, 2021 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow opening plaintext FASTA in desktop
2 participants