Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there any way to preview files larger than 2GB #57

Closed
LogWell opened this issue Jul 16, 2023 · 1 comment
Closed

Is there any way to preview files larger than 2GB #57

LogWell opened this issue Jul 16, 2023 · 1 comment

Comments

@LogWell
Copy link

LogWell commented Jul 16, 2023

My data is stored on the server and the file size is greater than 2GB, downloading and previewing locally can be time-consuming, is there a recommended preview method?

@Carnageous
Copy link

If you know the structure of your HDF5 file, you can stream individual datasets in chunks to a client, for example using https://github.com/bmaranville/lazyFileLRU (which uses an LRU cache to store the chunks). You can actually preview this functionality by putting your file URL into this example and loading datasets: https://bmaranville.github.io/lazyFileLRU/

This way, you can specify a specific dataset you want to look at and only download this (plus a bit of overhead, just because of the way HDF5 files work under the hood)

@LogWell LogWell closed this as completed Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants