-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Should create a ISMBFileStore streaming file read method. #14
Comments
You are missing multiple important points:
I suggest you look for the real reason you are getting "out of memory". |
for a simple use case, try to load a 10G file with a system with 6G available ram. you cannot load it all into a byte array, the program will crash with out of memory exception. if you store an entire file into streaming on the other hand will not at any point store the entire file into the ram instead streams it to the consumer of the data so they can access it incrementally. |
Of course you can't (and shouldn't) load an entire file into RAM. You also shouldn't load an entire file into a MemoryStream. In any case, I am not going to change the API. if you feel that the current API does not serve your special needs, feel free to fork the project and customize it to your needs. |
ah that makes sense. i didn't look closely enough to see that you don't load the whole file at once. That was the piece I was missing. |
By only supporting a read to byte[], causes out of memory situations on large files.
we should provide a
ReadStream
method that provides the stream to the data, or aWriteToOutputStream
method.The text was updated successfully, but these errors were encountered: