Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Append to file, empty contents of file #4

Closed
benpackard opened this issue Aug 20, 2017 · 4 comments
Closed

Append to file, empty contents of file #4

benpackard opened this issue Aug 20, 2017 · 4 comments

Comments

@benpackard
Copy link

Apologies if this is supported already - I have only reads the reader so far.

Is there a command for appending to a file? Or should we read the file first back into model objects, append objects, and rewrite?

Similarly, can we remove the contents of a file without removing the file itself? Or should we delete it and recreate it with an empty array for example?

@saoudrizwan
Copy link
Owner

saoudrizwan commented Aug 21, 2017

Is there a command for appending to a file? Or should we read the file first back into model objects, append objects, and rewrite?

No, that would require Disk to have the ability to modify existing files' data, which could lead to the possibility of corrupting the file and not being able to decode to an expected type. This level of modification to the file system should be within your own control, so yes you are right: you must first retrieve your desired objects and manipulate them however you want to that way, then save them again.

Similarly, can we remove the contents of a file without removing the file itself? Or should we delete it and recreate it with an empty array for example?

You should delete the file instead of 'emptying' it.

@benpackard
Copy link
Author

Thanks, makes sense. I was thinking of a use case where I might want to store objects to disk over time for processing later. Loading a large number to append one more might have a memory impact. But I get the point - what would stop me appending an apple to an array of oranges.

I think one solution I might consider is saving each object to a folder as a separate file. Each can be deleted once processed, and it is the consumers responsibility for saving the right kind of object (and handling an unexpected deserialzed type). I will have to research the impact of reading hundreds or thousands or small files though.

@saoudrizwan
Copy link
Owner

I was thinking of a use case where I might want to store objects to disk over time for processing later. Loading a large number to append one more might have a memory impact.

I've been on the fence about this too. There is a way to append data to a file instead of reading it, decoding it, encoding it, then writing again. But I wasn't sure if it would have been user friendly. I will try to code something up tonight and see what others have to say about it.

Thank you for your input!

@saoudrizwan
Copy link
Owner

Hey @benpackard, I added the ability to append in Disk thanks to your suggestion (I put your name in the README.) Thank you!

Please play around with it and let me know what you think!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants