Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing file initialized with LZ4 package causes I/O error #8450

Open
Beuc opened this issue Apr 16, 2019 · 4 comments

Comments

Projects
None yet
2 participants
@Beuc
Copy link
Contributor

commented Apr 16, 2019

It is possible to replace a file in the virtual filesystem when it was initialized using file_packager.py scripts.

However, when using its --lz4 option, replacing a file causes error:

# existing file:
> FS.writeFile('/renpy/common/DejaVuSans.txt', 'a')
Error: I/O error
# new file:
FS.writeFile('/renpy/common/new.txt', 'a')
undefined
@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Apr 16, 2019

Browsing the sources, I see that LZ4 files are kept compressed (I thought they were decompressed once and for all when the file package is extracted) and then made read-only.
Would there be a way to convert LZ4 files to regular files when they are written over?
(I noticed I could unlink them prior to rewritting them, but that's not something my unzip library does.)

@kripken

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

I think the idea with our LZ4 support is that you can have a huge amount of data in them, all kept compressed until you actually read it. So if you just read a small part, or read it incrementally, you can save a lot of memory.

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Apr 18, 2019

Yes that's what I understand too, but the read-only implementation breaks my use case where I have several layered filesets that may overwrite each others.

Being able to open(x, "w") LZ4 files for complete overwrite (without unlinking them first, with or without compression) would be nice.
Not high-priority though.

(In addition, maybe we can change the "I/O error" with a clearer read-only error or something.)

@kripken

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

Agree on both that adding an option to write would be interesting, and a nicer error meanwhile would be good too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.