Skip to content

Zip file streaming#18

Merged
driveton merged 5 commits intomainfrom
zip-file-streaming
Feb 25, 2026
Merged

Zip file streaming#18
driveton merged 5 commits intomainfrom
zip-file-streaming

Conversation

@driveton
Copy link
Copy Markdown
Owner

Summary

Stream ZIP and EPUB archives to disk instead of memory

  • Replaces the StringIO-backed ZipFile::Writer with a disk-backed Tempfile writer. ZipKit::Streamer now writes directly to the Tempfile, so large archives are never held fully in memory.
  • EpubArchiver#generate follows the same pattern. Archiver#write_epub pipes the EPUB Tempfile into the ZIP entry via IO.copy_stream with compress: false. EPUBs are already ZIP-compressed, so double-compressing is wasteful.

Also registers IO as an inflection acronym so Zeitwerk resolves zip_file/reader/io.rb to ZipFile::Reader::IO correctly during eager load.

driveton added 5 commits February 24, 2026 17:09
Writer now accepts an external IO (Tempfile) rather than managing its
own buffer. ZipKit::Streamer writes directly to that IO, avoiding
loading the whole archive in memory. Reader gains a block form that
yields a ZipFile::Reader::IO, a streaming IO wrapper around a single
zip entry backed by ZipKit's extractor.
EpubArchiver#generate now writes to a Tempfile instead of StringIO,
so large EPUBs are never held fully in memory. Archiver#write_epub
streams the Tempfile into the ZIP entry via IO.copy_stream with
compress: false — EPUBs are already ZIP-compressed, so double-
compressing is wasteful and slow.
Register IO as an inflection acronym so Zeitwerk resolves
reader/io.rb to ZipFile::Reader::IO instead of ZipFile::Reader::Io.

Update Book::ArchiverTest to pass a ZipFile::Writer to generate and
read back from the resulting Tempfile. Update EpubArchiverTest to
expect a Tempfile from generate instead of StringIO.
ZipFile::Writer is now an IO-like proxy: write() forwards to the
underlying IO while tracking byte_size and building an MD5 digest,
and the streamer is wired through self so tracking is accurate.
stream_to() and checksum() are restored.

Book::Exportable#export uses the Writer+Tempfile pattern to match
the updated Archiver#generate signature.
@driveton driveton merged commit 4296190 into main Feb 25, 2026
4 checks passed
@driveton driveton deleted the zip-file-streaming branch February 25, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant