Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] Reading files with mmap for compression level "no compression" #320

Open
vadimkantorov opened this issue Aug 15, 2024 · 0 comments

Comments

@vadimkantorov
Copy link

vadimkantorov commented Aug 15, 2024

Hi!

Am I right that for zip files with no compression, reading can be done simply via mmap after calculating offset and byte count?

Can miniz help in reading from such ZIPs: i.e. parsing the ZIP file tree and providing a requested file offset and byte size?

There is the mz_zip_archive_file_stat structure

miniz/miniz_zip.h

Lines 22 to 77 in 8714fd3

typedef struct
{
/* Central directory file index. */
mz_uint32 m_file_index;
/* Byte offset of this entry in the archive's central directory. Note we currently only support up to UINT_MAX or less bytes in the central dir. */
mz_uint64 m_central_dir_ofs;
/* These fields are copied directly from the zip's central dir. */
mz_uint16 m_version_made_by;
mz_uint16 m_version_needed;
mz_uint16 m_bit_flag;
mz_uint16 m_method;
/* CRC-32 of uncompressed data. */
mz_uint32 m_crc32;
/* File's compressed size. */
mz_uint64 m_comp_size;
/* File's uncompressed size. Note, I've seen some old archives where directory entries had 512 bytes for their uncompressed sizes, but when you try to unpack them you actually get 0 bytes. */
mz_uint64 m_uncomp_size;
/* Zip internal and external file attributes. */
mz_uint16 m_internal_attr;
mz_uint32 m_external_attr;
/* Entry's local header file offset in bytes. */
mz_uint64 m_local_header_ofs;
/* Size of comment in bytes. */
mz_uint32 m_comment_size;
/* MZ_TRUE if the entry appears to be a directory. */
mz_bool m_is_directory;
/* MZ_TRUE if the entry uses encryption/strong encryption (which miniz_zip doesn't support) */
mz_bool m_is_encrypted;
/* MZ_TRUE if the file is not encrypted, a patch file, and if it uses a compression method we support. */
mz_bool m_is_supported;
/* Filename. If string ends in '/' it's a subdirectory entry. */
/* Guaranteed to be zero terminated, may be truncated to fit. */
char m_filename[MZ_ZIP_MAX_ARCHIVE_FILENAME_SIZE];
/* Comment field. */
/* Guaranteed to be zero terminated, may be truncated to fit. */
char m_comment[MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE];
#ifdef MINIZ_NO_TIME
MZ_TIME_T m_padding;
#else
MZ_TIME_T m_time;
#endif
} mz_zip_archive_file_stat;
, but I need help figuring out how to get a byte offset from the ZIP file beginning from an instance of this structure (and if it's possible currently).

Or maybe this other mz_zip_reader_extract_iter_state structure could be used for this?

miniz/miniz_zip.h

Lines 186 to 208 in 8714fd3

typedef struct
{
mz_zip_archive *pZip;
mz_uint flags;
int status;
mz_uint64 read_buf_size, read_buf_ofs, read_buf_avail, comp_remaining, out_buf_ofs, cur_file_ofs;
mz_zip_archive_file_stat file_stat;
void *pRead_buf;
void *pWrite_buf;
size_t out_blk_remain;
tinfl_decompressor inflator;
#ifdef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS
mz_uint padding;
#else
mz_uint file_crc32;
#endif
} mz_zip_reader_extract_iter_state;

Or is compression level "zero" not supported by miniz decompression routines? I unfortunately can't understand where in code the MZ_NO_COMPRESSION decompression case is treated :(

If it's not possible, adding such a field for "no compression" files would be nice for mmap-based file readers.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant