Bit7z v4.1.0 Beta

[rikyoz]

Main Features and Improvements

• Nested archives support: the new BitNestedArchiveReader class lets you open and inspect archives that are embedded as items inside another archive, without extracting them to disk first. (#92)

  // Extract the .tar inside a .tar.gz without writing it to disk
  Bit7zLibrary lib{ "7z.dll" };
  BitArchiveReader gz{ lib, "archive.tar.gz", BitFormat::GZip };
  BitNestedArchiveReader tar{ lib, gz, BitFormat::Tar };
  tar.extractTo( "output/" );

• Native file I/O: file streams have been rewritten to use low-level Win32 APIs on Windows and POSIX I/O on Unix, replacing std::fstream. This dramatically improves the performance of both extraction and compression operations. (#319, #325)

  Due to the choice of using std::fstream for cross-platform support, v4.0 introduced a severe performance regression.

  For a quick comparison, I did some non-exhaustive benchmarks. The result is the following boxplot, which showcases the scale of that regression and how v4.1 resolves it. In this test, v4.0 extraction takes roughly six times as long as v3, while v4.1 brings it back down into the same range as v3:

  

  Zooming in on just v3 and v4.1 (the v4.0 box is omitted here), a small performance gap remains:

  

  Part of this residual gap is likely due to v4.1 performing additional security validation that v3 did not, so I anticipate a modest, permanent cost relative to v3 in exchange for safer behavior. I'll continue working to reduce, and where possible eliminate, the non-essential portion of this gap in the stable release.

  Benchmark methodology

  Each version was benchmarked by repeatedly extracting the same archive to the same destination, discarding warmup iterations. Boxes show the median (line) and interquartile range; the cross marks the mean; whiskers follow the Tukey 1.5×IQR convention. Measurements were collected on a single machine; absolute timings are environment-dependent and intended for relative comparison between versions, not as absolute throughput figures. The v4.0 regression and its resolution in v4.1 are large effects that are robust to measurement noise; the finer v3-vs-v4.1 gap is smaller and more sensitive to measurement conditions, so it should be read as indicative rather than exact.

• New extraction callbacks: RenameCallback (dynamically rename or skip items during extraction), RawDataCallback (stream raw bytes directly to user code without touching the filesystem), and BufferCallback (route each extracted file to a separate, independently chosen buffer) unlock extraction patterns that were previously impossible.

  Example

  BitFileExtractor extractor{ lib, BitFormat::Zip };
  
  // RenameCallback: skip .tmp files, extract everything else under a subfolder
  extractor.extract( "archive.zip", "output/", []( uint32_t, const tstring& path ) -> tstring {
      if ( path.find( ".tmp" ) != tstring::npos ) {
          return {}; // empty string = skip this item
      }
      return "backup/" + path;
  } );
  
  // RawDataCallback: receive a file's raw bytes without writing to disk
  std::vector<byte_t> rawData;
  extractor.extractTo( "archive.zip", [&]( const byte_t* data, std::size_t size ) -> bool {
      rawData.insert( rawData.end(), data, data + size );
      return true; // return false to abort
  }, /* index = */ 0 );
  
  // BufferCallback: extract all files, each into its own buffer
  std::map<tstring, buffer_t> files;
  extractor.extract( "archive.zip", [&]( uint32_t, const tstring& path ) -> buffer_t& {
      return files[ path ];
  } );

• Multi-volume LRU file-handle cache: when reading or creating large multi-volume archives, bit7z now keeps only the most recently used volume file handles open, preventing file descriptor exhaustion on archives with many volumes (#150, #161).

• Timestamp control: two complementary additions give full control over timestamps when creating archives. BitAbstractArchiveCreator gains setStoreLastWriteTime(), setStoreCreationTime(), and setStoreLastAccessTime() to control which timestamp types are stored globally (format support varies: 7z has the most complete support; TAR does not support creation/last-access timestamps). Additionally, BitOutputArchive::addFile() now returns a BitInputItem&, allowing per-item timestamp overrides via setCreationTime(), setLastWriteTime(), and setLastAccessTime(). (#184)

  Example

  // Global control: choose which timestamp types to embed in the archive
  BitFileCompressor compressor{ lib, BitFormat::SevenZip };
  compressor.setStoreCreationTime( true );    // opt in to creation timestamps
  compressor.setStoreLastAccessTime( true );  // opt in to last access timestamps
  compressor.setStoreLastWriteTime( false );  // suppress last write timestamps
  compressor.compress( { "file1.txt", "file2.txt" }, "archive.7z" );
  
  // Per-item control: override the timestamp of a specific item
  const auto customTime = std::chrono::system_clock::now();
  BitArchiveWriter writer{ lib, BitFormat::SevenZip };
  writer.addFile( "document.txt" ).setLastWriteTime( customTime );
  writer.compressTo( "archive.7z" );

• Deferred library loading: Bit7zLibraryLoader allows constructing a loader without immediately loading the 7-zip DLL/SO, then loading (and unloading) it at any later point. Useful for plugin systems and applications that need to control when the native library is brought in.

  Example

  Bit7zLibraryLoader loader; // no library loaded yet
  
  // ... later, e.g. after the user selects the library path ...
  std::error_code ec;
  loader.load( "7z.dll", ec );
  if ( ec ) {
      std::cerr << "Could not load 7-zip: " << ec.message() << "\n";
      return;
  }
  
  BitFileExtractor extractor{ loader, BitFormat::Zip }; // implicit conversion to Bit7zLibrary&
  extractor.extractTo( "archive.zip", "output/" );
  loader.unload();

New Features

• Bit7zLibrary::useLargePages(): replaces the deprecated setLargePageMode().
• BitArchiveItem::rawPath() / nativeName(): access item paths and names in their raw/native string forms.
• BitArchiveReader: nested archive constructors: open nested archives directly from a parent BitInputArchive.
• BitArchiveReader: ArchiveStartOffset constructors: open archives embedded in the middle of a file (e.g., self-extracting archives).
• BitArchiveReader::archiveProperties(): access archive-level format properties.
• BitArchiveReader::itemsMatching(): get items matching a wildcard pattern.
• BitError::NoMatchingFile: new error code for filter/regex extraction finding no matching item.
• BitExtractor::extractFolder(): extract a specific folder from an archive.
• BitExtractor::extract() with RenameCallback: rename items on the fly during extraction.
• BitFileCompressor::compress(vector<pair<path, alias>>): compress files using explicit in-archive path aliases. (#313)
• BitIndicesView: lightweight, non-owning span of item indices; implicitly constructible from a single index, vector, array, or initializer list.
• BitInputArchive::extractFolderTo(): extract a single folder from an archive.
• sevenzip_string type alias and to_native_string() conversion functions.

Improvements

• Compression indexing performance: BitItemsVector now stores items by value instead of via std::unique_ptr, eliminating one heap allocation per file when indexing items for compression. The improvement scales with the number of files being compressed.
• Filesystem performance: besides the usage of native I/O APIs, also the number of unnecessary accesses to the filesystem was reduced, which in some cases was causing performance degradation due to realtime inspections by antiviruses like Windows Defender (#320).
• Atomic in-place updates: archive updates now write to a temporary file and rename atomically on success.
• Error reporting: new OpenError category for richer archive-opening failure messages.
• MinGW: added Win32Category error category for correct std::error_code handling.
• String conversion: improved UTF-8 <-> UTF-16 correctness; to_tstring() is zero-copy when already a tstring. (#276)
• Performance: callback getters, password(), and itemProperties() return const references; file path retrieval in extract callbacks is skipped when no FileCallback is set; regex extraction accepts pre-compiled tregex objects.
• Default 7-Zip version: updated to 26.01.

Bug Fixes

• Fixed build on non-Windows platforms with BIT7Z_USE_NATIVE_STRING=ON.
• Fixed build with the C++20 standard.

Deprecated

• Bit7zLibrary::setLargePageMode() -> use useLargePages().

Planned Before Stable Release

• Performance optimizations to further reduce/close the remaining gap with 7-Zip and bit7z v3.
• Other small features that I missed to implement in the beta.
• Improved test coverage.
• Code cleanup.

Note

This release includes all improvements, patches and fixes introduced in the v4.0.x series up to and including v4.0.12.

---

Full Changelog: v4.0.12...v4.1.0-beta

---

This discussion was created from the release Bit7z v4.1.0 Beta.