Skip to content

v1.1.3 — Memory Efficiency & Performance Boost

Latest

Choose a tag to compare

@acpguedes acpguedes released this 22 Aug 15:00
· 10 commits to main since this release
a9bd25e

🚀 ztail v1.1.3 — Memory Efficiency & Performance Boost

This release delivers a deep refactor of memory usage and data handling strategies, removing critical bottlenecks while maintaining full compatibility.


🔧 Key Improvements

  1. Fewer copies and allocations
    The parser (parser.cpp) now consumes data in place, avoiding short-lived string allocations and reducing heap pressure.

  2. CharRingBuffer as default backend
    Replaced std::vector<std::string> with CharRingBuffer (char_ring_buffer.cpp) as the default line buffer, eliminating fragmentation and multiple allocations per line.

  3. Compact offsets
    Offsets are now stored using 32-bit integers when possible, cutting memory usage per line in half on 64-bit architectures.

  4. Block-based output
    Printing no longer builds a massive std::string in memory (circular_buffer.cpp, char_ring_buffer.cpp). Output is streamed in blocks, reducing RAM peaks on large outputs.

  5. Optimized plain file reading
    tailPlainFile no longer accumulates all chunks in memory. It processes in reverse flow, ensuring memory stays constant regardless of file size.

  6. Explicit ring buffer sizing
    Circular buffer allocation is now based on a byte budget (capacity * lineCapacity refactored), making memory usage predictable and avoiding waste.

  7. Dynamic read buffer tuning
    Multithreaded mode no longer doubles buffer reservations unconditionally (main.cpp). Buffers are now auto-tuned, with the option to disable threads, balancing speed vs. memory.

  8. Exposed compressor parameters
    Compressors (zlib, bzip2, xz, zstd, zip) now expose fine-tuning options such as --zlib-buffer (cli.cpp), letting users adjust speed vs. RAM trade-offs.

  9. Zero-copy line handling
    Eliminated redundant copies — lines are now placed directly into the ring buffer without reconstruction.

  10. Portable build profiles
    Builds no longer force -march=native. Added tuned profiles to keep portability across machines without losing optimizations.


📈 Expected Impact

  • Lower memory usage: compact offsets, zero-copy paths, block printing.
  • Higher performance: reduced allocations, cache-friendly buffers, fewer syscalls.
  • More user control: CLI options to fine-tune compression and buffer parameters.

👉 ztail v1.1.3 is a major step toward making the tool a high-performance and memory-efficient solution for inspecting large compressed files in modern data pipelines.