🚀 ztail v1.1.3 — Memory Efficiency & Performance Boost
This release delivers a deep refactor of memory usage and data handling strategies, removing critical bottlenecks while maintaining full compatibility.
🔧 Key Improvements
-
Fewer copies and allocations
The parser (parser.cpp) now consumes data in place, avoiding short-lived string allocations and reducing heap pressure. -
CharRingBuffer as default backend
Replacedstd::vector<std::string>withCharRingBuffer(char_ring_buffer.cpp) as the default line buffer, eliminating fragmentation and multiple allocations per line. -
Compact offsets
Offsets are now stored using 32-bit integers when possible, cutting memory usage per line in half on 64-bit architectures. -
Block-based output
Printing no longer builds a massivestd::stringin memory (circular_buffer.cpp,char_ring_buffer.cpp). Output is streamed in blocks, reducing RAM peaks on large outputs. -
Optimized plain file reading
tailPlainFileno longer accumulates all chunks in memory. It processes in reverse flow, ensuring memory stays constant regardless of file size. -
Explicit ring buffer sizing
Circular buffer allocation is now based on a byte budget (capacity * lineCapacityrefactored), making memory usage predictable and avoiding waste. -
Dynamic read buffer tuning
Multithreaded mode no longer doubles buffer reservations unconditionally (main.cpp). Buffers are now auto-tuned, with the option to disable threads, balancing speed vs. memory. -
Exposed compressor parameters
Compressors (zlib, bzip2, xz, zstd, zip) now expose fine-tuning options such as--zlib-buffer(cli.cpp), letting users adjust speed vs. RAM trade-offs. -
Zero-copy line handling
Eliminated redundant copies — lines are now placed directly into the ring buffer without reconstruction. -
Portable build profiles
Builds no longer force-march=native. Added tuned profiles to keep portability across machines without losing optimizations.
📈 Expected Impact
- Lower memory usage: compact offsets, zero-copy paths, block printing.
- Higher performance: reduced allocations, cache-friendly buffers, fewer syscalls.
- More user control: CLI options to fine-tune compression and buffer parameters.
👉 ztail v1.1.3 is a major step toward making the tool a high-performance and memory-efficient solution for inspecting large compressed files in modern data pipelines.