v1.1.3 — Memory Efficiency & Performance Boost

🚀 ztail v1.1.3 — Memory Efficiency & Performance Boost

This release delivers a deep refactor of memory usage and data handling strategies, removing critical bottlenecks while maintaining full compatibility.

---

🔧 Key Improvements

1. Fewer copies and allocations
   The parser (parser.cpp) now consumes data in place, avoiding short-lived string allocations and reducing heap pressure.

2. CharRingBuffer as default backend
   Replaced std::vector<std::string> with CharRingBuffer (char_ring_buffer.cpp) as the default line buffer, eliminating fragmentation and multiple allocations per line.

3. Compact offsets
   Offsets are now stored using 32-bit integers when possible, cutting memory usage per line in half on 64-bit architectures.

4. Block-based output
   Printing no longer builds a massive std::string in memory (circular_buffer.cpp, char_ring_buffer.cpp). Output is streamed in blocks, reducing RAM peaks on large outputs.

5. Optimized plain file reading
   tailPlainFile no longer accumulates all chunks in memory. It processes in reverse flow, ensuring memory stays constant regardless of file size.

6. Explicit ring buffer sizing
   Circular buffer allocation is now based on a byte budget (capacity * lineCapacity refactored), making memory usage predictable and avoiding waste.

7. Dynamic read buffer tuning
   Multithreaded mode no longer doubles buffer reservations unconditionally (main.cpp). Buffers are now auto-tuned, with the option to disable threads, balancing speed vs. memory.

8. Exposed compressor parameters
   Compressors (zlib, bzip2, xz, zstd, zip) now expose fine-tuning options such as --zlib-buffer (cli.cpp), letting users adjust speed vs. RAM trade-offs.

9. Zero-copy line handling
   Eliminated redundant copies — lines are now placed directly into the ring buffer without reconstruction.

10. Portable build profiles
    Builds no longer force -march=native. Added tuned profiles to keep portability across machines without losing optimizations.

---

📈 Expected Impact

• Lower memory usage: compact offsets, zero-copy paths, block printing.
• Higher performance: reduced allocations, cache-friendly buffers, fewer syscalls.
• More user control: CLI options to fine-tune compression and buffer parameters.

---

👉 ztail v1.1.3 is a major step toward making the tool a high-performance and memory-efficient solution for inspecting large compressed files in modern data pipelines.

---

v1.1.1 — Performance & Throughput Improvements

v1.1.1 — Performance & Throughput Improvements**

Overview

Building on the expanded format support of v1.1.0, this release focuses on boosting speed, reducing latency, and enhancing scalability through multiple performance optimizations.

---

What’s New in This Release

• Configurable Read Buffer (-r/--read-buffer)
  Allows tuning of the read buffer size (previously fixed at 1 MiB), enabling optimized I/O for different environments — especially useful for high-latency or low-bandwidth setups.
  ([GitHub][1])

• Larger zlib Internal Buffer
  Increased internal buffer size for .gz/​.bgz decompression to reduce the number of syscalls and improve throughput.

• Disable Sync Between iostream and stdio
  Performance boost for output operations by turning off unnecessary synchronization when using std::cout.

• Batched Output Writing
  Now grouping lines into a single write call instead of line-by-line printing significantly reduces overhead.

• True Ring Buffer Implementation
  The previous CharRingBuffer (which shifted data on each erase) has been replaced with a proper circular buffer, eliminating O(n) overhead and improving memory handling.

• Double Buffering with Threads
  Enables parallel decompression and parsing: while one buffer is being read, another is being processed, greatly reducing latency and increasing overall throughput.

---

Why It Matters

• Faster tail operations on large or remote files — especially critical with slow I/O (e.g., network shares or cloud storage).
• Lower CPU and syscalls overhead → plots smaller, faster, and more resource-efficient processes.
• Adaptable to various use cases — users can fine-tune buffer sizes to match machines with different memory and I/O profiles.
• Smoother CLI experience — fewer delays, more reliable performance under load.

---

Usage Example

# Tail 20 lines of a large compressed file with a custom read buffer (e.g., 4 MiB):
./ztail -n 20 -r 4194304 hugefile.zst

---

Reiteration of Formats & Usability from v1.1.0

• Supports .gz, .bgz, .bz2, .xz, .zip, and .zst
• Automatic format detection via magic bytes
• Unified CLI options across all formats
• Robust, portable builds with unit tests and BUILD_PORTABLE CMake option
  ([GitHub][2], [GitHub][1])

---

Build & Requirements

git clone https://github.com/acpguedes/ztail.git
cd ztail
mkdir build && cd build
cmake .. -DBUILD_TESTING=ON -DZTAIL_USE_THREADS=ON
make
ctest --output-on-failure  # optional

Optional tuning flags:

• -DZTAIL_USE_THREADS=OFF — disable threaded IO if unsupported
• -DBUILD_PORTABLE=ON — for cross-platform builds
  ([GitHub][1])

---

Quick Summary (for GitHub UI)

## v1.1.1 — Performance & Throughput Improvements

### Added / Improved
- Configurable read buffer (`-r/--read-buffer`) for optimized I/O.
- Larger zlib buffer to reduce syscalls during decompression.
- Disabled `iostream` ↔ `stdio` sync for faster `std::cout`.
- Batched output to minimize write overhead.
- Replaced `CharRingBuffer` with efficient circular buffer.
- Introduced double buffering with threads for overlapping I/O and parsing.

### Still Supports (from v1.1.0)
- `.gz`, `.bgz`, `.bz2`, `.xz`, `.zip`, `.zst` formats.
- Auto-detect compression format.
- Unified CLI flags, portable CMake build, and unit tests.

### Build Example
```bash
cmake .. -DBUILD_TESTING=ON -DZTAIL_USE_THREADS=ON
make

---

v1.1.0 — Fast Tail for Even More Compressed Formats

v1.1.0 — Fast Tail for Even More Compressed Formats

Summary

This release expands ztail’s capabilities to handle a broader range of compressed formats, improves CLI usability, and enhances portability.

---

What’s New

Added

• Expanded format support: In addition to .gz, .bgz, and .zip, ztail now supports .bz2, .xz, and .zst compressed files.
• Automatic format detection: Determines the compression format from file content (magic bytes), even if the file extension is missing.

Improved

• Unified CLI options: Arguments like -n/--lines, -c/--line-capacity, and -e/--entry now work consistently across all supported formats.

• Robustness & portability:

  • Added automated unit tests in the tests/ directory.
  • Improved CMake build system with a BUILD_PORTABLE option for easier compilation in different environments.

---

Usage Examples

# Tail the last 10 lines of any supported compressed file
./ztail -n 10 file.xz

# Tail a specific entry inside a ZIP archive
./ztail -n 20 -e my_file.txt archive.zip

# Tail last 5 lines with larger buffer for long lines
./ztail -n 5 -c 2048 file.bz2

---

Requirements

• GNU C++17 or newer, CMake ≥ 3.10

• Libraries:

  • zlib for .gz/.bgz
  • bzip2 for .bz2
  • liblzma for .xz
  • libzstd for .zst
  • libzip for .zip

---

Build Instructions

git clone https://github.com/acpguedes/ztail.git
cd ztail
mkdir build && cd build
cmake .. -DBUILD_TESTING=ON
make
ctest --output-on-failure  # Optional: run tests

---

v1.0.0 - Fast Decompression Tail for Compressed Files

First public release of ztail — a GNU-like command-line tool designed for fast access to the last lines of compressed files.

Features

• Supports .gz, .bgz, and .zip compressed files.
• Reads only the necessary portion of the file to retrieve the last N lines — avoiding full decompression.
• Performance up to 10x faster than zcat | tail in tests with large files.
• Simple CLI interface, compatible with typical tail usage.
• Uses efficient circular buffer implementation for memory management.

Usage example

ztail large_file.gz -n 20

Build

mkdir build && cd build
cmake ..
make

Notes

• Requires zlib and libzip installed on the system.
• Tested on Linux (GCC 13.3.0).