Skip to content

v4.0.0

Choose a tag to compare

View all tags
@snonux snonux released this 12 Jul 10:48
· 294 commits to master since this release
v4.0.0
45b2e60

Release v4.0.0

Release Notes

Overview

DTail v4.0.0 represents a major evolution in distributed log aggregation and analysis. This release introduces a powerful serverless mode, advanced map-reduce capabilities, comprehensive color customization, and significant architectural improvements that enhance both performance and usability.

🎯 Major Features

Serverless Mode

  • Run DTail without remote servers: Process local log files directly without requiring SSH connections or remote dserver instances
  • Stdin pipe support: Seamlessly integrate with existing shell pipelines (e.g., cat logfile.log | dmap "query")
  • Perfect for single-server scenarios: Ideal when you already have shell access and want to analyze logs locally

Enhanced Map-Reduce Engine

  • Advanced query language: Support for complex aggregations, grouping, and filtering with improved syntax
  • Built-in functions: New functions like md5sum(), maskdigits(), and enhanced string operations
  • Field sensitivity controls: Better handling of case-sensitive field names and values
  • Improved data types: Enhanced support for numeric operations and string manipulations

Color and Display Customization

  • Fully customizable color schemes: Configure colors for different log levels, sources, and data types
  • Color table preview: Use --colorTable and --wideColorTable flags to preview available color combinations
  • Plain mode: New --plain flag for clean output without colors or formatting
  • ANSI color support: Complete ANSI color code support with configurable color tables

🚀 Improvements

Performance Enhancements

  • Optimized file reading: Byte-wise file processing with improved buffering for better performance
  • Reduced memory usage: More efficient memory management with pooled buffers
  • Faster shutdown: Improved client shutdown times, especially useful for batch operations
  • Connection throttling: Better handling of concurrent connections to prevent resource exhaustion

User Experience

  • Simplified command structure: Support for positional arguments as shorthand (e.g., dtail "query" logfile.log)
  • Better error handling: More descriptive error messages and improved debugging output
  • Enhanced documentation: Comprehensive guides for query language, log formats, and custom implementations
  • Integration test suite: Extensive testing framework ensuring reliability across different scenarios

Architecture & Code Quality

  • Modular design: Restructured codebase with clear separation between client/server components
  • Reduced cognitive complexity: Refactored large functions into smaller, more maintainable units
  • Improved logging: New structured logging system with configurable levels and outputs
  • Better configuration management: Enhanced config file handling with JSON schema validation

🔧 Bug Fixes

Stability Improvements

  • Fixed deadlocks: Resolved race conditions in data aggregation and concurrent file operations
  • Better signal handling: Improved handling of interrupt signals and graceful shutdowns
  • Memory leak fixes: Addressed various memory leaks in long-running operations
  • SSH connection reliability: Enhanced SSH connection stability and error recovery

Data Handling

  • Newline character handling: Proper processing of files with or without trailing newlines
  • Long line support: Fixed issues with very long log lines that previously caused truncation
  • Protocol improvements: Enhanced client-server communication protocol for better reliability

📋 Breaking Changes

Removed Features

  • Deprecated drun command: Removed background command execution to focus on log processing
  • Relaxed SSH auth mode: Removed insecure authentication option for better security
  • Legacy timeout options: Simplified timeout configuration

Configuration Changes

  • Updated JSON schema: Configuration files may need updates to match new schema
  • New default ports and paths: Some default values have changed for better security
  • Log format updates: Some built-in log formats have been enhanced with new fields

🔄 Migration Guide

For Existing Users

  1. Update configuration files: Validate your dtail.json against the new schema
  2. Replace drun usage: Migrate any background job scripts to use standard shell tools
  3. Review custom log formats: Check if custom log format implementations need updates for new features
  4. Update monitoring scripts: The health check functionality has moved to a separate dtailhealth binary

New Installation

📚 Documentation

This release includes extensive new documentation:

  • Query Language Guide: Complete EBNF grammar and function reference
  • Log Format Guide: How to implement and customize log format parsers
  • Serverless Mode Examples: Practical examples for local log analysis
  • Color Customization: Full guide to appearance customization
  • Testing Guide: How to run and contribute to the test suite

DTail v4.0.0 maintains backward compatibility for core functionality while introducing powerful new capabilities. The serverless mode opens up entirely new use cases, while the enhanced map-reduce engine provides deeper insights into your log data.

Assets 2
Loading