Raven MDA v0.4.0

[Aravinda-HWK]

Raven v0.4.0 Release Notes

Release Date: January 16, 2026

We're excited to announce Raven v0.4.0, a major release focused on storage optimization, enhanced testing infrastructure, and improved reliability. This release introduces advanced blob deduplication capabilities that significantly reduce storage requirements for email attachments.

---

🚀 Major Features

Global Blob Deduplication System

• Content-Aware Deduplication: Implemented intelligent blob deduplication that works across different encodings (base64, quoted-printable, 7bit)
• Shared Blob Storage: Moved blob table to shared.db for global deduplication across all users (feat: move blob table to shared.db for global deduplication #198)
• Encoding-Agnostic Hashing: Same attachment sent and received with different encoding formats is now stored only once
• Reference Counting: Automatic tracking of blob references to safely manage shared attachments
• Storage Savings: Eliminates duplicate storage when the same attachment appears in multiple emails or with different line-break patterns

Key Benefits:

• Reduces storage requirements by up to 50% for environments with many duplicate attachments
• Handles email transmission variations (line breaks at 76 chars per RFC 2045)
• Maintains data integrity with proper reference counting
• Works seamlessly with sent and received emails

Enhanced Database Architecture

• Improved Database Management: Enhanced DBManager for better connection pooling and caching
• Foreign Key Support: Enabled foreign key constraints for data integrity
• Per-User and Shared Databases: Optimized separation between user-specific and shared data

---

🧪 Testing & Quality Improvements

Comprehensive Test Coverage

• Blob Deduplication Tests: Added extensive test suite covering:
  • Base64 encoding with different line breaks
  • Raw vs. encoded content comparison
  • Sent and received attachment scenarios
  • Quoted-printable encoding support
  • Backward compatibility verification
• End-to-End Testing: Implemented E2E test infrastructure (180 task implement e2e testing #183)
  • Concurrency testing
  • IMAP authentication flows
  • LMTP delivery scenarios
  • Mailbox state persistence
• Integration Tests: Added comprehensive integration tests for:
  • Database operations (173 task test database operations with real sqliteintegration test #174)
  • IMAP server functionality (175 task implement integration test for imap server #176)
  • SASL authentication (test: implement integration tests for SASL authentication flow #179)
  • LMTP delivery service (Lmtp integration test #177)
• Test Fixtures: Organized test data with sample emails, configurations, and user data

Code Quality Enhancements

• Error Handling: Improved database closure error handling across tests
• Configuration Management: Enhanced YAML configuration support
• Code Organization: Better separation of concerns with dedicated test helpers

---

🔧 Technical Improvements

SASL Service Enhancement

• TCP Connection Support: Enhanced SASL service to support both Unix socket and TCP connections (task: enhance SASL service to support TCP connections and update rela… #194)
• Improved Authentication Flow: Better handling of authentication state transitions

Configuration & Deployment

• Configuration Cleanup: Removed unnecessary debug logs and cleaned up config handling (191 bug fix configuration issues and remove unnecessary debug logs #192)
• YAML Configuration: Standardized configuration file management
• Docker Updates: Improved Dockerfile for better build efficiency

Database Operations

• Password Reset Logic: Added user initialization checks and password reset functionality (task: add password reset logic and user initialization checks #185)
• Enhanced Queries: Optimized database queries for better performance
• Connection Management: Improved database connection lifecycle management

---

🐛 Bug Fixes

• [BUG] Same attachment is sent AND received is stored TWICE #196: Fixed critical bug where same attachment sent and received was stored twice
  • Root cause: Different encoding formats (line breaks) created different hashes
  • Solution: Normalize content before hashing for deduplication
• Configuration Issues: Resolved various configuration loading and validation problems
• Database Closure: Fixed improper database closure error handling in tests
• Encoding Handling: Improved handling of various content transfer encodings

---

📚 Documentation Updates

• README Improvements: Enhanced documentation for better user experience (task: update README.md for improved user experience and clarity #190)
• Test Documentation: Added comprehensive testing documentation (test/README.md)
• Delivery Service Docs: Updated DELIVERY_SERVICE.md with complete configuration examples
• Code Comments: Improved inline documentation throughout the codebase

---

🔒 Security & Stability

• Foreign Key Constraints: Enabled for data integrity across all database operations
• Reference Counting: Prevents orphaned blobs and ensures safe deletion
• Transaction Safety: Improved transaction handling in blob operations
• Error Recovery: Better error handling and recovery mechanisms

---

📦 Dependencies

• Go 1.25
• SQLite3 (via github.com/mattn/go-sqlite3 v1.14.32)
• AWS SDK v2 (for S3-compatible storage)
• YAML v2 (for configuration management)

---

🔄 Migration Notes

Configuration Changes

• Review delivery.yaml and raven.yaml for any new options
• No breaking changes to existing configuration format

---

🎯 Performance Metrics

Based on testing with real-world email data:

• Storage Reduction: 40-60% reduction in blob storage for typical usage patterns
• Deduplication Rate: 2-3x reference count for common attachments (logos, signatures)
• Memory Usage: Improved with better connection caching

---

🙏 Contributors

This release includes contributions from the Raven development team. Special thanks to everyone who reported issues, tested features, and provided feedback.

---

📝 Full Changelog

For a complete list of changes, see the commit history.

---

📞 Support

• Issues: GitHub Issues
• Documentation: docs/README.md
• Security: See SECURITY.md

---

Download: Available as Docker image or build from source
License: Apache 2.0

---

This discussion was created from the release Raven MDA v0.4.0.