Skip to content

Lockbox v0.1.0

Pre-release
Pre-release

Choose a tag to compare

View all tags
@TFMV TFMV released this 12 Jun 05:11
· 38 commits to main since this release
v0.1.0
05dc062
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Lockbox v0.1.0 Release Notes

Overview

Lockbox v0.1.0 is our first official release, providing a secure, high-performance columnar data storage system built on Apache Arrow. This release focuses on delivering core functionality with enterprise-grade security features.

Key Features

🔐 Security

  • AES-256-GCM encryption for all data at rest
  • Column-level encryption with individual keys
  • PBKDF2 key derivation with 100,000 iterations
  • Password-based authentication
  • Secure metadata storage

🚀 Performance

  • Zero-copy Arrow operations
  • Column-level selective decryption
  • Efficient memory usage
  • Optimized file format

🛠️ Core Functionality

  • Arrow IPC and Feather format support
  • Comprehensive CLI interface
  • Go SDK with functional options
  • Schema management
  • Basic audit logging

Technical Details

Dependencies

  • Apache Arrow v18.3.0
  • Zerolog v1.34.0
  • Cobra v1.9.1
  • Viper v1.20.1
  • Go 1.24.3+

Supported Data Types

  • Integers (int8, int16, int32, int64, uint8, uint16, uint32, uint64)
  • Floating Point (float32, float64)
  • Strings (utf8, binary)
  • Boolean
  • Temporal (date32, timestamp, time32ms, duration)

CLI Commands

Create

lockbox create [file] --password [password] [options]

Create a new lockbox file with specified schema.

Write

lockbox write [file] --password [password] [options]

Write data to an existing lockbox file.

Query

lockbox query [file] --password [password] [options]

Query data from a lockbox file.

Info

lockbox info [file] --password [password] [options]

Display information about a lockbox file.

File Format (.lbx)

The Lockbox file format consists of:

  • Magic bytes and version header
  • Metadata offset
  • Encrypted data blocks (columnar)
  • Metadata block (schema, encryption params, audit trail)

Security Model

Encryption

  • Algorithm: AES-256-GCM
  • Key Derivation: PBKDF2 (100,000 iterations)
  • Column-Level Keys: Derived from master key
  • Integrity: SHA-256 checksums

Key Management

Master Key = PBKDF2(Password, Salt, 100000 iterations)
Column Key = PBKDF2(Master Key + Column Name, Salt, 100000 iterations)

Performance Characteristics

  • Encryption Overhead: ~15-20% compared to unencrypted Arrow
  • Column Selectivity: Only decrypt requested columns
  • Memory Efficiency: Zero-copy Arrow operations
  • File Size: ~5-10% overhead for metadata and encryption

Installation

# Clone the repository
git clone https://github.com/TFMV/lockbox
cd lockbox

# Build the CLI
go build ./cmd/lockbox

# Run tests
go test ./pkg/lockbox -v

Known Limitations

  1. Complex Arrow types (lists, maps, etc.) are not yet supported
  2. No built-in compression
  3. Limited query capabilities in v0.1.0
  4. No concurrent access support
  5. No built-in backup/restore functionality
Assets 2
Loading