v1.1.2

[cwt]

NeoSQLite v1.1.2 Release Notes

Overview

This is a significant enhancement release that adds full GridFS support with MongoDB-compatible ObjectId functionality. The release includes a complete GridFS implementation that maintains backward compatibility while adding modern ObjectId support for file identification.

Highlights

MongoDB-compatible ObjectId Support for GridFS

• Full ObjectId Implementation: Complete MongoDB-compatible ObjectId support in GridFS operations with automatic generation and hex interchangeability
• Backward Compatibility: Maintains compatibility with existing integer-based file IDs while adding ObjectId support
• Dual-ID System: Supports both integer IDs (primary key) and ObjectIds (_id field) for maximum flexibility
• Enhanced Robustness: Improved error handling and type safety throughout GridFS operations

Comprehensive GridFS Implementation

• PyMongo-compatible API: Full GridFSBucket and legacy GridFS APIs compatible with PyMongo
• Performance Optimized: Uses JSONB type when available for efficient ObjectId storage
• Index Support: Automatic indexing for faster lookups and queries
• Complete Feature Set: Supports upload, download, streaming, metadata, and query operations

New Features

GridFS with ObjectId Support

• GridFSBucket Class: Complete PyMongo-compatible GridFSBucket implementation with ObjectId support
• GridFS Legacy Class: Compatible with legacy PyMongo GridFS API with ObjectId enhancements
• Automatic ObjectId Generation: New ObjectIds automatically generated when no file ID provided during upload
• Manual ID Assignment: Support for user-provided ObjectIds during file upload operations
• Dual Storage System: Files stored with both integer primary key (id) and ObjectId (_id) for compatibility

Enhanced GridFS Capabilities

• Upload Operations: upload_from_stream(), upload_from_stream_with_id() with ObjectId return values
• Download Operations: download_to_stream(), open_download_stream() compatible with ObjectId, integer, and hex string IDs
• Query Support: find() operations support ObjectId queries against _id field
• Metadata Handling: Complete metadata serialization/deserialization with JSON support
• Streaming Operations: Full support for streaming file uploads and downloads
• File Management: Rename, delete, and other file operations work with both ID types

GridFS Operations

• Upload Methods: upload_from_stream(), open_upload_stream(), upload_from_stream_with_id(), open_upload_stream_with_id()
• Download Methods: download_to_stream(), open_download_stream(), open_download_stream_by_name(), download_to_stream_by_name()
• Query Methods: find(), get(), list(), find_one(), exists(), _id-compatible queries
• File Management: delete(), rename(), drop(), delete_by_name(), rename_by_name()
• Metadata Operations: Full metadata support with JSON serialization

Performance Improvements

• JSONB Support: Uses JSONB type when available for efficient ObjectId storage in GridFS
• Index Creation: Automatic unique index on _id column for faster file lookups
• Optimized Queries: Enhanced query processing for GridFS operations with proper ID handling
• Memory Efficiency: Streaming operations maintain memory efficiency for large files

API Changes

Return Value Updates

• upload_from_stream(): Now returns ObjectId instead of integer ID for MongoDB compatibility
• put() in legacy GridFS: Now returns ObjectId instead of integer ID
• All file creation operations: Return ObjectIds for consistent MongoDB compatibility

Parameter Updates

• File ID Parameters: Methods now accept ObjectId, integer, or hex string for file identification
• Backward Compatibility: All existing integer-based operations continue to work unchanged
• Enhanced Flexibility: Can now use ObjectIds, integers, or hex strings interchangeably in most operations

GridFSBucket Methods

• upload_from_stream(): Returns ObjectId, accepts metadata
• download_to_stream(): Accepts ObjectId, integer, or hex string file ID
• open_download_stream(): Accepts ObjectId, integer, or hex string file ID
• find(): Supports ObjectId queries against _id field
• delete(): Accepts ObjectId, integer, or hex string file ID
• rename(): Accepts ObjectId, integer, or hex string file ID

GridFS Legacy Methods

• put(): Returns ObjectId, accepts filename and metadata
• get(): Accepts ObjectId, integer, or hex string file ID
• delete(): Accepts ObjectId, integer, or hex string file ID
• exists(): Accepts ObjectId, integer, or hex string file ID

Technical Benefits

• MongoDB Compatibility: Full compatibility with MongoDB GridFS concepts and ObjectId usage
• Performance Optimization: JSONB type and unique indexing provide enhanced performance
• Thread Safety: Proper locking mechanisms ensure safe concurrent ObjectId generation
• Memory Efficiency: Optimized storage using JSONB format when available
• Backward Compatibility: Full support for existing GridFS code with automatic migration
• Enhanced Error Handling: Better error reporting and validation throughout GridFS operations

Migration Notes

For Existing GridFS Code

All existing GridFS code continues to work unchanged. The main change is that upload operations now return ObjectIds instead of integer IDs. If your code relies on integer IDs being returned, you'll need to update the type assertions.

Updated Code Patterns

# Before v1.1.2 - upload operations returned integers:
file_id = bucket.upload_from_stream("test.txt", data)
assert isinstance(file_id, int)  # This will now fail

# After v1.1.2 - upload operations return ObjectIds:
file_id = bucket.upload_from_stream("test.txt", data)
assert isinstance(file_id, ObjectId)  # This is the new expected behavior

# Both ID types work for subsequent operations:
grid_out = bucket.open_download_stream(file_id)  # Works with ObjectId
grid_out = bucket.open_download_stream(str(file_id))  # Works with hex string
# If you have integer IDs, they still work:
grid_out = bucket.open_download_stream(123)  # Works with integer ID

File Schema Changes

New files will have ObjectIds in the _id field while the integer ID remains in the id field. Existing files maintain their original structure until updated.

Installation

# Standard installation
pip install neosqlite==1.1.2

# For enhanced JSON/JSONB support  
pip install neosqlite[jsonb]==1.1.2

# For memory-constrained processing of large result sets
pip install neosqlite[memory-constrained]==1.1.2

# Install multiple extras
pip install neosqlite[jsonb,memory-constrained]==1.1.2

This release represents a major step forward for NeoSQLite's file storage capabilities, providing a complete and robust GridFS implementation with full MongoDB compatibility while maintaining the performance and reliability that NeoSQLite is known for.

---

This discussion was created from the release v1.1.2.