v1.13.10

[cwt]

NeoSQLite v1.13.10 Release Notes

TL;DR

NeoSQLite v1.13.10 is a bug fix and code quality release that resolves 28 bugs across all severity levels, modernizes code with Python 3.10 pattern matching, and improves reliability of ChangeStream, bulk operations, aggregation, and connection lifecycle management.

• Upgrade if you use: ChangeStream (CPU spin fix), bulk operations, aggregation pipelines with $bucketAuto/$merge, or transactions
• No need to upgrade if: You're on v1.13.9 and don't hit any of the fixed issues
• Breaking changes: None — all existing code continues to work

---

Breaking Changes

None — v1.13.10 is fully backward compatible with v1.13.9.

---

Overview

This release focuses on reliability and correctness:

• 28 bugs fixed across high, medium, and low severity levels
• Python 3.10 pattern matching modernization across type_correction, query_operators, and collection modules
• ChangeStream CPU spin fixed — incorrect None check on fetchall() result
• NULL data column crash prevented in aggregation query results
• Transaction leaks fixed in document processing and unparseable data handling
• Unordered bulk operations now correctly continue on individual failures (MongoDB semantics)
• Aggregation correctness fixes for $bucketAuto, $merge, $sample, and $lookup
• Connection lifecycle improvements for cloned connections and interpreter shutdown safety
• Logging visibility improvements for write concern errors, pending transactions, and aggregation fallbacks

---

Bug Fixes

High Severity

Fixed: Savepoint leak in bulk_write

Problem: If RELEASE SAVEPOINT failed, the savepoint was never cleaned up, corrupting the transaction stack.

Before:

self.collection.db.execute("SAVEPOINT bulk_write")
try:
    for req in requests:
        # ... process requests
    self.collection.db.execute("RELEASE SAVEPOINT bulk_write")
except Exception as e:
    self.collection.db.execute("ROLLBACK TO SAVEPOINT bulk_write")
    raise e
# No cleanup if RELEASE itself fails

After:

released = False
self.collection.db.execute("SAVEPOINT bulk_write")
try:
    for req in requests:
        # ... process requests
    self.collection.db.execute("RELEASE SAVEPOINT bulk_write")
    released = True
except Exception as e:
    self.collection.db.execute("ROLLBACK TO SAVEPOINT bulk_write")
    raise e
finally:
    if not released:
        try:
            self.collection.db.execute("RELEASE SAVEPOINT bulk_write")
        except Exception:
            pass

Fixed: Unordered bulk operations stop on first error

Problem: _execute_unordered delegated to _execute_ordered, violating MongoDB semantics where unordered operations should continue on individual failures.

After: Each operation is wrapped in its own try/except. Failures are logged and execution continues to the next operation.

Fixed: Clone connection write concern mutation

Problem: with_options() called _apply_write_concern() on the clone, which mutated the shared sqlite3.Connection's PRAGMAs and affected the original connection.

Fix: Write concern is now stored for API compatibility only, without mutating the shared connection.

Fixed: Connection.del crash during interpreter shutdown

Problem: During garbage collection at interpreter shutdown, module-level imports (like sqlite3) may already be None, causing AttributeError.

Fix: Wrapped self.close() in try/except to silently ignore errors during interpreter shutdown.

Fixed: ChangeStream CPU spin

Problem: Incorrect None check on fetchall() result caused ChangeStream to spin in a tight loop consuming 100% CPU.

Fix: Corrected the check to properly handle empty result lists vs None.

Fixed: NULL data column crash in aggregation

Problem: When aggregation query results contained a NULL data column, the code crashed instead of handling it gracefully.

Fix: Added NULL check before processing aggregation results.

Fixed: Transaction leak in document processing

Problem: Commit was executed before document processing completed, causing data loss if processing failed.

Fix: Moved commit after document processing to ensure data integrity.

Fixed: Unparseable document data leaves transactions open

Problem: When document data couldn't be parsed, the transaction was left open, causing subsequent operations to fail.

Fix: Added graceful error handling that properly closes transactions on unparseable data.

Medium Severity

Fixed: $bucketAuto returns numeric index instead of {min, max}

Problem: MongoDB's $bucketAuto sets _id to {min: <lower>, max: <upper>}. NeoSQLite was using a simple numeric index.

After: _id is now {"min": <bucket_min>, "max": <bucket_max>} matching MongoDB behavior.

Fixed: $merge "replace" and "merge" modes identical

Problem: Both modes did $set only. In MongoDB, "replace" should also remove fields that exist in the old document but not in the new one.

After: "replace" mode now also $unset fields missing from the source document.

Fixed: bulk_write ignores session parameter

Problem: The session parameter was accepted but never passed to individual CRUD operations.

Fix: session is now passed through to insert_one, update_one, and delete_one.

Fixed: update_many/delete_many double-translate filter

Problem: translate_match was called twice — once before the SQL attempt and again in the fallback path.

Fix: The already-translated where_clause is now reused in the fallback path.

Fixed: $lookup and aggregation fallback don't respect session

Problem: $lookup stage and the Python fallback path in aggregate_with_constraints called find() without passing the session parameter.

Fix: session=session is now passed through.

Fixed: Dead code — duplicate $unset case in aggregation

Problem: A second $unset case in the match statement was unreachable because the first match always won.

Fix: Removed the duplicate unreachable case.

Fixed: Logging visibility for write concern and pending transactions

• _apply_write_concern errors now log at warning level (not debug-only)
• Connection.close() now warns when auto-committing a pending transaction
• $expr evaluation failures now log with exception details

Fixed: Silent exception handlers in aggregation tier fallbacks

Problem: Exceptions in aggregation SQL tier fallbacks were silently swallowed, causing incorrect results without any indication.

Fix: Added debug logging to silent exception handlers to aid troubleshooting.

Fixed: Incomplete logical condition in SQL fallback

Problem: A logical condition wasn't properly checking all cases, causing partial results to be silently discarded instead of triggering Python fallback.

Fix: Corrected the condition to properly signal when Python fallback is needed.

Fixed: Empty pipeline stages cause StopIteration crash

Problem: Empty pipeline stages raised StopIteration instead of a proper error.

Fix: Added validation that rejects empty pipeline stages with a clear error message.

Fixed: Negative $sample size causes ValueError

Problem: Passing a negative $sample size raised an unhandled ValueError.

Fix: Added validation that rejects negative sample sizes.

Fixed: Redundant exception type in JSON validation

Problem: json.JSONDecodeError was listed in an except clause alongside Exception, which already covers it.

Fix: Removed the redundant exception type and added debug logging for JSON validation failures.

Low Severity

• Cursor.clone() now copies session context
• compact now calculates actual bytes freed (before/after VACUUM)
• dbstats uses proper ternary instead of fragile and/or pattern
• ClientSession raises InvalidOperation instead of bare Exception
• ChangeStream cleanup no longer has redundant pass after logging
• _exists unreachable code removed

---

Code Modernization: Python 3.10 Pattern Matching

Three modules have been refactored to use Python 3.10 match/case pattern matching:

type_correction.py

# Before
if isinstance(value, dict):
    ...
elif isinstance(value, list):
    ...

# After
match value:
    case dict():
        ...
    case list():
        ...

query_operators.py

Bitwise operators now use pattern matching for cleaner value extraction:

match value:
    case {"$bitsAllSet": bits}:
        return _bits_all_set(field, bits, document)
    case {"$bitsAnySet": bits}:
        return _bits_any_set(field, bits, document)
    # ...

collection/init.py

_parse_stored_id now uses pattern matching for cleaner type dispatch.

---

Test Statistics

Unit Tests

Metric	Result
Total Tests	2,415
Passed	2,410
XFailed	5
Failed	0
Code Coverage	81%

API Comparison Tests

NeoSQLite vs MongoDB

Metric	v1.13.10
Total Tests	377
Passed	360
Skipped	17
Failed	0
Compatibility	100%

---

Installation

pip install neosqlite
# or
poetry install

NX-27017

pip install "neosqlite[nx27017]"
# or
poetry install -E nx27017

---

Migration Guide

Upgrading from v1.13.9

No breaking changes — v1.13.10 is fully backward compatible.

If You Use Unordered Bulk Operations

Unordered bulk operations now correctly continue on individual failures (matching MongoDB semantics). Previously, they stopped on the first error like ordered operations. If your code relied on the old broken behavior, review your error handling.

If You Use $bucketAuto

The _id field in $bucketAuto output now returns {"min": ..., "max": ...} instead of a numeric index. Update any code that expected numeric indices.

If You Use $merge with "replace" Mode

"replace" mode now correctly removes fields from the target document that are not present in the source document. Previously it only updated fields (same as "merge").

---

Known Limitations

NX-27017 Limitations

• No replication/sharding: Coming never™ — This is NX-class, not NCC-1701!
• Change stream persistence: Changes are tracked in memory; restart server = lost streams
• find_raw_batches with batch_size: Requires cursor state management

---

Credits

Bug fixes and code modernization developed as part of the NeoSQLite project.

---

NeoSQLite v1.13.10 — Reliability: 28 Bugs Fixed, Python 3.10 Pattern Matching, MongoDB Semantics Corrected.

---

This discussion was created from the release v1.13.10.