Skip to content

v0.3.4 - Critical Data Integrity Fixes

Choose a tag to compare

View all tags
@pillowsoft pillowsoft released this 03 Jan 11:19
· 4 commits to main since this release
v0.3.4
a18253c

🐛 Critical Data Integrity Fixes

This release addresses 5 critical bugs identified in a comprehensive package audit, focusing on data integrity, performance, and error handling.

Critical Fixes

1. Fixed clearAll() Database Crash

  • ✅ Removed non-existent table operations - clearAll() no longer attempts to delete 'entities' and 'attributes' tables
  • ✅ Added metadata table cleanup - Now properly clears all three tables: triples, transactions, metadata
  • 🔥 Impact: Prevented guaranteed crash when calling db.clearLocalDatabase()

Root Cause: Method attempted to delete from tables that don't exist in the schema.

2. Added Critical Performance Index

  • ✅ Index on retracted column - All queries now use indexed lookups instead of full table scans
  • ⚡ Performance improvement - Query performance now O(log n) instead of O(n) as database grows
  • 📊 Impact: Prevents performance degradation as database accumulates retracted triples

Root Cause: Every query filters by WHERE retracted = FALSE but no index existed on this column.

3. Implemented Database Vacuum/Cleanup

  • ✅ New vacuum() method - Deletes retracted triples older than retention period (default: 30 days)
  • 🧹 Disk space reclamation - Runs VACUUM to reclaim disk space after cleanup
  • 📈 Prevents unbounded growth - Database no longer grows indefinitely from retracted data
  • 🔧 Configurable retention - vacuum(retentionDays: 30) allows custom retention policies

Usage:

// Clean up old retracted triples and reclaim disk space
final deletedCount = await (db._store as TripleStore).vacuum();
print('Cleaned up $deletedCount old triples');

4. Server Error Rollback

  • ✅ Automatic rollback on server rejection - Optimistic updates are now rolled back when server sends errors
  • ✅ Prevents data divergence - Client state no longer shows data that server rejected
  • 🔍 Enhanced error logging - Clear logs when transactions are rejected and rolled back

Impact: Prevents scenarios where UI shows created/updated entities that server never accepted.

5. Attribute Cache Failure Protection

  • ✅ Fail loudly instead of silently - Unknown attributes now throw exceptions instead of being skipped
  • 🛡️ Prevents silent data loss - Ensures all attributes are sent to server or transaction fails
  • 📝 Detailed error messages - Shows available attributes for debugging schema sync issues

Impact: Transactions now fail fast with clear error messages when schema is out of sync.

📊 Audit Summary

This release is based on a comprehensive package audit that reviewed:

  • ✅ All 11 REST API endpoints (verified correct)
  • ✅ SQLite triple store architecture
  • ✅ WebSocket synchronization engine
  • ✅ Transaction flow and error handling
  • ✅ Cache invalidation strategies

Severity Breakdown:

  • 🔥 5 Critical issues fixed (this release)
  • ⚠️ 6 Moderate issues (planned for future releases)
  • 📝 4 Minor issues (planned for future releases)

Testing: All fixes tested against existing test suite (149/155 tests passing).

Migration: No breaking changes - all fixes are backward compatible improvements.

Files Changed

  • lib/src/storage/triple_store.dart - clearAll() fix, index addition, vacuum() method
  • lib/src/sync/sync_engine.dart - Server error rollback, attribute cache protection
  • pubspec.yaml - Version bump to 0.3.4
  • CHANGELOG.md - Complete documentation
Assets 2
Loading