Skip to content

Conversation

@dluc
Copy link
Collaborator

@dluc dluc commented Nov 27, 2025

Summary

Implements SQLite-based content storage system with distributed locking and queue management.

Key Features

  • Two-phase write pattern: Queue operation → Optimize queue → Process synchronously
  • Distributed locking: Multi-VM support via LastAttemptTimestamp field
  • Operations queue: Automatic cancellation of superseded operations
  • Idempotent operations: Upsert (insert or replace) and Delete
  • JSON storage: Tags as string arrays, metadata as key-value dictionaries
  • Cuid2 IDs: Lowercase collision-resistant identifiers

Architecture

Database Schema

Content Table (km_content):

  • Source of truth for all content
  • String content with MIME type, byte size
  • Ready flag for locking during operations
  • Timestamps: ContentCreatedAt, RecordCreatedAt, RecordUpdatedAt
  • Optional: Title, Description, Tags[], Metadata{}

Operations Table (km_operations):

  • Queue management per ContentId
  • Timestamp-ordered processing
  • Multi-step support (single step in this implementation)
  • Tracks: PlannedSteps, CompletedSteps, RemainingSteps
  • LastAttemptTimestamp for distributed locking

Components

  • Entity classes with JSON property handling
  • EF Core DbContext with SQLite configuration
  • Service layer implementing two-phase write pattern
  • DTOs for clean API boundaries
  • Custom exception types
  • Cuid2-based ID generation

Test Coverage

74 tests, 100% passing:

  • 43 configuration tests (existing)
  • 31 storage tests (19 unit + 12 integration)

Implementation Details

  • Zero build warnings (strict compliance)
  • DateTimeOffset for all timestamps (culture-invariant ISO 8601)
  • Hardcoded table names with km_ prefix to avoid collisions
  • No recovery logic in user requests (deferred to future background worker)
  • Comprehensive error handling and logging

Files Changed

New files (10):

  • src/Core/Storage/ - Complete implementation
  • tests/Core.Tests/Storage/ - Test suite

Modified files:

  • src/Core/Core.csproj - Added Cuid.Net package
  • Test visibility changes for xUnit compliance
  • Package version management

dluc added 2 commits November 27, 2025 11:22
Implements feature 00002: SQLite-based content storage system with distributed locking and queue management.

Key features:
- Two-phase write pattern (queue + optimize + process)
- Distributed locking via LastAttemptTimestamp for multi-VM support
- Operations queue with automatic cancellation optimization
- Upsert and Delete operations (both idempotent)
- JSON storage for tags and metadata
- Complete test coverage (74 tests: 43 config + 31 storage)

Components:
- Entity classes: ContentRecord, OperationRecord
- DbContext with SQLite configuration and indexes
- Service layer with two-phase write implementation
- DTOs and models for clean API boundaries
- Cuid2-based ID generation
- Custom exception types for error handling

All tests passing (100%), zero build warnings.
…zer warnings

- Removed all ConfigureAwait calls from test files (xUnit1030 compliance)
- Made all test classes public (xUnit1000 requirement)
- Added analyzer suppressions to test project (CA1812, xUnit1030)
- Build now passes with 0 warnings, 0 errors
@dluc dluc merged commit 118d2fd into microsoft:main Nov 27, 2025
3 checks passed
@dluc dluc deleted the contentstorage branch November 27, 2025 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant