Skip to content

Release Notes - v0.3.0

Choose a tag to compare

@oranheim oranheim released this 02 Nov 16:29
· 20 commits to master since this release

πŸŽ‰ Native pgvector Support

This release adds native support for PostgreSQL's pgvector extension with automatic vector field detection, following the same pattern as our existing JSONB support.

✨ New Features

pgvector Integration

  • Automatic vector field detection - Auto-detects list[float] type hints as pgvector columns
  • Lazy async adapter registration - Uses register_vector_async() for proper async handling
  • Graceful degradation - Works seamlessly even if pgvector package is not installed
  • Per-connection registration - Properly handles connection pooling
  • Type-safe operations - Full type safety with list[float] <-> PostgreSQL vector

Repository Parameters

  • auto_detect_vector (default: True) - Automatically detect vector fields from type hints
  • vector_fields - Explicitly specify vector field names (overrides auto-detection)
  • Vector fields automatically excluded from JSON processing

Usage Example

from pydantic import BaseModel
from uuid import UUID

class DocumentEmbedding(BaseModel):
    id: UUID
    embedding: list[float]  # Auto-detected as vector!
    metadata: dict[str, Any]

class EmbeddingRepository(BaseRepository[DocumentEmbedding, UUID]):
    def __init__(self, db_connection):
        super().__init__(
            db_connection=db_connection,
            table_name="embeddings",
            model_class=DocumentEmbedding,
            primary_key="id",
            # auto_detect_vector=True is default
        )

# Just works - no manual parsing needed!
doc = DocumentEmbedding(
    id=uuid4(),
    embedding=[0.1] * 384,
    metadata={"model": "all-MiniLM-L6-v2"}
)
created = await repo.create(doc)
assert isinstance(created.embedding, list)  # βœ… Returns as list[float]

πŸ§ͺ Testing Improvements

New Tests

  • 12 unit tests - Vector field detection and type inspection
  • 4 integration tests - Full CRUD operations with pgvector
  • 227 total tests - All passing with zero regressions

Test Organization

  • Reorganized tests into proper subdirectories:
    • tests/unit/ - Unit tests (mocked, no database)
    • tests/integration/ - Integration tests (real PostgreSQL)
    • tests/performance/ - Performance benchmarks
  • Updated testcontainers to use pgvector/pgvector:pg17
  • Added float32 precision handling (pgvector uses float32, Python uses float64)

πŸ“¦ Dependencies

  • Optional dependency: pgvector>=0.4.1 in vector dependency group
  • Install with: pip install psycopg-toolkit[vector]
  • Graceful degradation if not installed

πŸ“š Documentation

  • βœ… Updated CLAUDE.md with pgvector support details
  • βœ… Updated README.md with pgvector feature
  • βœ… Added comprehensive usage examples
  • βœ… Documented vector field configuration options
  • βœ… Added pgvector support components section

πŸ› οΈ Code Quality

  • Fixed all ruff linting issues (SIM102, B905, SIM222)
  • Applied consistent code formatting
  • Added comprehensive async testing guide to README
  • Improved test organization and structure

πŸ“Š Statistics

+681 lines added
-388 lines removed
18 files changed

Core implementation:
- src/psycopg_toolkit/repositories/base.py (+72 lines)
- src/psycopg_toolkit/utils/type_inspector.py (+102 lines)

New tests:
- tests/integration/test_pgvector.py (119 lines, 4 tests)
- tests/unit/test_type_inspector.py (+174 lines, 12 tests)

πŸ”§ Technical Details

Type Detection (TypeInspector)

  • detect_vector_fields() - Detects list[float] fields
  • Supports Optional[list[float]] and list[float] | None
  • Handles legacy typing.List[float] syntax
  • Excludes vector fields from JSON processing

Adapter Registration (BaseRepository)

  • Lazy registration on first database operation
  • Uses async variant register_vector_async()
  • Per-connection registration (not global)
  • Logs helpful warnings if pgvector not installed

Float Precision

  • pgvector uses float32 (4 bytes per dimension)
  • Python uses float64 (8 bytes)
  • Tests use approximate comparison with tolerance of 0.001

πŸ”„ Migration Guide

No breaking changes! Existing code continues to work without modifications.

To use pgvector support:

  1. Install pgvector: pip install pgvector>=0.4.1
  2. Enable pgvector extension in PostgreSQL: CREATE EXTENSION vector;
  3. Use list[float] type hints in your Pydantic models
  4. Vector fields are automatically detected and handled!

πŸ™ Credits

Feature implemented following the AI instruction specification and integrated following the existing JSONB adapter pattern for consistency.


Full Changelog: v0.2.2...v0.3.0