Skip to content

v0.5.3

Choose a tag to compare

View all tags
@dbsmedya dbsmedya released this 22 Apr 03:11
· 300 commits to main since this release
v0.5.3
42fcfc6

Correctness and performance pass over the search and ingestion paths. No
API breakage; no new dependencies.

Bug fixes

  • Hybrid search now reports scores correctly. Results from the
    hybrid (vector + FTS) path were silently flagged as FTS-only and
    rendered as N/A (FTS Match). The LanceDB _relevance_score column
    is now read separately from the pure-vector _distance column.
  • Search filters survive the FTS fallback. Previously, when the
    FTS index was missing and the query fell back to pure-vector search,
    source_filter and min_time pushdowns were silently dropped —
    callers received unfiltered results. Filters are now re-applied via
    a single helper on whichever op actually runs.
  • MCP tools no longer block the event loop. search_documents and
    search_sql_logs offload SearchService.execute_query via
    asyncio.to_thread, mirroring the FastAPI pattern. Concurrent MCP
    requests over the streamable-HTTP transport are no longer serialized.

Performance

  • Dedup hash scan projects one column. get_existing_hashes now
    reads only content_hash via table.search().select([...]).to_arrow()
    instead of materialising the full table (including 768-dim vectors).
    On a 100k-row SQL index this drops per-ingest I/O from ~300 MB to
    ~2 MB.
  • Unchanged files skip chunking entirely. Ingestion short-circuits
    the chunker when a document's file-level hash is already in the
    store. Re-running ingest over an unchanged markdown tree no longer
    re-parses it.

Internal

  • DI factory extracted to services/bootstrap.py. The API layer
    (api/state.py) no longer imports from cli.py, removing the
    typer + Hugging Face env-var side effects from the API import graph.
    cli._build_dependencies remains as a thin wrapper that converts
    schema-mismatch errors to typer.Exit.

Upgrade notes

  • Storage-stack refresh: lance==1.2.1 (dead PyPI pin from 2020) removed; lancedb 0.29.2 → 0.30.2; pyarrow 19 → 24 (5 major versions, zero code change); polars 1.38 → 1.40.
  • ML stack refresh: mlx-embeddings 0.0.5 → 0.1.0, which pulled transformers 4.x → 5.x. Validated with live model load and smoke-test search against the embeddinggemma-300m-bf16 model.
  • Bug B2 (hybrid-score display): services/search.py:53, api/mcp_server.py:42, api/mcp_server.py:91 — all three display sites were branching only on distance, mislabeling every hybrid hit as N/A (FTS Match). Fixed with a 3-branch check (distance → score → N/A), new TDD red-then-green unit tests for both the CLI and MCP surfaces, and live e2e verification (Score: 0.0325 on paraphrased semantic query, correctly retrieved semantically-close chunks).
  • Total test count: 193 → 195 (2 added for B2).

Full Changelog: v0.5.1...v0.5.3

Assets 4
Loading