Skip to content

pycontextdb 0.4.8: backdated event_time validity, point-in-time as_of recall, and BM25 lexical fusion

Choose a tag to compare

@atomsai atomsai released this 21 Oct 14:19
· 2 commits to main since this release

Backdated event_time validity, point-in-time as_of recall, and BM25 lexical fusion for AI agent memory — pycontextdb 0.4.8

pycontextdb 0.4.8 ships three retrieval and temporal upgrades in the open-source ContextDB SDK for teams building AI agents that backfill history, answer point-in-time questions, and look up exact names, dates, and schema fields alongside action checks:

  • Backdated event_time validity: factual.add(), add_fast(), and add_many() accept optional event_time and set valid_from = event_time (closing superseded slot occupants at valid_until = event_time), so backfilled conversation turns and documents carry their real validity window instead of the ingestion wall clock.
  • Pre-truncation as_of recall and salience: RetrievalEngine.search_scored() applies as_of validity filtering in both SQLiteStore and PostgresStore before top-k candidate truncation and computes exponential salience decay relative to as_of.
  • BM25 keyword + semantic RRF fusion: RetrievalEngine.search_scored() adds a general BM25 keyword ranking arm (_bm25_ranking) fused via Reciprocal Rank Fusion with semantic and graph rankings. On a warm PostgresStore, search_text() reads from the in-process revision-gated cache with zero extra SQL queries. The recall_for_action relevance-floor bypass stays restricted to digit-bearing identifier_tokens.
  • Chronological & contested prompt rendering: render_recalled_context() adds optional include_timestamps=True and chronological=True ordering and marks CONTESTED memories.

What was wrong in 0.4.7:

  • Passing event_time to ContextDB.add() left valid_from set to the current wall clock, and factual.add(), add_fast(), and add_many() did not accept event_time. Backfilled turns could not be queried as of a historical timestamp.
  • ContextDB.search(..., as_of=...) filtered is_valid_at(as_of) after search_scored() had already truncated the candidate pool to top_k * 3, so 30 newer turns drowned out an earlier turn that was valid at as_of.
  • Lexical search only triggered on tokens containing a digit (identifier_tokens). Queries for non-numeric terms ("Flask-Login", "March", "category") relied on vector similarity alone.

Install or upgrade

pip install --upgrade 'pycontextdb==0.4.8'

No schema migration is required. Existing SQLite and PostgreSQL stores work without changes.

Runnable example

Offline, with the mock embedder and mock model:

import asyncio
import tempfile
from datetime import datetime, timezone

import contextdb
from contextdb.core.config import ContextDBConfig


async def main():
    folder = tempfile.mkdtemp()
    db = contextdb.init(
        user_id="synthetic_customer_8",
        config=ContextDBConfig(
            storage_url=f"sqlite:///{folder}/demo.db",
            embedding_model="mock",
            llm_model="mock",
            llm_api_key="mock",
            enable_auto_link=False,
        ),
    )
    async with db:
        await db.factual.add_many(
            [
                {
                    "content": "Pilot budget is 32k for Q1.",
                    "source": "user_stated",
                    "entity": "pilot",
                    "attribute": "budget",
                    "event_time": "2026-02-03T09:00:00Z",
                },
                *[
                    {
                        "content": f"July status update {i}: pilot budget moved to 40k.",
                        "source": "user_stated",
                        "entity": "pilot",
                        "attribute": "budget",
                        "event_time": "2026-07-28T16:00:00Z",
                    }
                    for i in range(30)
                ],
            ]
        )
        march = await db.factual.recall(
            "what was the pilot budget?",
            top_k=3,
            as_of=datetime(2026, 3, 1, tzinfo=timezone.utc),
        )
        august = await db.factual.recall(
            "what is the pilot budget?",
            top_k=3,
            as_of=datetime(2026, 8, 1, tzinfo=timezone.utc),
        )
        print("as of March:", [m.content for m in march])
        print("as of August:", [m.content for m in august][:1])


asyncio.run(main())

Verified output on October 21, 2026:

pycontextdb 0.4.7
as of March: []
as of August: ['July status update 29: pilot budget moved to 40k.']

pycontextdb 0.4.8
as of March: ['Pilot budget is 32k for Q1.']
as of August: ['July status update 29: pilot budget moved to 40k.']

Use cases

  • Support and account teams backfilling historical tickets or call transcripts: stamp each turn with its original event_time and query what the agent knew at any past date with as_of.
  • Engineering and operations copilots querying long multi-session histories: BM25 + semantic RRF finds exact library names, schema fields, and dates alongside vector similarity without extra database round-trips on a warm PostgresStore.
  • Voice and scheduling agents enforcing recall_for_action: action-gating rules stay strict (identifier_tokens only for relevance-floor bypass; CONTESTED memories marked in rendered context).

Verified behavior

  • 215 unit and trust-model eval tests pass across Python 3.10, 3.11, and 3.12, plus 29 real PostgreSQL 16 concurrency, connector, evolution, and warm-cache query-count tests (test_warm_project_recall_uses_only_revision_query).
  • On Conversation 1 of the public BEAM 100K benchmark (20 questions across 10 categories, unmodified ExaBase prompts, intfloat/multilingual-e5-small embeddings, gemini-3.1-pro-preview answer model), knowledge_update rose from 25.0% to 100.0% and information_extraction from 0.0% to 100.0% (overall 60.33% to 76.58% with zero LLM ingestion).

Compatibility and limitations

  • Apache-2.0 grant: pycontextdb 0.4.8 is released under Apache-2.0 with no profile cap, no license key, and no phone-home check.
  • No schema migration: event_time, valid_from, and valid_until already exist in both SQLite and PostgreSQL schemas.
  • Warm Postgres cache: PostgresStore.search_text() scans the in-process revision-gated index cache when warm and falls back to ILIKE SQL when cold.
  • Action-floor safety: General BM25 matches participate in RRF ranking for recall(), while recall_for_action() relevance-floor bypass remains restricted to digit-bearing identifier_tokens.

Links

FAQ

Do I need a database migration to upgrade from 0.4.7 to 0.4.8?
No. pycontextdb 0.4.8 uses the existing memories schema in both SQLite and PostgreSQL.

Does BM25 keyword fusion weaken recall_for_action safety?
No. Only digit-bearing identifiers (identifier_tokens) can bypass a configured relevance_floor, and untrusted (third_party, unconfirmed agent_inferred, or contested) memories never authorize an act outcome.

Is pycontextdb 0.4.8 still Apache-2.0?
Yes. Releases through 0.4.8 are permanently Apache-2.0 with no profile cap.