Skip to content

v0.3.0

Choose a tag to compare

@PaulArgoud PaulArgoud released this 09 Jun 17:17
· 40 commits to main since this release

Feature pass: retrieval quality, observability, operations.

Added

  • Schema versioningmxchat_duckdb_schema_meta table tracks the current
    schema version; migrations run sequentially and idempotently. Target version
    is 3 in this release.
  • Hybrid BM25 + vector search — when hybrid_enabled is on and a
    mxchat_duckdb_query_text filter supplies the user query, results from
    DuckDB's FTS extension are min-max-normalised and blended with cosine
    similarity using configurable hybrid_alpha. Falls back gracefully to pure
    vector when FTS is unavailable.
  • Query result cache — top-K matches are cached in a transient keyed by
    md5(embedding) + bot_id + filter + top_k. Default TTL 300 s, configurable.
    Automatically invalidated on upsert/delete.
  • Per-source dedup — optional collapse of multiple chunks from the same
    source_url in the final top-K (the LLM gets distinct sources, not five
    near-duplicates).
  • Pinecone-style filter operators$eq, $ne, $in, $nin, $gt,
    $gte, $lt, $lte on content_type, role_restriction, source_url,
    chunk_index. Unknown operators/fields are silently skipped (Pinecone parity).
  • Re-ranking hookmxchat_duckdb_rerank_matches filter receives the
    top-K and can return a re-ordered set (cross-encoders, Cohere Rerank, …).
  • Metrics class — rolling 1-hour window of latency samples; exposes
    p50/p95/p99 + cache hit rate + counters via MxChat_DuckDB_Metrics::snapshot()
    and the admin diagnostics panel.
  • Slow-query log — queries slower than slow_query_ms (default 500) are
    written to the PHP error log with bot id and hybrid/dedup flags.
  • MotherDuck retry + backoff — idempotent queries (SELECT/WITH/PRAGMA/…)
    automatically retry up to 3× on transient errors (timeout, EOF, 502/503,
    rate-limit) with exponential backoff + jitter.
  • /health REST endpointGET /wp-json/mxchat-duckdb/v1/health returns
    backend status, vector count, last sync age, metrics snapshot. Public by
    default; restrict with the mxchat_duckdb_health_public filter.
  • WP-CLI command surfacewp mxchat-duckdb {test|stats|sync|reprocess| compact|metrics|cache}. Indispensable for scripted deployments.
  • Compactor — daily mxchat_duckdb_compact cron job that deletes vectors
    whose vector_id no longer maps to any row in the MySQL KB. Capped per run
    (filter mxchat_duckdb_compactor_max_deletes, default 5000) and skipped if
    the last sync was within the past hour.
  • Per-namespace proxy tokensMxChat_DuckDB_Pinecone_Proxy::get_or_create_token_for($namespace)
    issues tokens scoped to a single bot, so a leak on bot A no longer grants
    access to bot B's vectors. The legacy global token still works as a
    fallback (wildcard scope).
  • Nonce + capability check on cascade delete — the
    wp_ajax_mxchat_delete_pinecone_prompt handler now validates the nonce and
    manage_options capability itself instead of relying on mxchat's check
    running afterwards.
  • Composer autoloadercomposer.json ships with a classmap autoload
    over includes/; the bootstrap prefers vendor/autoload.php when present
    and falls back to manual require_once otherwise. Non-breaking.
  • i18n — full gettext setup: Text Domain + Domain Path headers,
    load_plugin_textdomain(), English source strings, mxchat-duckdb.pot
    template + ready-to-use mxchat-duckdb-fr_FR.po/.mo.

Changed

  • All __() / _e() source strings converted from French to English; French
    is now shipped as the fr_FR translation (canonical WordPress convention).
  • Default embedded_path directory now hosts a .htaccess + index.php +
    web.config trio (was added in 0.2.0; mentioned here for completeness).
  • New options added with sensible defaults:
    hybrid_enabled (false), hybrid_alpha (0.7), query_cache_enabled (true),
    query_cache_ttl (300), dedup_per_source (false), slow_query_ms (500),
    last_compact_at (0).

New filters

  • mxchat_duckdb_query_text — populate the user query text for BM25.
  • mxchat_duckdb_rerank_matches — custom reranker hook.
  • mxchat_duckdb_max_retries — override the retry attempts (default 3).
  • mxchat_duckdb_health_public — gate the /health endpoint behind auth.
  • mxchat_duckdb_compactor_max_deletes — per-run cap (default 5000).

Fixed

  • Cascade-delete handler no longer trusts mxchat's nonce check happening
    later; verifies the nonce itself first. Defense-in-depth.
  • uninstall.php — dotfile cleanup. glob($dir.'/*') silently skipped
    the .htaccess we wrote into the data directory, leaving an orphan that
    prevented rmdir() from succeeding. Now uses recursive scandir().
  • uninstall.php — custom embedded_path honoured. Plugin options are
    read before deletion so a user-configured path outside uploads/ is
    also cleaned (file + .wal / .tmp / .lock companions). The parent
    directory is left untouched in case it's shared with other tools.
  • uninstall.php — multisite. On multisite the cleanup now iterates
    every blog (switch_to_blog() loop) so per-site options / cron / transients
    are removed for the whole network, not just the main site.

Notes

  • uninstall.php deliberately never makes a network call. To wipe data on
    MotherDuck, run DROP TABLE mxchat_vectors; DROP TABLE mxchat_duckdb_schema_meta;
    manually at app.motherduck.com.