Skip to content

narrow purge_stale_index_entries scan to indexed fields #90

Description

@fabracht

Context

PR #89 introduced purge_stale_index_entries in crates/mqdb-agent/src/database/query.rs. It scans the full idx/{entity}/ prefix and filters by ends_with("/{id}").

Proposed optimization

Ask the schema registry (or index manager) for the list of indexed fields on the entity, then scan only idx/{entity}/{field}/ for each. Typically a smaller cardinality than the entity-wide prefix scan.

Why

  • Cheaper self-heal on entities with many indexed fields and large per-field key sets.
  • Lets us drop the suffix-matching heuristic in favour of a more precise scan (still need ends_with because value may contain /, but the search space shrinks).

Out of scope

  • The current full-prefix scan is correct and the self-heal path is rare. This is a perf optimisation, not a bug.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions