Summary
The metadata_filter clauses constructed in ValkeyStorage._vector_search() reference arbitrary metadata key fields (e.g. @key:value), but the memory_index FT schema only defines the following fields:
VectorField("embedding")TagField("scope")TagField("categories")NumericField("created_at")NumericField("importance")
As a result, metadata_filter cannot actually restrict FT.SEARCH results and will either error or silently return wrong results.
Affected file: lib/crewai/src/crewai/memory/storage/valkey_storage.py (around lines 513–519 and 1237–1245)
Remediation options
Option A — Materialize metadata into the FT index:
Add the necessary metadata fields to the memory_index schema (e.g. as TagField or NumericField depending on type) and update the schema construction accordingly. This requires defining a known set of metadata keys upfront or dynamically managing the index schema.
Option B — Post-filter outside FT.SEARCH:
Remove metadata clauses from the FT query in _vector_search() and instead apply metadata_filter as a post-filter over the FT.SEARCH results before returning final hits. This is simpler but may return more documents from Valkey than strictly necessary.
Context
Raised during review of PR #5703 (ValkeyStorage vector memory backend, part 4/4). Deferred to a follow-up PR so maintainers can decide the preferred approach, given the broader schema implications.
References:
Summary
The
metadata_filterclauses constructed inValkeyStorage._vector_search()reference arbitrary metadata key fields (e.g.@key:value), but thememory_indexFT schema only defines the following fields:VectorField("embedding")TagField("scope")TagField("categories")NumericField("created_at")NumericField("importance")As a result,
metadata_filtercannot actually restrictFT.SEARCHresults and will either error or silently return wrong results.Affected file:
lib/crewai/src/crewai/memory/storage/valkey_storage.py(around lines 513–519 and 1237–1245)Remediation options
Option A — Materialize metadata into the FT index:
Add the necessary metadata fields to the
memory_indexschema (e.g. asTagFieldorNumericFielddepending on type) and update the schema construction accordingly. This requires defining a known set of metadata keys upfront or dynamically managing the index schema.Option B — Post-filter outside FT.SEARCH:
Remove metadata clauses from the FT query in
_vector_search()and instead applymetadata_filteras a post-filter over theFT.SEARCHresults before returning final hits. This is simpler but may return more documents from Valkey than strictly necessary.Context
Raised during review of PR #5703 (ValkeyStorage vector memory backend, part 4/4). Deferred to a follow-up PR so maintainers can decide the preferred approach, given the broader schema implications.
References: