haystack.errors.FilterError: 'field' or 'value' not found for 'in'

[Dharmik-Solanki]

Describe the bug
I'm running deepseek-r1 model with ollama. I followed step by step launching process via launcher app on windows.

After that i have selected sample e-commerce dataset as my database and asked example question after that it will display the process of loading. It throws the error after few minutes (around 2 minutes).

attaching the log of wren ai service below ,(Note : i have not include above repeated part here first 3 line is repeated part ).

log :
2025-03-25 14:45:51 INFO: 172.18.0.6:57090 - "GET /v1/question-recommendations/10c01a2d-a7d1-47de-b35f-5ba820312f60 HTTP/1.1" 200 OK
2025-03-25 14:45:52 INFO: 172.18.0.6:57102 - "GET /v1/question-recommendations/10c01a2d-a7d1-47de-b35f-5ba820312f60 HTTP/1.1" 200 OK
2025-03-25 14:45:53 INFO: 172.18.0.6:57104 - "GET /v1/question-recommendations/10c01a2d-a7d1-47de-b35f-5ba820312f60 HTTP/1.1" 200 OK
2025-03-25 14:45:53 I0325 09:15:53.273 8 wren-ai-service:482] Ask Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.277 8 wren-ai-service:147] SqlPairs Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.279 8 wren-ai-service:178] Instructions Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.282 8 wren-ai-service:482] Ask Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.284 8 wren-ai-service:147] SqlPairs Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.286 8 wren-ai-service:178] Instructions Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.287 8 wren-ai-service:482] Ask Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.289 8 wren-ai-service:147] SqlPairs Retrieval pipeline is running...
2025-03-25 14:45:53 I0325 09:15:53.290 8 wren-ai-service:178] Instructions Retrieval pipeline is running...
2025-03-25 14:45:53
2025-03-25 14:45:53 ********************************************************************************
2025-03-25 14:45:53 > table_retrieval [src.pipelines.retrieval.retrieval.table_retrieval()] encountered an error<
2025-03-25 14:45:53 > Node inputs:
2025-03-25 14:45:53 {'embedding': "<Task finished name='Task-1410' coro=<AsyncGraphAd...",
2025-03-25 14:45:53 'project_id': '',
2025-03-25 14:45:53 'table_retriever': '<src.providers.document_store.qdrant.AsyncQdrantEm...',
2025-03-25 14:45:53 'tables': None}
2025-03-25 14:45:53 ********************************************************************************
2025-03-25 14:45:53 Traceback (most recent call last):
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 122, in new_fn
2025-03-25 14:45:53 await fn(**fn_kwargs) if asyncio.iscoroutinefunction(fn) else fn(**fn_kwargs)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 219, in async_wrapper
2025-03-25 14:45:53 self._handle_exception(observation, e)
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 520, in _handle_exception
2025-03-25 14:45:53 raise e
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 217, in async_wrapper
2025-03-25 14:45:53 result = await func(*args, **kwargs)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/pipelines/retrieval/retrieval.py", line 160, in table_retrieval
2025-03-25 14:45:53 return await table_retriever.run(
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/providers/document_store/qdrant.py", line 359, in run
2025-03-25 14:45:53 docs = await self._document_store._query_by_filters(
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/providers/document_store/qdrant.py", line 217, in _query_by_filters
2025-03-25 14:45:53 qdrant_filters = convert_filters_to_qdrant(filters)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/haystack_integrations/document_stores/qdrant/filters.py", line 66, in convert_filters_to_qdrant
2025-03-25 14:45:53 current_filter = convert_filters_to_qdrant(item.get("conditions", []), is_parent_call=False) or []
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/haystack_integrations/document_stores/qdrant/filters.py", line 86, in convert_filters_to_qdrant
2025-03-25 14:45:53 raise FilterError(msg)
2025-03-25 14:45:53 haystack.errors.FilterError: 'field' or 'value' not found for 'in'
2025-03-25 14:45:53
2025-03-25 14:45:53 ********************************************************************************
2025-03-25 14:45:53 > table_retrieval [src.pipelines.retrieval.retrieval.table_retrieval()] encountered an error<
2025-03-25 14:45:53 > Node inputs:
2025-03-25 14:45:53 {'embedding': "<Task finished name='Task-1432' coro=<AsyncGraphAd...",
2025-03-25 14:45:53 'project_id': '',
2025-03-25 14:45:53 'table_retriever': '<src.providers.document_store.qdrant.AsyncQdrantEm...',
2025-03-25 14:45:53 'tables': None}
2025-03-25 14:45:53 ********************************************************************************
2025-03-25 14:45:53 Traceback (most recent call last):
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 122, in new_fn
2025-03-25 14:45:53 await fn(**fn_kwargs) if asyncio.iscoroutinefunction(fn) else fn(**fn_kwargs)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 219, in async_wrapper
2025-03-25 14:45:53 self._handle_exception(observation, e)
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 520, in _handle_exception
2025-03-25 14:45:53 raise e
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 217, in async_wrapper
2025-03-25 14:45:53 result = await func(*args, **kwargs)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/pipelines/retrieval/retrieval.py", line 160, in table_retrieval
2025-03-25 14:45:53 return await table_retriever.run(
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/providers/document_store/qdrant.py", line 359, in run
2025-03-25 14:45:53 docs = await self._document_store._query_by_filters(
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/providers/document_store/qdrant.py", line 217, in _query_by_filters
2025-03-25 14:45:53 qdrant_filters = convert_filters_to_qdrant(filters)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/haystack_integrations/document_stores/qdrant/filters.py", line 66, in convert_filters_to_qdrant
2025-03-25 14:45:53 current_filter = convert_filters_to_qdrant(item.get("conditions", []), is_parent_call=False) or []
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/haystack_integrations/document_stores/qdrant/filters.py", line 86, in convert_filters_to_qdrant
2025-03-25 14:45:53 raise FilterError(msg)
2025-03-25 14:45:53 haystack.errors.FilterError: 'field' or 'value' not found for 'in'
2025-03-25 14:45:53
2025-03-25 14:45:53 ********************************************************************************
2025-03-25 14:45:53 > table_retrieval [src.pipelines.retrieval.retrieval.table_retrieval()] encountered an error<
2025-03-25 14:45:53 > Node inputs:
2025-03-25 14:45:53 {'embedding': "<Task finished name='Task-1454' coro=<AsyncGraphAd...",
2025-03-25 14:45:53 'project_id': '',
2025-03-25 14:45:53 'table_retriever': '<src.providers.document_store.qdrant.AsyncQdrantEm...',
2025-03-25 14:45:53 'tables': None}
2025-03-25 14:45:53 ********************************************************************************
2025-03-25 14:45:53 Traceback (most recent call last):
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 122, in new_fn
2025-03-25 14:45:53 await fn(**fn_kwargs) if asyncio.iscoroutinefunction(fn) else fn(**fn_kwargs)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 219, in async_wrapper
2025-03-25 14:45:53 self._handle_exception(observation, e)
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 520, in _handle_exception
2025-03-25 14:45:53 raise e
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 217, in async_wrapper
2025-03-25 14:45:53 result = await func(*args, **kwargs)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/pipelines/retrieval/retrieval.py", line 160, in table_retrieval
2025-03-25 14:45:53 return await table_retriever.run(
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/providers/document_store/qdrant.py", line 359, in run
2025-03-25 14:45:53 docs = await self._document_store._query_by_filters(
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/src/providers/document_store/qdrant.py", line 217, in _query_by_filters
2025-03-25 14:45:53 qdrant_filters = convert_filters_to_qdrant(filters)
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/haystack_integrations/document_stores/qdrant/filters.py", line 66, in convert_filters_to_qdrant
2025-03-25 14:45:53 current_filter = convert_filters_to_qdrant(item.get("conditions", []), is_parent_call=False) or []
2025-03-25 14:45:53 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-25 14:45:53 File "/app/.venv/lib/python3.12/site-packages/haystack_integrations/document_stores/qdrant/filters.py", line 86, in convert_filters_to_qdrant
2025-03-25 14:45:53 raise FilterError(msg)
2025-03-25 14:45:53 haystack.errors.FilterError: 'field' or 'value' not found for 'in'
2025-03-25 14:45:53 -------------------------------------------------------------------
2025-03-25 14:45:53 Oh no an error! Need help with Hamilton?
2025-03-25 14:45:53 Join our slack and ask for help! https://join.slack.com/t/hamilton-opensource/shared_invite/zt-2niepkra8-DGKGf_tTYhXuJWBTXtIs4g
2025-03-25 14:45:53 -------------------------------------------------------------------
2025-03-25 14:45:53
2025-03-25 14:45:53 -------------------------------------------------------------------
2025-03-25 14:45:53 Oh no an error! Need help with Hamilton?
2025-03-25 14:45:53 Join our slack and ask for help! https://join.slack.com/t/hamilton-opensource/shared_invite/zt-2niepkra8-DGKGf_tTYhXuJWBTXtIs4g
2025-03-25 14:45:53 -------------------------------------------------------------------
2025-03-25 14:45:53
2025-03-25 14:45:53 -------------------------------------------------------------------
2025-03-25 14:45:53 Oh no an error! Need help with Hamilton?
2025-03-25 14:45:53 Join our slack and ask for help! https://join.slack.com/t/hamilton-opensource/shared_invite/zt-2niepkra8-DGKGf_tTYhXuJWBTXtIs4g
2025-03-25 14:45:53 -------------------------------------------------------------------
2025-03-25 14:45:53
2025-03-25 14:45:53 E0325 09:15:53.349 8 wren-ai-service:151] Request 10c01a2d-a7d1-47de-b35f-5ba820312f60: Error validating question: 'field' or 'value' not found for 'in'
2025-03-25 14:45:53 E0325 09:15:53.350 8 wren-ai-service:151] Request 10c01a2d-a7d1-47de-b35f-5ba820312f60: Error validating question: 'field' or 'value' not found for 'in'
2025-03-25 14:45:53 E0325 09:15:53.352 8 wren-ai-service:151] Request 10c01a2d-a7d1-47de-b35f-5ba820312f60: Error validating question: 'field' or 'value' not found for 'in'
2025-03-25 14:45:54 INFO: 172.18.0.6:57114 - "GET /v1/question-recommendations/10c01a2d-a7d1-47de-b35f-5ba820312f60 HTTP/1.1" 200 OK

[cyyeh]

@Dharmik-Solanki could you also provide Wren AI version and config.yaml? thanks

[Dharmik-Solanki]

@Dharmik-Solanki could you also provide Wren AI version and config.yaml? thanks

.env file:

COMPOSE_PROJECT_NAME=wrenai
PLATFORM=linux/amd64

PROJECT_DIR=.

service port

WREN_ENGINE_PORT=8080
WREN_ENGINE_SQL_PORT=7432
WREN_AI_SERVICE_PORT=5555
WREN_UI_PORT=3000
IBIS_SERVER_PORT=8000
WREN_UI_ENDPOINT=http://wren-ui:${WREN_UI_PORT}

ai service settings

QDRANT_HOST=qdrant
SHOULD_FORCE_DEPLOY=1

vendor keys

OPENAI_API_KEY=

version

CHANGE THIS TO THE LATEST VERSION

WREN_PRODUCT_VERSION=0.17.0
WREN_ENGINE_VERSION=0.14.8
WREN_AI_SERVICE_VERSION=0.17.1
IBIS_SERVER_VERSION=0.14.8
WREN_UI_VERSION=0.22.1
WREN_BOOTSTRAP_VERSION=0.1.5

user id (uuid v4)

USER_UUID=

for other services

POSTHOG_API_KEY=phc_nhF32aj4xHXOZb0oqr2cn4Oy9uiWzz6CCP4KZmRq9aE
POSTHOG_HOST=https://app.posthog.com
TELEMETRY_ENABLED=true

this is for telemetry to know the model, i think ai-service might be able to provide a endpoint to get the information

GENERATION_MODEL=gpt-4o-mini
LANGFUSE_SECRET_KEY=
LANGFUSE_PUBLIC_KEY=

the port exposes to the host

OPTIONAL: change the port if you have a conflict

HOST_PORT=3000
AI_SERVICE_FORWARD_PORT=5555

Wren UI

EXPERIMENTAL_ENGINE_RUST_VERSION=false

---

config file:

you should rename this file to config.yaml and put it in ~/.wrenai

please pay attention to the comments starting with # and adjust the config accordingly, 3 steps basically:

1. you need to use your own llm and embedding models

2. you need to use the correct pipe definitions based on https://raw.githubusercontent.com/canner/WrenAI/<WRENAI_VERSION_NUMBER>/docker/config.example.yaml

3. you need to fill in correct llm and embedding models in the pipe definitions

type: llm
provider: litellm_llm
models:

• api_base: http://host.docker.internal:11434 # if you are using mac/windows, don't change this; if you are using linux, please search "Run Ollama in docker container" in this page: https://docs.getwren.ai/oss/ai_service/guide/custom_llm#running-wren-ai-with-your-custom-llm-embedder
  model: ollama_chat/deepseek-r1:7b # ollama_chat/<ollama_model_name>
  alias: default
  timeout: 600
  kwargs:
  n: 1
  temperature: 0

---

type: embedder
provider: litellm_embedder
models:

• model: ollama/nomic-embed-text # put your ollama embedder model name here, ollama/<ollama_model_name>
  alias: default
  api_base: http://host.docker.internal:11434 # if you are using mac/windows, don't change this; if you are using linux, please search "Run Ollama in docker container" in this page: https://docs.getwren.ai/oss/ai_service/guide/custom_llm#running-wren-ai-with-your-custom-llm-embedder
  timeout: 600

---

type: engine
provider: wren_ui
endpoint: http://wren-ui:3000

---

type: engine
provider: wren_ibis
endpoint: http://wren-ibis:8000

---

type: document_store
provider: qdrant
location: http://qdrant:6333
embedding_model_dim: 768 # put your embedding model dimension here
timeout: 120
recreate_index: true

---

please change the llm and embedder names to the ones you want to use

the format of llm and embedder should be .<model_name> such as litellm_llm.gpt-4o-2024-08-06

the pipes may be not the latest version, please refer to the latest version: https://raw.githubusercontent.com/canner/WrenAI/<WRENAI_VERSION_NUMBER>/docker/config.example.yaml

type: pipeline
pipes:

• name: db_schema_indexing
  embedder: litellm_embedder.default
  document_store: qdrant
• name: historical_question_indexing
  embedder: litellm_embedder.default
  document_store: qdrant
• name: table_description_indexing
  embedder: litellm_embedder.default
  document_store: qdrant
• name: db_schema_retrieval
  llm: litellm_llm.default
  embedder: litellm_embedder.default
  document_store: qdrant
• name: historical_question_retrieval
  embedder: litellm_embedder.default
  document_store: qdrant
• name: sql_generation
  llm: litellm_llm.default
  engine: wren_ui
• name: sql_correction
  llm: litellm_llm.default
  engine: wren_ui
• name: followup_sql_generation
  llm: litellm_llm.default
  engine: wren_ui
• name: sql_summary
  llm: litellm_llm.default
• name: sql_answer
  llm: litellm_llm.default
  engine: wren_ui
• name: sql_breakdown
  llm: litellm_llm.default
  engine: wren_ui
• name: sql_expansion
  llm: litellm_llm.default
  engine: wren_ui
• name: semantics_description
  llm: litellm_llm.default
• name: relationship_recommendation
  llm: litellm_llm.default
  engine: wren_ui
• name: question_recommendation
  llm: litellm_llm.default
• name: question_recommendation_db_schema_retrieval
  llm: litellm_llm.default
  embedder: litellm_embedder.default
  document_store: qdrant
• name: question_recommendation_sql_generation
  llm: litellm_llm.default
  engine: wren_ui
• name: chart_generation
  llm: litellm_llm.default
• name: chart_adjustment
  llm: litellm_llm.default
• name: intent_classification
  llm: litellm_llm.default
  embedder: litellm_embedder.default
  document_store: qdrant
• name: data_assistance
  llm: litellm_llm.default
• name: sql_pairs_indexing
  document_store: qdrant
  embedder: litellm_embedder.default
• name: sql_pairs_retrieval
  document_store: qdrant
  embedder: litellm_embedder.default
  llm: litellm_llm.default
• name: preprocess_sql_data
  llm: litellm_llm.default
• name: sql_executor
  engine: wren_ui
• name: sql_question_generation
  llm: litellm_llm.default
• name: sql_generation_reasoning
  llm: litellm_llm.default
• name: followup_sql_generation_reasoning
  llm: litellm_llm.default
• name: sql_regeneration
  llm: litellm_llm.default
  engine: wren_ui
• name: instructions_indexing
  embedder: litellm_embedder.default
  document_store: qdrant
• name: instructions_retrieval
  embedder: litellm_embedder.default
  document_store: qdrant
• name: sql_functions_retrieval
  engine: wren_ibis
  document_store: qdrant
• name: project_meta_indexing
  document_store: qdrant

---

settings:
column_indexing_batch_size: 50
table_retrieval_size: 10
table_column_retrieval_size: 100
allow_using_db_schemas_without_pruning: false # if you want to use db schemas without pruning, set this to true. It will be faster
allow_intent_classification: true
allow_sql_generation_reasoning: true
query_cache_maxsize: 1000
query_cache_ttl: 3600
langfuse_host: https://cloud.langfuse.com
langfuse_enable: true
logging_level: DEBUG
development: true
historical_question_retrieval_similarity_threshold: 0.9
sql_pairs_similarity_threshold: 0.7
sql_pairs_retrieval_max_size: 10
instructions_similarity_threshold: 0.7
instructions_top_k: 10

i have pulled latest version of everything

[Dharmik-Solanki]

@Dharmik-Solanki could you also provide Wren AI version and config.yaml? thanks

@cyyeh

In latest .env file's example you put the following( EXPERIMENTAL_ENGINE_RUST_VERSION ) to false but in launcher it shows true. if this is important then please check it once .