[BUG] Q2E generates metatext in rewritten queries

[LuizGlomyer]

Bug Description

Hello, I'm studying query rewriting and have been using QueryGym for my experiments, it is a very useful library! But I noticed an issue with the Q2E (query2e) method. I rewrote the first 5 queries from topics.msmarco-v2-passage.dev.txt using Query2E. The generated outputs consistently contain substantial metatext (e.g., "Here is a list of relevant keywords") instead of only the keywords/reformulated query.

2	"Androgen receptor define Androgen receptor define Androgen receptor define Androgen receptor define Androgen receptor define Here is a list of relevant keywords and phrases based on the query **""Androgen receptor define""** categorized by their utility for search or content organization *Core Terms** Androgen receptor (AR) Androgen receptor definition AR protein Nuclear receptor Steroid hormone receptor *Biological Function & Mechanism** Gene expression regulation DNA binding Ligand-dependent transcription factor Signal transduction Cell differentiation Cell proliferation Apoptosis regulation *Associated Hormones & Ligands** Testosterone Dihydrotestosterone (DHT) Estradiol"
524332	"treating tension headaches without medication treating tension headaches without medication treating tension headaches without medication treating tension headaches without medication treating tension headaches without medication Here is a list of relevant keywords and phrases for the query ""treating tension headaches without medication categorized by their search intent *Core Concepts & Alternatives** Non-pharmacological headache relief Natural tension headache remedies Alternative headache treatments Holistic headache management Drug-free headache solutions Lifestyle changes for tension headaches *Specific Techniques & Therapies** Progressive muscle relaxation Biofeedback for headaches Heat and cold therapy for tension headaches Acupuncture for tension headaches Massage therapy for head pain Yoga for headache relief Mindfulness meditation for headaches Cognitive behavioral therapy (CBT) for headaches Trigger point therapy Neck stretching exercises for headaches"
524447	treatment of varicose veins in legs treatment of varicose veins in legs treatment of varicose veins in legs treatment of varicose veins in legs treatment of varicose veins in legs Here is a comprehensive list of keywords and phrases related to the **treatment of varicose veins in legs** categorized by their relevance and search intent ### Core Medical Procedures Sclerotherapy Endovenous laser ablation (EVLA) Radiofrequency ablation (RFA) Vein stripping (surgical ligation and stripping) Ambulatory phlebectomy Foam sclerotherapy Cyanoacrylate glue (VenaSeal) Mechanochemical ablation (MOCA) Transilluminated powered phlebectomy (TPV) ### Symptom Relief & Conservative Management Compression stockings Compression therapy Leg elevation techniques Venous insufficiency treatment Chronic venous disease (CVD) management Leg pain relief Swelling reduction
786674	"what is prime rate in canada what is prime rate in canada what is prime rate in canada what is prime rate in canada what is prime rate in canada Here is a list of relevant keywords and phrases for the query ""what is prime rate in canada"" *Core Concepts** Prime Rate Prime Interest Rate Prime Lending Rate Bank Prime Rate *Canadian Context** Canada Prime Rate Canadian Prime Rate Bank of Canada Prime Rate Canada Prime Rate 2024 (or current year) Canadian Banking Sector *Related Financial Terms** Variable Interest Rate Mortgage Rates Canada Personal Loan Rates Credit Card Rates Bank of Canada Benchmark Rate Overnight Rate Lending Rate"
1048917	"what is operating system misconfiguration what is operating system misconfiguration what is operating system misconfiguration what is operating system misconfiguration what is operating system misconfiguration Here is a list of relevant keywords and concepts associated with the query **""what is operating system misconfiguration""** categorized for clarity ### Core Concepts Operating System (OS) Misconfiguration Security Misconfiguration Vulnerability Hardening Default Settings Security Baseline ### Common Causes & Indicators Open Ports Unnecessary Services Weak Passwords Default Credentials Exposed Admin Interfaces Unpatched Software Insecure File Permissions Debug Mode Enabled Verbose Error Messages"

Root Cause / Investigation

While investigating the issue, I found that the Q2E prompt configuration in prompt_bank.yaml did not include an appropriate system prompt in the version I was using.

I performed a quick test by adding a system prompt similar to those used by the other reformulation methods (example below), instructing the model to output only the requested keywords. After this change, the metatext was no longer generated in the rewritten queries.

- id: q2e.zs.v1
  method_family: q2e
  version: 1
  introduced_by: "Q2E Zero-Shot Prompt"
  license: "CC-BY-4.0"
  authors: ["queryGym Team"]
  tags: ["keywords"]
  template:
    system: |
      You output only comma-separated keywords. No sentences.
    user: |
      Write a list of keywords for the following query: {query}
  notes: |
    Zero-shot Q2E reformulation prompt

Steps to Reproduce

1. Install QueryGym
2. Use the Q2E (query2e) method with a local LLM through the Ollama OpenAI-compatible API.
3. Run the method on the first five queries from topics.msmarco-v2-passage.dev.txt
4. Inspect the generated rewritten queries.
5. All five queries contain metatext similar to: Here is a list of relevant keywords and phrases for the query

Expected Behavior

I expected the model output to contain only the generated keywords, for example:

prime rate, Canada prime rate, prime lending rate, Bank of Canada prime rate

Actual Behavior

The model generates a verbose response containing explanatory text and Markdown formatting, for example:

Here is a list of relevant keywords and phrases for the query

Code Example

import querygym as qg
from querygym.core.llm import OpenAICompatibleClient


class OllamaClient(OpenAICompatibleClient):
    def chat(self, *messages, **kwargs):
        kwargs["reasoning_effort"] = "none"
        return super().chat(*messages, **kwargs)


model = "qwen3.5:9b"

reformulator = qg.create_reformulator(
    "query2e",
    model=model,
    params={
        "mode": "zs",
        "max_keywords": 20,
    },
    llm_config={
        "base_url": "http://127.0.0.1:11434/v1",
        "api_key": "ollama",
        "seed": 42,
        "temperature": 0,
        "max_tokens": 256,
    },
)

reformulator.llm = OllamaClient(
    model=model,
    base_url="http://127.0.0.1:11434/v1",
    api_key="ollama",
)

query = qg.QueryItem(
    "test",
    "what is prime rate in canada",
)

result = reformulator.reformulate_batch([query])[0]

print("Original:")
print(result.original)

print("\nReformulated:")
print(result.reformulated)

Environment

OS: Manjaro Linux x86_64
Kernel: 6.6.152-1-MANJARO
Python: 3.12.3
QueryGym: querygym>=0.2.0
LLM: Qwen3.5 9B
LLM serving: Ollama
Installation method: uv