Summary
When a user searches for a punctuation-heavy code phrase, search can tokenize the query into broad FTS terms and return noisy results. The CLI should suggest --exact-substring when the query looks like a literal code fragment.
Observed
This query was initially noisy:
dotnet ./src/CodeIndex/bin/Debug/net8.0/cdidx.dll search "CommandText = $" --path src/ --json --snippet-lines 5 --limit 80
The result set included many unrelated lines because the query was tokenized around =, $, and ordinary terms. Re-running with --exact-substring produced the intended code-phrase results.
Requested behavior
For queries containing meaningful punctuation/operators, especially when broad FTS terms dominate the result set, surface a hint such as:
This looks like a literal code phrase; try --exact-substring for punctuation-sensitive matching.
This could appear in text output and JSON/MCP recovery hints.
Impact
This would make code audits faster and reduce the need for agents to remember when FTS tokenization loses punctuation semantics.
Summary
When a user searches for a punctuation-heavy code phrase,
searchcan tokenize the query into broad FTS terms and return noisy results. The CLI should suggest--exact-substringwhen the query looks like a literal code fragment.Observed
This query was initially noisy:
dotnet ./src/CodeIndex/bin/Debug/net8.0/cdidx.dll search "CommandText = $" --path src/ --json --snippet-lines 5 --limit 80The result set included many unrelated lines because the query was tokenized around
=,$, and ordinary terms. Re-running with--exact-substringproduced the intended code-phrase results.Requested behavior
For queries containing meaningful punctuation/operators, especially when broad FTS terms dominate the result set, surface a hint such as:
This looks like a literal code phrase; try --exact-substring for punctuation-sensitive matching.This could appear in text output and JSON/MCP recovery hints.
Impact
This would make code audits faster and reduce the need for agents to remember when FTS tokenization loses punctuation semantics.