Evidence
Running:
cdidx search --exact-substring "CommandText = $" --path src/ --json --snippet-lines 2 --limit 8
returns rows where the literal phrase is found, but highlights.terms includes both the literal and tokenized fragments:
["CommandText = $", "CommandText", "=", "$"]
The term_occurrences array then highlights the individual fragments as well as the exact phrase.
Impact
Clients that render highlights or summarize matches from terms cannot tell that this was intended as a literal-only match. For punctuation-heavy audits, the extra =, $, or identifier fragments add visual noise.
Requested behavior
For --exact-substring, expose a literal-match highlight mode or a separate literal_terms field so consumers can render only the exact substring while still preserving any diagnostic tokenization data if needed.
Evidence
Running:
cdidx search --exact-substring "CommandText = $" --path src/ --json --snippet-lines 2 --limit 8returns rows where the literal phrase is found, but
highlights.termsincludes both the literal and tokenized fragments:The
term_occurrencesarray then highlights the individual fragments as well as the exact phrase.Impact
Clients that render highlights or summarize matches from
termscannot tell that this was intended as a literal-only match. For punctuation-heavy audits, the extra=,$, or identifier fragments add visual noise.Requested behavior
For
--exact-substring, expose a literal-match highlight mode or a separateliteral_termsfield so consumers can render only the exact substring while still preserving any diagnostic tokenization data if needed.