Search for terms beginning with colon (e.g. ":if")

[rhcarvalho]

Unexpected results when searching Phoenix documentation.

Cannot search for Heex template attributes like :if and :for, as that sort of input gets interpreted in some particular way by the search tokenizer. I tried wrapping in quotes, but that doesn't work either.

[josevalim]

It is understand that :if does not work, because that's the syntax used for fields, but in my opinion ":if" should work. PRs welcome.

[rhcarvalho]

Upon first look, the relevant code / limitation seems to be within the lunr dependency, in particular the QueryParser implementation.

ex_doc/assets/package.json

Line 48 in 4f5a221

"lunr": "^2.3.8",

I see ex_doc configures lunr in

ex_doc/assets/js/search-page.js

Lines 48 to 51 in 4f5a221

	lunr.QueryLexer.termSeparator = /\s+/
	lunr.Pipeline.registerFunction(elixirTokenFunction, 'elixirTokenSplitter')
	lunr.Pipeline.registerFunction(elixirTrimmerFunction, 'elixirTrimmer')
	lunr.Pipeline.registerFunction(hyphenSearchFunction, 'hyphenSearch')

---

Any suggestions or tips how to test changes to ex_doc in a feature branch? Is there an easy way to render some docs and interact with it in the browser and/or write unit tests?

[rhcarvalho]

https://lunrjs.com/guides/searching.html#fields

I was not familiar with Lunr.js. I think one path forward is to use a pre-processor to replace a colon ":" in the beginning of a word (quite common in Elixir, atom syntax) with something that is not a special character to Lunr (e.g. _COLON_), and also do a similar processing when indexing documents.

That would solve not only the :if query (same for ":if", as apparently " doesn't mean anything to Lunr), but also allow searching for atoms 😄

If I figure out how to do the above, we may also find a way to search for trailing colons, because of keyword lists. E.g. finding ":current_user" and "current_user:".

[rhcarvalho]

BTW, I learned \ can be used to escape special characters https://lunrjs.com/docs/lunr.Index.html#~QueryString.

Searching for \:if returns results, but not the most relevant results -- it is as if the colon prefix is ignored.

[josevalim]

Yeah, I was playing with this code and so i pushed a fix. :)