Upgrade to lunr 2.x #1319
Upgrade to lunr 2.x #1319
Comments
There is a JSON schema describing the format of a serialised index. The repository is marked as WIP since I really want to get some feedback from implementors before committing to it. That said it is pretty stable, the only open questions I have are on how to version the schema. I've also started working on a Rust library for generating an index, it might prove useful in getting started with a Python implementation. Of course, if anyone is interested on taking on that work (lunr.py) then I'm more than happy to answer questions and offer guidance, I'm not a Python programmer though so its unlikely I'll be able to work on an implementation. |
When you implement the pre-building - please, please, please make it optional. Material builds the index in the browser in a very specific way that is incompatible with a pre-built index. |
@squidfunk when I played with this some time ago, my proof of concept pre-built the index but also provided the current JSON file. Your theme could ignore the pre-build index although given the huge improvement it makes, I can't imagine why you would want to. Seems like it would be worth it to make you theme work with the pre-built index. Of course, any such change will come with a complete refactor of the JavaScript provided by the plugin, so perhaps you will no longer even feel the need to provide your own at that point (that is one of the goals I have with the refactor). |
Well, I do some post-processing on the |
I'm not so sure we want to upgrade to Lunr 2.x. Being able to "boost" a search term is nice, but we need to always have the |
Material already uses lunr 2.x with very good results. You can take a look at the setup here: https://github.com/squidfunk/mkdocs-material/blob/master/src/assets/javascripts/components/Material/Search/Result.jsx#L172 |
@squidfunk my point is that version 2 ignores the Btw, how are you passing user settings from Python to Lunr (such as lang, ect)? Are you rendering then as JS variables in the template or some other way? I was thinking of including plugin specific settings in the JSON file. By way of explanation, the JSON currently is structured like this:
So I was thinking of adding a second key,
Of course, you could ignore |
I'm passing them as a configuration in the EDIT: ... and for everything locale/translation related, it's mostly meta tags, see, which are then read like this EDIT2: the reason why I'm doing this is that the translations for Material are done within the templates, so it's the only way to go. Some values (like |
@waylan lunr.js 2.x moved the boost to the search according to upgrading docs so theoretically in the search call we could do something like |
AFAICT, that makes things worse. If |
I've been doing some research on how to best fit our use case for boosting only the title, and looks like the Lunr 2.x API makes it slightly harder. I made an attempt using Lunr 2.x
I'm not sure if hacking the user's query string is the best approach though. Maybe I'm missing something in the Lunr API that will allows to simply boost the title in 2.x as we did previously. @olivernn @squidfunk @waylan any thoughts? |
@yeraydiazdiaz I can tell a little about my experience with Material for MkDocs has been using |
Based on my own testing and according to the docs, it has no effect. However, it certainly did in version 1.x. In #931 elasticlunr is referenced, which is a fork of lunr. According to their docs, "boosting" is done at query time, but by field, not by term, which makes more sense that lunr2.x to me. elasticlunr also appears to support lunr-languages. Given the above, and the added customizability, I'm wondering if we should switch to elasticlunr. To be clear, I have no experience with it myself. It just seems like a better fit from a read of the docs. |
* Use a web worker in the browser with a fallback (fixes #859 & closes #1396). * Optionally pre-build search index (fixes #859 & closes #1061). * Upgrade to lunr.js 2.x (fixes #1319). * Support search in languages other than English (fixes #826). * Allow the user to define the word separators (fixes #867). * Only run searches for queries of length > 2 (fixes #1127). * Remove dependency on require.js, mustache, etc. (fixes #1218). * Compress the search index (fixes #1128).
At some point recently, lunr 2.0 was released. A summary of changes are here: https://lunrjs.com/guides/upgrading.html
More importantly, there is a page on Pre-Building Indexes which includes an example script. Unfortunately, I still don't see a spec (of the index format) so that a Python lib could reliably build the index.
In any event, when search is refactored, we should use lunr 2.x.
The text was updated successfully, but these errors were encountered: