Replace tipue search with lunr.js and lunr.py

[codecachet]

I am distinctly getting the impression that tipue search is no longer being maintained, or is otherwise less than desirable. I suggest that we replace tipue search with lunr - lunr.py to create the index, and lunr.js to query the index when a search is launched. Both projects seem alive and well, and both Jekyll and Hugo seem to recommend using lunr for search.

Just a thought - like others, I have been very frustrated trying to get search to work for Elegant. One thing about tipue is I am not crazy about the jQuery dependency. I know jQuery is a dependency of Elegant currently, but if we get rid of Bootstrap, we could get rid of jQuery.

[talha131]

I too want to get rid of jquery. It's on our roadmap.

As for tipue. When elegant was first released, tipue was the only available search solution for static sites. Now we have much better and powerful options like

1. https://fusejs.io/
2. https://github.com/fergiemcdowall/search-index
3. https://lunrjs.com/
4. https://github.com/nextapps-de/flexsearch

Down the line, we should replace or add support for other search solutions too.

[iranzo]

@codecachet do you want to create a PR for this?

[talha131]

@iranzo this is a big change. It should be handled by the core team. More importantly, we need to compare different solutions first.

[talha131]

An issue noticed with tipue is that it treats plural differently from singular

Search for "variable" and then "variables" in the docs site. Result is different.

We can use stem words option to fix it but then how many words are we going to add in the list. Tipue should have this feature by default.

[iranzo]

I'm looking at lunr.js for other aspects, seems that the data it uses for the search can be contained/generated on publish time without plugin, just a jinja loop (at least on jekyll). I'll provide some information once I finish my investigation.

[talha131]

@iranzo I appreciate that you have taken up this task. Before you go ahead and look deeply into lunr.js, can you please step back and decide which library we should use.

In terms of Github stars, Fuse is way ahead of lunr.js, which is followed by flex search.

Github stars are not a reliable metric but they do give a pretty good idea about the community interest and support, which we are going to need.

Then there is performance factor. Flex claims that it is the most performant.

Finally, we have to consider the ease of use for the end user.

Can you please investigate these three libraries and let me know how they fare on the criteria I mentioned?

[iranzo]

Hi, No worries, just wanted to mention that I was testing it as a website I'm using has it integrated (but it was pretty old version and using jekyll) about features. Lunr is not that bad, but my concern here is that it, as with tipue, pregenerates a javascript array with all website content... that means that for a big website, it will have to preload all information in the html generated and then perform the search.

I don't mind looking at flex directly to see how it performs, but probably doing perf test will require lot of articles and I was thinking that some one here in the team had a busy one that could be used for testing if the website source is available somewhere.

[talha131]

This is the result of flex search benchmark that I got.

Flex is much more performant than other libraries. How well it is supported and how actively it is under development is to be investigated.

[talha131]

that means that for a big website, it will have to preload all information in the html generated and then perform the search.

You hit the nail right on the head.

If we can avoid it using these new libraries then it would become a high priority feature to implement and ship.

[talha131]

Flex search is fast but development has seem to have lost momentum.

1. https://github.com/nextapps-de/flexsearch/graphs/commit-activity
2. Status of the project? nextapps-de/flexsearch#150

Also, the documentation is not friendly. We will have a hard time making a head and tail of it.