switch gitbook search engine to fuse.js

[jtbayly]

Two other issues have mentioned the possibility of switching the gitbook theme to use fuse.js. This would solve language problems (Chinese not currently supported in Lunr) and speed problems (long books are unusable without pre-building the search index in Lunr).

Currently bs4_book uses fuse, and I'm interested to hear whether others would like to see the search engine replaced in the gitbook theme, especially from the maintainers.

---

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('bookdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/bookdown').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

[yihui]

Yes, we'll replace lunr.js with fuse.js. Thanks for the suggestion!

[apreshill]

I would just like to more enthusiastically upvote this issue 🤣

[Scinawa]

Thank you! :)

[cderv]

Listing some search improvement :

• matching special char (Search: does not match underscore character #735)
• only show chapters that contains the search term (Search: filtering does not remove chapters that contain search-term stem words #734)
• Show number of hits in the document, and counter when navigating through match (Search: Display the number of search term results in the document #743)
• Multibyte character support (Feature request: Search Engine for multibyte characters.  #792, Search function doesn't work bookdown-demo#12 (comment))
• Speed of search - e.g gitbook prebuild search index in plugin(Feature: Pre-build search index #975)
• prefill search box using a url query (Prefill search in bookdown e.g. https://mybook.com/?s=my_search_term #1064) and have a search page (Add a search page to show all results of a search in a single page #1115)

[cderv]

For reference another alternative: https://stork-search.net/

We need to evaluate with other projects as it could be a better candidate.

[jtbayly]

Stork Search looks fairly heavy. I left a comment about the index size here, but that is already ignoring the sizes of stork.wasm and stork.js, which together add up to about 600KB on their own.

It's fast, but I'm not sure it scales until they get the index size under control. Even then I'm leery of the extra 600KB added to the page size.

For the sake of comparison, the search engine that SphinxDoc uses doesn't search on every keystroke, just when you hit return. However, it creates an index that is less than 40KB for my book that is 1.1MB. Then when you do a search it loads the data only from the files where the result is found. In other words, the worst case scenario in terms of size is the size of the content being searched, plus the size of the index (tiny) and all the rest of the javascript on the page is 51KB, the majority of that being jquery. I threw the large book that is unusable on bookdown into sphinx-doc for a quick test. You can check it out and see the search working here.

[jtbayly]

Note that this post was linked in a previous discussion, and it discusses some of the options and tradeoffs on speed.

Here's another comparison of several engines:
https://github.com/fiatjaf/search-comparison

And here's another showing popularity, if that matters:
https://www.npmtrends.com/elasticlunr-vs-fuse.js-vs-fuzzy-vs-fuzzysearch-vs-lunr-vs-reds-vs-search-index

[cderv]

Just sharing this issue #1141 so that we don't forget that search should work in all kind of text, included indent text

[jtbayly]

Perhaps I should open another issue, but is it possible to also get Elasticsearch supported by bookdown? Am I right in understanding that it’s already supported in related projects like pkgdown?

[cderv]

I think pkgdown will use fuse.js too to have a local search. Currently, pkgdown allows to connect an online service provided by Algolia called DocSearch: https://pkgdown.r-lib.org/articles/search.html
but this require an API key, some other requirement and online service I believe.
This is indeed another topic than this change to fuse.js here.

[jtbayly]

Sorry to distract, then.

[yihui]

You can use fuse now: https://github.com/rstudio/bookdown/blob/master/NEWS.md#changes-in-bookdown-version-023

remotes::install_github('rstudio/bookdown')

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.