Feature: Pre-build search index

[jtbayly]

In resources/gitbook/js/plugin-search.js, there is the following comment:

        // [Yihui] In bookdown, I use a character matrix to store the chapter
        // content, and the index is dynamically built on the client side.
        // Gitbook prebuilds the index data instead: https://github.com/GitbookIO/plugin-search
        // We can certainly do that via R packages V8 and jsonlite, but let's
        // see how slow it really is before improving it.

I'm wondering if it is possible to add the ability (or even make it default) to pre-build the search index as mentioned above. I'm dealing with some books that are quite long, and every single page-load has a multi-second delay after load where the page is non-responsive as the search index gets rebuilt. Not surprisingly, it seems that processor gets slammed in the process, too.

---

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('bookdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/bookdown').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

[jtbayly]

The lunr page on prebuilding indexes says the following:

For large numbers of documents, it can take time for Lunr to build an index. The time taken to build the index can lead a browser to block; making your site seem unresponsive.

In my case, I'm seeing 6-8 second delays before a page is usable on a fast computer. When using search, this makes it appear as though search is broken, since nothing happens after clicking enter to go to the next result. Users give up before the page can scroll to the result.

[hadley]

@jtbayly rather than prebuild the index, I think it would be better to switch to fuse.js — I've done that in my experimental bs4_book() format, and I don't see any noticeable delay when searching mastering shiny. You can get some sense of the performance difference in https://wiki.gpii.net/w/Technology_Evaluation_-_Static_Site_Search#Speed_Comparison.

[jtbayly]

Is the bs4_book() format available somewhere for me to try it out?

[hadley]

It's experimental, so no, it's not usable by others yet.

[jtbayly]

I'd be very curious to see the code for how you integrated fuse.js.

For that matter, I'd love to work on a replacement for the gitbook theme, since it doesn't work well on mobile. And I don't think any of the other formats have search built in, do they?

[hadley]

You can see it in action at https://mastering-shiny.org. It's still very much a work in progress, but I think search is reasonably solid, and I've spent a decent amount of time making sure it works well on mobile.

[jtbayly]

That seems fast, and I'm really liking the design.

One criticism related to search: I think it makes a lot more sense to display a search results page than to just do a hover menu (or whatever that's called) with a few results. As it is, am I right in saying that you can only ever see 6 search results? I couldn't figure out how to go past the first 6 anyway.

[hadley]

The six was just a bug; should be fixed momentarily. I don't think having a separate search page is that useful, especially given that I can't currently show any context for the search results. I'd maybe consider it if I could get that data out of fuse.

[jtbayly]

Ah. I see. Working now. Yeah, if you can't provide additional context, then I guess there's not much point.

[jtbayly]

Thanks for all your work on this! Super exciting!

[jtbayly]

So, should this issue be retitled to "Switch to Fuse.js for search"?

Edit, or Close it and start a new issue, I suppose.

[hadley]

No, I think it can just be closed because the new bs4_book() theme will just use fuse.js. I doubt it is worth the effort to rewrite the gitbook theme to use fuse.js (but I'm sure a PR would be considered).

[jtbayly]

So is the gitbook theme being deprecated for bs4_book?

I mean, I’ve got an awful lot invested in gitbook, and it has quite a bit of functionality that is not available (yet?) in bs4_book.