Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize search by building index locally. #1061

Closed
wants to merge 1 commit into from

Conversation

@waylan
Copy link
Member

@waylan waylan commented Oct 3, 2016

This builds an actual lunr.js search index via Node.js in addition to a JSON
array of page objects previously used to build the index client side. Note that
both are needed as the array of page objects is used to display search
results.

If Node fails (not installed or errors out), then fallback to old behavior
by creating an empty index. If the client (browser) receives an empty index,
then it builds the index from the array of pages as previously, which is
fine for most (small) sites. Large sites will want to make sure Node is
installed to avoid the browser from hanging during index creation.

Partially addresses #859.

@waylan
Copy link
Member Author

@waylan waylan commented Oct 3, 2016

I've decided that there is no reason to not commit this part of the solution outlined in #859 (see this comment). The rest can come when we do the refactor later (webworkers will require a refactor anyway, so we might as well do that with the move to a plugin). For most users (who won't have Node.js installed) nothing will change. But for those who have Node.js installed (and/or need this feature), they can start using/testing it now and it can even be better in the next release.

This builds an actual lunr.js search index via Node.js in addition to a JSON
array of page objects previously used to build the index client side. Note that
both are needed as the array of page objects is used to display search
results.

If Node fails (not installed or errors out), then fallback to old behavior
by creating an empty index. If the client (browser) receives an empty index,
then it builds the index from the array of pages as previously, which is
fine for most (small) sites. Large sites will want to make sure Node is
installed to avoid the browser from hanging during index creation.

Partially addresses #859.
@waylan waylan force-pushed the waylan:search-index branch from 5e5dda8 to 0f5a187 Oct 3, 2016
@waylan
Copy link
Member Author

@waylan waylan commented Oct 3, 2016

Oh, I forgot PyExecJS doesn't work on Python 2.6. Unless we want to drop Py26 now, this will have to wait till the next release.

Regardless, this PR packages the feature up in a nice diff for users to manually merge if they desire. Or you can always just install MkDocs from this branch:

pip install https://github.com/waylan/mkdocs/archive/search-index.zip

Any feedback is always welcome.

@waylan waylan added this to the 1.0.0 milestone Oct 3, 2016
@waylan waylan self-assigned this Oct 3, 2016
@waylan waylan added this to To Do in Refactor search. May 2, 2017
@micahhausler
Copy link

@micahhausler micahhausler commented Sep 14, 2017

Any update on this? This affects us and we'd love to see faster search

@waylan
Copy link
Member Author

@waylan waylan commented Sep 14, 2017

Work on search was delayed until the Plugin API was put in place. The Plugin API is now available in the 1.0.dev branch and the existing search feature has been broken out into a plugin . We intend to refactor search in that (or a new) plugin in the near future. Of course, as it is now a plugin, third parties could create their own plugin which offers a different set of defaults. The 1.0.dev branch will be available in the next major release; however feedback on the new API now would help move move development along.

@waylan waylan added the Plugin label Nov 1, 2017
@waylan waylan moved this from To Do to In Progress in Refactor search. Feb 22, 2018
@waylan waylan mentioned this pull request Feb 27, 2018
@waylan
Copy link
Member Author

@waylan waylan commented Feb 28, 2018

This is being closed in favor of the simpler solution in #1418. That is, the prebuilt index solution is simpler. In other respects #1418 is a complete refactor of search so we also get web worker support etc.

@waylan waylan closed this Feb 28, 2018
Refactor search. automation moved this from In Progress to Done Feb 28, 2018
waylan added a commit that referenced this pull request Mar 6, 2018
* Use a web worker in the browser with a fallback (fixes #859 & closes #1396).
* Optionally pre-build search index (fixes #859 & closes #1061).
* Upgrade to lunr.js 2.x (fixes #1319).
* Support search in languages other than English (fixes #826).
* Allow the user to define the word separators (fixes #867).
* Only run searches for queries of length > 2 (fixes #1127).
* Remove dependency on require.js, mustache, etc. (fixes #1218).
* Compress the search index (fixes #1128).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.