Add search to Rails Guides #39299

LukasSkywalker · 2020-05-15T15:35:45Z

For better discoverability and to prevent users from having to leave
the Guides webiste, this commit implements a client side search based
on the open lunr library.

Different libraries were compared:

lucaong/minisearch, where it was not possible to pre-build an index.
Building it on the fly would take about 3 seconds and render the
browser unresponsive.
krisk/Fuse, where it was possible to pre-build an index, but the
fuzzy search was very slow and would block for about 600ms.
olivernn/lunr.js, where the index cloud be pre-built and the fuzzy
search could be disabled.

The new search uses Lunr with a pre-built index and by using a lookup
table where the id of the documents are mapped to their respective url
to keep the index size small.

To build the index, the Guides HTML is parsed and split into documents
that are then indexed using ExecJS. This generates an index of about
3.3 MB in size which is only loaded when the user focuses the search
input.

The index is then stored in the browsers localstorage for up to a week
for performance.

Demo here

Summary

Other Information

References:

lebalz

Thanks for this addition, nice job!

guides/assets/javascripts/search.js

LukasSkywalker · 2020-05-19T08:25:00Z

cc @fxn

rolandasb · 2020-05-27T08:52:56Z

Nice work 👍 I'm not sure about the positioning between navigation items. Maybe it would make sense to put that in the top bar?

LukasSkywalker · 2020-05-27T14:38:00Z

@rolandasb I was thinking about that as well, but thought it was a bit of a mismatch – the content of the top bar references items outside the Guides while the red navigation bar references things contained in the Guides. Since the search only works inside the Guides, but not on rubyonrails.org, I put it in the lower bar.

But I don't have a very strong stance on this.

p8 · 2020-06-16T19:26:59Z

Is anyone else getting an endless "Please wait.."?

Edit: Ah, that is a "QuotaExceededError: The quota has been exceeded."

LukasSkywalker · 2020-06-19T11:53:36Z

@p8 It should work if you try again. The "Please wait..." indicator appears the first time you use the search since the index needs to be downloaded. Due to the size (3.2MB) it is lazily loaded.

rails-bot · 2020-09-17T11:54:27Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.

rails-bot · 2020-12-23T13:11:25Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.

jaredcwhite · 2021-03-30T15:17:56Z

What can we do to push this along? I think search would be hugely welcome addition to the Rails guides. Better documentation and onboarding seems to be top of mind for a number of Rubyists these days and this could help. @rafaelfranca @fxn

yorch · 2021-11-08T14:23:10Z

I don't really understand the lack of interest for adding search capabilities from the maintainers. For those of us not extremely familiar with the guides, search is a huge help, and as other have commented before, many other projects have embraced adding searching capabilities to their docs.

fxn · 2021-11-08T15:12:39Z

/cc @zzak

p8 · 2022-03-02T17:44:55Z

guides/rails_guides/search_index.rb

+        link_map_js = "var linkMap = #{link_map.to_json};"
+        File.write("#{@output_dir}/javascripts/lunr-documents.js", documents_js + link_map_js)
+
+        lunr = open("https://unpkg.com/lunr@2.3.8/lunr.js").read


I think it would be simpler to just 'vendorize' the lunr source, instead of downloading it everytime.

Austio · 2022-07-20T05:40:04Z

Bump :) seems like a really great addition

Schwad

A good hunk of this leans on vanilla JS which isn't my forte so will defer to others for their opinions on how it's put together 😅

But from a product feel this seems like a big win to me. would be a nifty addition for Rails 7.1!

As I haven't generated the guides to tophat a change in a while can someone link me the quick TLDR on tophatting this locally? Thanks!

When going through the demo - I think one of the guides strengths already is to navigate via headers.

But what I could really use is searching by text - e.g. form_with returns nothing. How hard would it be to search text body so I can search code snippets to find the right guide?

Schwad · 2022-10-05T10:35:32Z

Also 10/10 for giving us a working demo!

skipkayhil · 2022-10-05T12:41:52Z

As I haven't generated the guides to tophat a change in a while can someone link me the quick TLDR on tophatting this locally? Thanks!

https://guides.rubyonrails.org/ruby_on_rails_guides_guidelines.html#html-guides-generation
Under /guides run $ bundle exec rake guides:generate which puts them in /guides/output. To serve them you can use any static file server, I like this ruby one liner (in the output folder): ruby -run -ehttpd . -p8000 -b0.0.0.0

For better discoverability and to prevent users from having to leave the Guides webiste, this commit implements a client side search based on the open lunr library. Different libraries were compared: * lucaong/minisearch, where it was not possible to pre-build an index. Building it on the fly would take about 3 seconds and render the browser unresponsive. * krisk/Fuse, where it was possible to pre-build an index, but the fuzzy search was very slow and would block for about 600ms. * olivernn/lunr.js, where the index cloud be pre-built and the fuzzy search could be disabled. The new search uses Lunr with a pre-built index and by using a lookup table where the id of the documents are mapped to their respective url to keep the index size small. To build the index, the Guides HTML is parsed and split into documents that are then indexed using ExecJS. This generates an index of about 4.8 MB in size which is only loaded when the user focuses the search input. The index is then stored in the browsers localstorage for up to a week for performance.

LukasSkywalker · 2022-10-06T07:14:50Z

@Schwad I added the code blocks to the index in the above demo. You need to invalidate the cache for the new index to load by typing localStorage.clear() into the browser console.

Schwad · 2022-10-06T11:03:24Z

@LukasSkywalker nice! 🙌

p8 · 2022-10-06T13:52:07Z

For consistency I think it would be nice if it used the same styles as the index dropdown.
Something like:

We could probably also remove the "Contribute" item from the menu as it's already present in the top menu.
And swap the index with the search input, to make it a bit easier to find. Maybe even add an inner shadow to make it look more like an input.
I also think it can be a bit smaller as most search terms will probably fit easily.

JohnAnon9771 · 2023-03-01T04:04:04Z

There is a version of the guide for Brazilian Portuguese where this search is already implemented, maybe it will be a good reference.

https://guiarails.com.br/

akirataguchi115 · 2023-10-10T09:17:44Z

One idea that hasn't been mentioned here yet is the way Ruby on Rails API implements search functionality: https://api.rubyonrails.org/.

Right now I'll have to use the Google web search operators f.ex. site:guides.rubyonrails.org destroy.

akirataguchi115 · 2023-10-10T10:09:58Z

There is a version of the guide for Brazilian Portuguese where this search is already implemented, maybe it will be a good reference.

https://guiarails.com.br/

This is great and I would definitely implement this and make a pull request to this repository. The translation however is not forked and (thus) not licensed, so the contributors own the copyright to the search implementation.

rails-bot bot added the docs label May 15, 2020

LukasSkywalker force-pushed the guide-search branch 2 times, most recently from 60a706c to 3f1178e Compare May 17, 2020 12:28

lebalz suggested changes May 17, 2020

View reviewed changes

guides/assets/javascripts/search.js Outdated Show resolved Hide resolved

guides/assets/javascripts/search.js Show resolved Hide resolved

guides/assets/javascripts/search.js Outdated Show resolved Hide resolved

guides/assets/javascripts/search.js Outdated Show resolved Hide resolved

LukasSkywalker force-pushed the guide-search branch 2 times, most recently from 1b734f6 to 60c90c7 Compare May 18, 2020 06:58

rails-bot bot added actionmailer actionpack actiontext actionview activemodel activerecord labels May 18, 2020

LukasSkywalker force-pushed the guide-search branch from 60c90c7 to aec62a4 Compare May 18, 2020 07:00

eugeneius removed actionmailer actionpack actiontext actionview activemodel activerecord labels May 19, 2020

composerinteralia mentioned this pull request Aug 26, 2020

[POC] Search in rails guides #39252

Closed

rails-bot bot added the stale label Sep 17, 2020

LukasSkywalker mentioned this pull request Sep 23, 2020

Add Search to guides.rubyonrails.org website #36218

Closed

rails-bot bot closed this Sep 24, 2020

fxn reopened this Sep 24, 2020

rails-bot bot removed the stale label Sep 24, 2020

rails-bot bot added the stale label Dec 23, 2020

rafaelfranca added the pinned label Dec 30, 2020

rails-bot bot removed the stale label Dec 30, 2020

Base automatically changed from master to main January 14, 2021 17:01

LukasSkywalker force-pushed the guide-search branch 2 times, most recently from 49ffdc5 to 33ef886 Compare November 8, 2021 16:42

p8 reviewed Mar 2, 2022

View reviewed changes

Schwad reviewed Oct 5, 2022

View reviewed changes

LukasSkywalker force-pushed the guide-search branch 2 times, most recently from 9e0bded to 180374d Compare October 6, 2022 06:20

LukasSkywalker force-pushed the guide-search branch from 180374d to 4e4e025 Compare October 6, 2022 06:30

vipulnsward mentioned this pull request Oct 10, 2023

Rails Guides search functionality #49561

Closed

Add search to Rails Guides #39299

Are you sure you want to change the base?

Add search to Rails Guides #39299

Uh oh!

Conversation

LukasSkywalker commented May 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Other Information

Uh oh!

lebalz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LukasSkywalker commented May 19, 2020

Uh oh!

rolandasb commented May 27, 2020

Uh oh!

LukasSkywalker commented May 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

p8 commented Jun 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LukasSkywalker commented Jun 19, 2020

Uh oh!

rails-bot bot commented Sep 17, 2020

Uh oh!

rails-bot bot commented Dec 23, 2020

Uh oh!

jaredcwhite commented Mar 30, 2021

Uh oh!

yorch commented Nov 8, 2021

Uh oh!

fxn commented Nov 8, 2021

Uh oh!

p8 Mar 2, 2022

Choose a reason for hiding this comment

Uh oh!

Austio commented Jul 20, 2022

Uh oh!

Schwad left a comment

Choose a reason for hiding this comment

Uh oh!

Schwad commented Oct 5, 2022

Uh oh!

skipkayhil commented Oct 5, 2022

Uh oh!

LukasSkywalker commented Oct 6, 2022

Uh oh!

Schwad commented Oct 6, 2022

Uh oh!

p8 commented Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JohnAnon9771 commented Mar 1, 2023

Uh oh!

akirataguchi115 commented Oct 10, 2023

Uh oh!

akirataguchi115 commented Oct 10, 2023

Uh oh!

Uh oh!

LukasSkywalker commented May 15, 2020 •

edited

Loading

LukasSkywalker commented May 27, 2020 •

edited

Loading

p8 commented Jun 16, 2020 •

edited

Loading

p8 commented Oct 6, 2022 •

edited

Loading