Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev Docs: New Glossary & JS Search Box #793

Merged
merged 5 commits into from Apr 10, 2015

Conversation

@harding
Copy link
Contributor

harding commented Mar 19, 2015

This PR adds a new glossary page and a fully Javascript (no server-side scripting) search box that searches keywords for glossary entries, RPCs, BIPs, op codes, and P2P protocol messages.

Developer glossary main page

2015-03-19-162757_788x364_scrot

Preview: http://dg0.dtrt.org/en/developer-glossary

The glossary main page provides a search box and a sorted list of terms. It uses a floating list so that it looks good on both mobiles and desktop/laptop screens.

A link is provided at the bottom of the page to allow readers to recommend new glossary terms. (It works like the Submit New Event link on the Events page.)

Glossary entries

2015-03-19-162928_470x532_scrot

Preview for "block": http://dg0.dtrt.org/en/glossary/block

Each glossary entry has its own page providing a short definition (limited to 255 characters), one or more synonyms, zero or more "not to be confused with" terms, and zero or more links to resources on and off Bitcoin.org. Separate pages were used to allow search engines to link directly to the most relevant entry, rather than forcing users to scroll through a giant page of definitions.

This pull adds:

  • 89 glossary entry pages
  • 51 additional synonyms
  • 222 links (all internal to Bitcoin.org or to Bitcoin Wiki, BitcoinTalk, or Bitcoin StackExchange)

Each page includes links to edit it, view its history, or report an issue about it. Note: those links won't work right until this is merged into master because they're hardcoded for master.

Dev search box

2015-03-19-163205_1280x800_scrot

Preview: http://dg0.dtrt.org/en/developer-documentation

A Javascript-powered search box is added to the following pages: Dev Guide, Dev Reference, Dev Examples, Dev glossary, and individual glossary entry pages.

The box allows finding the following by keyword, listed in the order they appear in the search:

  • Glossary entries (Bitcoin.org)
  • RPCs (Bitcoin.org)
  • Op codes (Bitcoin Wiki)
  • BIPs (just notable and non-withdrawn BIPs; GitHub.com BIPs repo)
  • Bitcoin P2P protocol messages (Bitcoin.org)

The search does use two largish Javascript libraries: JQuery and JQuery UI, both MIT-licensed. I used the plain minified files from upstream for anyone who wants to verify the shasums. This PR adds the libraries to the site, so there's no stats leakage when people load them.

Only the pages that include the search box load the JS libraries and the search data, so other pages on the site are unaffected.

I've tested the search on several Linux desktop browsers and my ancient Android 2.3.5 phone. It works well in all cases.

Backend: autocrossref and links

All glossary pages are built from data files in the _data/glossary/en directory using the new glossary.rb plugin. In addition to rendering the pages, this plugin also generates autocrossreference links and Markdown reference links for the terms.

This was part of the motivation for this pull, as discussed on the mailing list: https://groups.google.com/forum/#!topic/bitcoin-documentation/s_mL1syxYoQ

In the documenation, autoxref links that used to point to other parts of the docs now point to the glossary pages. Readers who want to learn more can follow links from the glossary page back to the docs or on to the other Bitcoin Wiki/BitcoinTalk/Bitcoin StackExchange links.

Also on the backend, this pull passes all previous tests as well as a few new ones.

Commits

I broke this PR into several commits to make review easier. It may be more convenient to review the separate commits rather than the whole diff at once.

harding added 4 commits Mar 3, 2015
This commit adds the code necessary to generate a new
en/developre-glossary page with entry pages in the en/glossary/
directory, e.g. en/glossary/51-percent-attack.

The glossary page and the individual term pages feature a JavaScript
search engine (no CGI) for just glossary terms.  This search box has
also been added to the following pages:

    * Developer Documentation (the index page)
    * Developer Guide
    * Developer Reference
    * Developer Examples

The search box requires the following MIT-licensed libraries:

    * JQuery
    * JQuery UI
    * JQuery UI CSS stylesheet

These allow our JS code to be almost trivially simple in js/devsearch.js

This commit adds only code.  Actual glossary entry data will be added in
a subsequent commit.
This adds the data used to populate the glossary and search terms.

Closes #593
@harding harding added the Dev Docs label Apr 5, 2015
@saivann

This comment has been minimized.

Copy link
Contributor

saivann commented Apr 9, 2015

@harding That is a huge change, thank you so much for your work on this!

I have sent you a pull request with various small fixes and suggestions. I didn't get to check everything in depth, but I have quickly reviewed everything and this LGTM!

I wonder if at some point we could somehow standardize all of this configuration into something more consistent, for instance with a single formatting. As I understand it, data from _data/glossary/en/* currently sends data to _config.yml, _autocrossref.yaml, _includes/references.yml (it's actually used side by side). Although I think it's OK (and jekyll often gives us no choice), the current setup looks a bit intertwined to me. Just in case you'd enjoy spending time on simplifying / standardizing this (if that is even possible).

Again, this is awesome :) Sorry for the time it took for me to review your pull request.

@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Apr 9, 2015

@saivann thanks for your review! It was worth the wait :-)

I was also hoping when I started working on this that we could completely get rid of _autocrossref.yml and _includes/references.md, but I couldn't see any reasonably way to put everything from them into the glossary. However, I was thinking that now that we have the glossary entries out of those two files, we can probably merge them into a single file (or part of _config.yml) that basically looks like:

term 1: http://example.com/foo
term 1 alternative name: foo
term 2: /en/developer-documenation

That's possible because we don't need the title text for the links anymore.

And then we'd have a very small plugin that puts them in the format expected by the autocrossref plugin and the Kramdown reference link parser.

However, that sounds like a separate PR to me.

@saivann

This comment has been minimized.

Copy link
Contributor

saivann commented Apr 9, 2015

@harding Indeed, something like that sounds like an interesting plan! (but sure, preferably at some point in the future to not block this pull request).

Add link to the glossary from each definition
Fix missing BIP37 link
Fix searchbox width on mobiles
Apply more consistent margin and padding for the searchbox
Use H3 titles in the glossary_entry layout for better readability
Always use HTML classes for CSS stylesheets
Fix one typo in glossary.rb
@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Apr 9, 2015

Added suggested fixes/improvements by @saivann.

In the absence of critical feedback, I'll merge this around 20:00 UTC Friday.

@jlopp

This comment has been minimized.

Copy link
Contributor

jlopp commented Apr 10, 2015

Should searching for "satoshi" or "denominations" bring up glossary results? It looks like there are definitions for the terms but when I search for them on your test site I don't get any results.

@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Apr 10, 2015

@jlopp a few of the common units are auto-cross-referenced to their definitions in the Payment Processing section, as they were before this PR. It would make sense to have a denominations Glossary entry that links to that table as well as to the Wiki's Unit page. I'll make a separate PR for that after this gets merged so it can be properly reviewed. Thanks!

@harding harding merged commit a790f35 into bitcoin-dot-org:master Apr 10, 2015
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
harding added a commit that referenced this pull request Apr 10, 2015
harding added a commit that referenced this pull request Apr 10, 2015
This reverts commit e3dcf0c, reversing
changes made to c71e9fd.

Once again we had a broken new plugin that Travis CI and local building
didn't catch.
@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Apr 10, 2015

Note this was merged and then reverted as we had the same problem with a new plugin as was documented in #773, although because the search box is so prominent, I decided it would be better to revert than to try to fix it live. (At least I was ready for it this time; I wasn't fooled by Travis CI passing.)

I already told @saivann I'll work on fixing the build system so it corresponds with Travis CI, so I'll do that before attempting to merge again.

@saivann

This comment has been minimized.

Copy link
Contributor

saivann commented Apr 10, 2015

@harding Thanks for keeping a close eye on what happens when merging changes to the plugins. Do you know if it is still caused by inconsistent relative path or something else? The glossary.rb plugin seemed to load files exactly the same way as releases.rb so I didn't expect it would fail.

@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Apr 10, 2015

@saivann it was really weird, and I have no idea why it happened. (If it wasn't close to my usual bedtime, I'd look into more.) In case you didn't see, what happened was that if you searched for "locktime" it would direct you to http:///locktime/ and the same for any other term.

I never actually got to a glossary page to see if they built correctly. I should've done that while the reverted tree was building, but I didn't think that far ahead.

I confirmed that my local bulid still works as expected here on top of master, so it's almost certainly another inconsistency between the bundle-based builds and the Ubuntu native Ruby libs build.

harding added a commit that referenced this pull request Apr 14, 2015
…ox""

This reverts commit 961d6c9.
@harding

This comment has been minimized.

Copy link
Contributor Author

harding commented Apr 24, 2015

@saivann I spent several minutes today thinking about the idea I proposed in this comment and I don't think it's as easy or as correct as I initially thought. Unless you have any ideas, I'm going to just leave our link definitions spread over _autocrossref.yaml, _includes/references.md, and _data/glossary/en/* for now and see if something better occurs to me in the future.

@jlopp I'll be starting on the denominations glossary entry after lunch and should have a PR for it up later today. Sorry it took so long to get around to it.

@saivann

This comment has been minimized.

Copy link
Contributor

saivann commented Apr 24, 2015

@harding Sure!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.