Permalink
Browse files

Fix syntax highlighting

Before we switched to kramdown, our syntax highlighting was handled by
a combination of pygments and highlight.js. Basically, pygments output
some esoteric classes throughout each code block, and it also output
the language of the code block (from markdown) into the html.
Highlight.js then read this language indicator, used it to do its own
parsing, and added a different set of classes to each code block. Then,
our css mostly used the highlight.js class names to do the styling, but
we occasionally took advantage of the pygments ones too to make the
styling more precise than just highlight’s classes would have allowed.

Then, with the switch to kramdown, we had to switch our syntax
highlighter from pygments to rouge. Rouge output the same class names
as pygments but, crucially, it didn’t pass through the language name of
each code block to the HTML. That meant that highlight.js had to detect
each automatically language, rather than reading the language code.
That wouldn’t have been a problem except that many of our json code
blocks aren’t strictly json (e.g. they contain comments) and we’d been
using the language code in the HTML to force highlight.js to parse them
as JSON anyway. With that language code gone, highlight.js was failing
to identify the language and so wasn’t highlighting the code at all.

We could’ve tried a few things:

1. Sending a PR to rouge to output the language tag. Even if this PR
were accepted, though, the updated rouge dependency wouldn’t have been
adopted by github pages until after redcarpets support was turned off,
so our highlighting would’ve been broken until then.

2. Pass through the language code some other way. E.g., write `<pre>`
tags manually in our markdown with the relevant class name. This
could’ve worked, but it would have made authoring the markdown messier,
and harder for others to contribute, so I didn’t want to do that.

3. Get highlight.js to auto detect and parse our malformed json
correctly, so that the language codes are unnecessary.

I went with option 3, replacing highlight’s built-in json language
definition with a new definition called “jsonlike” which works on our
site. Then, this commit includes the raw, edited highlight source files
and an updated build. Note: this build uses highlight 8.9.1, because
version 9 contained some backwards-incompatible class name changes.

Lastly, this commit includes some css tweaks to match rouge’s small
changes in the HTML wrapping highlighted code blocks, and our new
highlighting strategy more accurately categorizing comments.
1 parent 12931a4 commit de80a0408f32d7e691678be95ef75f75b62d5b75 @ethanresnick ethanresnick committed May 1, 2016
Showing 407 changed files with 21,206 additions and 28 deletions.
@@ -518,7 +518,7 @@ The following `self` link is simply a URL:
```json
"links": {
- "self": "http://example.com/posts",
+ "self": "http://example.com/posts"
}
```
@@ -0,0 +1,8 @@
+[*]
+end_of_line = lf
+insert_final_newline = true
+
+[*.{js,css}]
+charset = utf-8
+indent_style = space
+indent_size = 2
@@ -0,0 +1,8 @@
+.DS_Store
+build
+docs/_build
+.idea/
+__pycache__
+*.swp
+node_modules
+.project
@@ -0,0 +1,8 @@
+language: node_js
+node_js:
+ - "0.12"
+script:
+ - "node tools/build.js -t node"
+ - "npm test"
+
+sudo: false # Use container-based architecture
@@ -0,0 +1,192 @@
+Syntax highlighting with language autodetection.
+
+URL: https://highlightjs.org/
+
+Core developers (in order of appearance):
+
+- Ivan Sagalaev (original author) <maniac@softwaremaniacs.org>
+- Jeremy Hull <sourdrums@gmail.com>
+- Oleg Efimov <efimovov@gmail.com>
+
+Contributors:
+
+- Peter Leonov <gojpeg@gmail.com>
+- Victor Karamzin <Victor.Karamzin@enterra-inc.com>
+- Vsevolod Solovyov <vsevolod.solovyov@gmail.com>
+- Anton Kovalyov <anton@kovalyov.net>
+- Nikita Ledyaev <lenikita@yandex.ru>
+- Konstantin Evdokimenko <qewerty@gmail.com>
+- Dmitri Roudakov <dmitri@roudakov.ru>
+- Yuri Ivanov <ivanov@supersoft.ru>
+- Vladimir Ermakov <vooon341@mail.ru>
+- Vladimir Gubarkov <xonixx@gmail.com>
+- Brian Beck <exogen@gmail.com>
+- MajestiC <majestic2k@gmail.com>
+- Vasily Polovnyov <vast@whiteants.net>
+- Vladimir Epifanov <voldmar@voldmar.ru>
+- Alexander Makarov <sam@rmcreative.ru>
+- Vah <vahtenberg@gmail.com>
+- Shuen-Huei Guan <drake.guan@gmail.com>
+- Jason Diamond <jason@diamond.name>
+- Michal Gabrukiewicz <mgabru@gmail.com>
+- Ruslan Keba <rukeba@gmail.com>
+- Sergey Baranov <segyrn@yandex.ru>
+- Zaripov Yura <yur4ik7@ukr.net>
+- Oleg Volchkov <oleg@volchkov.net>
+- Vasily Mikhailitchenko <vaskas@programica.ru>
+- Jan Berkel <jan.berkel@gmail.com>
+- Vladimir Moskva <vladmos@gmail.com>
+- Loren Segal <lsegal@soen.ca>
+- Andrew Fedorov <dmmdrs@mail.ru>
+- Igor Kalnitsky <igor@kalnitsky.org>
+- Valerii Hiora <valerii.hiora@gmail.com>
+- Nikolay Zakharov <nikolay.desh@gmail.com>
+- Dmitry Kovega <arhibot@gmail.com>
+- Sergey Ignatov <sergey@ignatov.spb.su>
+- Antono Vasiljev <self@antono.info>
+- Stephan Kountso <steplg@gmail.com>
+- pumbur <pumbur@pumbur.net>
+- John Crepezzi <john.crepezzi@gmail.com>
+- Andrey Vlasovskikh <andrey.vlasovskikh@gmail.com>
+- Alexander Myadzel <myadzel@gmail.com>
+- Evgeny Stepanischev <imbolk@gmail.com>
+- Dmytrii Nagirniak <dnagir@gmail.com>
+- Luigi Maselli <grigio.org@gmail.com>
+- Denis Bardadym <bardadymchik@gmail.com>
+- Aahan Krish <geekpanth3r@gmail.com>
+- Ilya Baryshev <baryshev@gmail.com>
+- Aleksandar Ruzicic <aleksandar@ruzicic.info>
+- Joe Cheng <joe@rstudio.org>
+- Angel G. Olloqui <angelgarcia.mail@gmail.com>
+- Jason Tate <adminz@web-cms-designs.com>
+- Sergey Tikhomirov <me@stikhomirov.com>
+- Marc Fornos <marc.fornos@gmail.com>
+- Yoshihide Jimbo <yjimbo@gmail.com>
+- Casey Duncan <casey.duncan@gmail.com>
+- Eugene Nizhibitsky <nizhibitsky@gmail.com>
+- Alberto Gimeno <gimenete@gmail.com>
+- Kirk Kimmel <kimmel.k.programmer@gmail.com>
+- Nathan Grigg <nathan@nathanamy.org>
+- Dr. Drang <drdrang@gmail.com>
+- Robin Ward <robin.ward@gmail.com>
+- Dmitry Medvinsky <me@dmedvinsky.name>
+- Jason Jacobson <jason.a.jacobson@gmail.com>
+- Jonas Follesø <jonas@follesoe.no>
+- Dan Allen <dan.j.allen@gmail.com>
+- noformnocontent <i@noformnocontent.com>
+- Damien White <damien.white@visoftinc.com>
+- Alexander Marenin <great_muchacho@mail.ru>
+- Cédric Néhémie <cedric.nehemie@gmail.com>
+- Simon Madine <simon@angryrobotzombie.com>
+- Benjamin Pannell <contact@sierrasoftworks.com>
+- Eric Knibbe <eric@lassosoft.com>
+- Poren Chiang <ren.chiang@gmail.com>
+- Kelley van Evert <kelleyvanevert@gmail.com>
+- Kurt Emch <kurt@kurtemch.com>
+- Mehdi Dogguy <mehdi@dogguy.org>
+- Nicolas Braud-Santoni <nicolas.braud-santoni@ens-cachan.fr>
+- Ralf Bitter <rabit@revigniter.com>
+- Sylvestre Ledru <sylvestre.ledru@scilab-enterprises.com>
+- Troy Kershaw <hello@troykershaw.com>
+- Zena Treep <zena.treep@gmail.com>
+- Daniel Kvasnicka <dkvasnicka@vendavo.com>
+- Carlo Kok <ck@remobjects.com>
+- Bram de Haan <info@atelierbramdehaan.nl>
+- Seongwon Lee <dlimpid@gmail.com>
+- Zaven Muradyan <megalivoithos@gmail.com>
+- Jan T. Sott <git@idleberg.com>
+- Brent Bradbury <brent@brentium.com>
+- Martin Dilling-Hansen <martindlling@gmail.com>
+- Ilya Vassilevsky <vassilevsky@gmail.com>
+- Josh Adams <josh@isotope11.com>
+- Dan Tao <daniel.tao@gmail.com>
+- Jeff Escalante <hello@jenius.me>
+- Jun Yang <yangjvn@126.com>
+- Nikolay Lisienko <info@neor.ru>
+- Heiko August <post@auge8472.de>
+- Domen Kožar <domen@dev.si>
+- Travis Odom <travis.a.odom@gmail.com>
+- innocenat <innocenat@gmail.com>
+- Arthur Bikmullin <devolonter@gmail.com>
+- Pascal Hurni <phi@ruby-reactive.org>
+- Roman Shmatov <romanshmatov@gmail.com>
+- Nic West <nic@letolab.com>
+- Panu Horsmalahti <panu.horsmalahti@iki.fi>
+- Flaviu Tamas <tamas.flaviu@gmail.com>
+- Damian Mee <mee.damian@gmail.com>
+- Christopher Kaster <ikasoki@gmail.com>
+- Chris Eidhof <chris@eidhof.nl>
+- Nate Cook <natecook@gmail.com>
+- Matt Diephouse <matt@diephouse.com>
+- Erik Osheim <d_m@plastic-idolatry.com>
+- Guillaume Laforge <glaforge@gmail.com>
+- Lucas Mazza <lucastmazza@gmail.com>
+- Maxim Dikun <dikmax@gmail.com>
+- Henrik Feldt <henrik@haf.se>
+- Anton Kochkov <anton.kochkov@gmail.com>
+- Michael Allen <Michael.Allen@benefitfocus.com>
+- JP Verkamp <me@jverkamp.com>
+- Adam Joseph Cook <adam.joseph.cook@gmail.com>
+- Sergey Vidyuk <svidyuk@gmail.com>
+- Radek Liska <radekliska@gmail.com>
+- Jose Molina Colmenero <gaudy41@gmail.com>
+- Max Mikhailov <seven.phases.max@gmail.com>
+- Bryant Williams <b.n.williams@gmail.com>
+- Erik Paluka <erik.paluka@gmail.com>
+- Luke Holder <lukemh@gmail.com>
+- David Mohundro <david@mohundro.com>
+- Nicholas Blumhardt <nblumhardt@nblumhardt.com>
+- Christophe de Dinechin <christophe@taodyne.com>
+- Taneli Vatanen <taneli.vatanen@gmail.com>
+- Jen Evers-Corvina <jen@sevvie.net>
+- Kassio Borges <kassioborgesm@gmail.com>
+- Cedric Sohrauer <sohrauer@googlemail.com>
+- Mickaël Delahaye <mickael.delahaye@gmail.com>
+- Hakan Özler <ozler.hakan@gmail.com>
+- Trey Shugart <treshugart@gmail.com>
+- Vincent Zurczak <vzurczak@linagora.com>
+- Adam Joseph Cook <adam.joseph.cook@gmail.com>
+- Edwin Dalorzo <edwin@dalorzo.org>
+- mucaho <mkucko@gmail.com>
+- Dennis Titze <dennis.titze@gmail.com>
+- Jon Evans <jon@craftyjon.com>
+- Brian Quistorff <bquistorff@gmail.com>
+- Jonathan Suever <suever@gmail.com>
+- Alexis Hénaut <alexis@henaut.net>
+- Chris Kiehl <audionautic@gmail.com>
+- Peter Piwowarski <oldlaptop654@aol.com>
+- Kenta Sato <bicycle1885@gmail.com>
+- Anthony Scemama <scemama@irsamc.ups-tlse.fr>
+- Taufik Nurrohman <latitudu.latitudu@gmail.com>
+- Pedro Oliveira <kanytu@gmail.com>
+- Gu Yiling <justice360@gmail.com>
+- Thomas Applencourt <thomas.applencourt@irsamc.ups-tlse.fr>
+- Andrew Farmer <ahfarmer@gmail.com>
+- Sergey Mashkov <cy6erGn0m@gmail.com>
+- Raivo Laanemets <raivo@infdot.com>
+- Kenneth Fuglsang <kfuglsang@gmail.com>
+- David Anson <david@dlaa.me>
+- Louis Barranqueiro <louis.barranqueiro@gmail.com>
+- Tim Schumacher <tim@datenknoten.me>
+- Lucas Werkmeister <mail@lucaswerkmeister.de>
+- Dan Panzarella <alsoelp@gmail.com>
+- Bruno Dias <bruno.r.dias@gmail.com>
+- Jay Strybis <jay.strybis@gmail.com>
+- Guillaume Gomez <guillaume1.gomez@gmail.com>
+- Janis Voigtländer <janis.voigtlaender@gmail.com>
+- Dirk Kirsten <dk@basex.org>
+- MY Sun <simonmysun@gmail.com>
+- Vadimtro <vadimtro@yahoo.com>
+- Benjamin Auder <benjamin.auder@gmail.com>
+- Dotan Dimet <dotan@corky.net>
+- Manh Tuan <junookyo@gmail.com>
+- Philippe Charrière <ph.charriere@gmail.com>
+- Stefan Bechert <stefan.bechert@gmx.net>
+- Samuel Reed <sam@bitmex.com>
+- Yury Selivanov <yselivanov@gmail.com>
+- Tsuyusato Kitsune <make.just.on@gmail.com>
+- Mick MacCallum <micksmaccallum@gmail.com>
+- Kristoffer Gronlund <kgronlund@suse.com>
+- Søren Enevoldsen <senevoldsen90@gmail.com>
+- Daniel Rosenwasser <DanielRosenwasser@users.noreply.github.com>
+- Ladislav Prskavec: <ladislav@prskavec.net>
Oops, something went wrong.

0 comments on commit de80a04

Please sign in to comment.