WIP: Switch to using LaTeXML

[bfirsh]

This is a proof of concept for switching from Pandoc to LaTeXML to see if it's a good idea. So far, it's looking more promising than our Pandoc hacks. See also #159.

My rough todo list:

• Switch out pandoc with latexml and see what happens (it works!)
• Get the postprocessor to convert latexml output into what Distill expects, and style misc things
• Get integration test suite working
• Compare output with loads of docs, particular ones that are known not to work with pandoc (failed renders on arxiv-vanity.com, reported bugs, etc)
• Linkify URLs (probably with https://github.com/sindresorhus/linkify-urls because the regex we use in the Pandoc filter doesn't work in JS).
• Add http:// to links that don't have it
• Align math labels correct (probably by putting it in the latex like we did for pandoc instead of using a table)
• Put table captions on bottom if they're on top (e.g. http://localhost:8010/html/1707.08945v1/ )
• Get some common publication styles working properly (ICML and IEEE are the ones we have explicit integration tests for)
• Don't allow resizing of text where we don't want it (e.g. headings, figure captions, bibliography items, etc)
• Get h5 and h6 working again (just commented out in headings postprocessor)
• Make authors bold
• Test on mobile
• Fix tables and equations on mobile
• Make PDF/EPS conversion higher res Make graphics rendering higher resolution #248

Can possibly be deferred to after shipping:

• Generally clean everything up. Particularly stylesheets, etc.
• Fix sizing/cropping of SVG images. A lot of them seem to be rendered out of the viewport.
• Style theorems Support theorems #157
• Fix styling of citet/citep references. They look like shit with current style. Improve styling of citet/citep #249
• Make click target for references bigger (e.g. "Section 2" only links "2". should link whole thing) Make click target for references bigger  #254
• Implement subcaption package? http://localhost:8010/html/1709.10478/ Implement subcaption package #245
• Get \citep working (e.g. http://localhost:8010/html/1706.02515/ ) Fix \citep #233
• Do something about papers with no titles http://localhost:8010/html/1707.00860v2/
• Make \And and \AND make author bold http://localhost:8010/html/1707.08952v1/ Make \And and \AND work #252
• Make www.example.com convert to a link (ie no http://) Make URLs without http:// convert to link #253
• Don't allow changing font-size all all text! This is probably most robustly done by getting our hands dirty in XLST. Don't allow changing font-size in any text #250
• Make listings less shit (why isn't it just plain text <pre>?!) Style line numbers in listings #226
• Make PDF/EPS convert to SVG instead of PNG Convert PDF, EPS, etc to scalable SVGs #247
• Make issues for common style files we don't support Figure out which the most common unsupported styles are #251
• Get common CS/ML styles working (ICML, IEEE, ACL) Support common CS/ML styles #241

Long term goals:

• Instead of rewriting latexml's HTML to fit Distill's stylesheet, write CSS for latexml's output. This means latexml becomes standard HTML output and we're just doing some styling. As part of that, we'll be forced to improve latexml's output if it needs improving, too. Instead of rewriting HTML to fit Distill, write CSS for LaTeXML's output #255

A rough list of known problems this fixes (!):

Closes #159
Closes #9
Closes #23
Closes #24
Closes #25
Closes #34
Closes #51 (latexml renders tables so, so much better)
Closes #52
Closes #66
Closes #74
Closes #79
Closes #80
Closes #84 (I think? or at least makes it not our problem)
Closes #97
Closes #92
Closes #99
Closes #103
Closes #105
Closes #109
Closes #107
Closes #148
Closes #139
Closes #142
Closes #160
Closes #131
Closes #175
Closes #154

(and probably opens lots of things too)

🎉

[bfirsh]

A bit of background on the motivation behind this https://twitter.com/bfirsh/status/938416106687029249?s=17

[andreasjansson]

belated 👍 👍 👍

[jilljenn]

Thanks for the posterity. I feel like these kinds of links should somehow be featured on the README:

• LaTeXML for arXiv conversion? #159
• WIP: Switch to using LaTeXML #210

My first question was: why not pandoc? and I somehow got some answers there. Thanks!