Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make language specification available as a PDF #7432

Merged
merged 3 commits into from Dec 10, 2020

Conversation

@sake92
Copy link
Contributor

@sake92 sake92 commented Nov 14, 2018

The PDF is now published at https://scala-lang.org/files/archive/spec/2.13/spec.pdf


This is an attempt to help with scala/bug#10218
Wkhtmltopdf (v 0.12.6-dev) is used for generating PDF from HTML(s).

I tried first to render with existing setup, but had to make a few changes in print CSS and hide some divs from output.
Unfortunately, Mathjax doesn't behave well:

  • takes long to load
  • outputs small fonts when using wkhtmltopdf (I tried to set zoom etc, doesn't help)
  • puts newline before every math expression in PDF... (no idea why does that)

This solution uses a more lightweight library called KaTeX.
A problem with KaTeX is that you can't escape a $ sign outside of math rendering scope.
Mathjax has this processEscapes: true property so you can do a \$ anywhere.
So I had to pick a different delimiter, ACUTE ACCENT (麓) character (I also call it "forward-tick").

Here is PDF example: https://www.dropbox.com/s/yjcrs2ztk9myd0a/Spec.pdf?dl=0
NOTE: I used Batch script, didn't try Bash script, if someone could verify it works also that would be great. 馃檪

@scala-jenkins scala-jenkins added this to the 2.13.0-RC1 milestone Nov 14, 2018
@sake92 sake92 changed the title Specification pdf wkhtmltopdf katex Generate Specification PDF Nov 14, 2018
@sake92
Copy link
Contributor Author

@sake92 sake92 commented Nov 14, 2018

I signed the CLA.

@adriaanm
Copy link
Contributor

@adriaanm adriaanm commented Nov 15, 2018

Wow, great work! 馃槏

I browsed the generated html locally and it looks really good.

@smarter
Copy link
Contributor

@smarter smarter commented Nov 22, 2018

Very cool. May I request that we store the generated output at http://www3.scala-lang.org/files/archive/nightly/pdfs/ScalaReference.pdf where version 2.9 currently lives ? This is what you get when you search for "scala specification" on Google Scholar currently.

It would also be useful to have the first page of the PDF explain that this document was generated from

@sake92
Copy link
Contributor Author

@sake92 sake92 commented Nov 24, 2018

Thanks guys! I have 2 quick questions:

  1. In section function types the return type in code snippet is R but should be U?
  2. In Return Expressions the NonLocalReturnException is mentioned but I couldn't find it in Scaladocs.. Should that be NonLocalReturnControl?

@smarter "that this document was generated from" ..? HTML version of Scala spec?
If that's what you meant, I added it now.

@smarter
Copy link
Contributor

@smarter smarter commented Nov 24, 2018

In section function types the return type in code snippet is R but should be U?

No because it's describing the definition of FunctionN which uses R, I think it's fine as is.

In Return Expressions the NonLocalReturnException is mentioned but I couldn't find it in Scaladocs.. Should that be NonLocalReturnControl?

Yes.

@smarter "that this document was generated from" ..? HTML version of Scala spec?
If that's what you meant, I added it now.

Yes, that's what I meant.

@SethTisue
Copy link
Member

@SethTisue SethTisue commented Jan 11, 2019

@sake92 interested in returning to this...?

@sake92
Copy link
Contributor Author

@sake92 sake92 commented Jan 12, 2019

@SethTisue I merged the latest changes and updated the linked PDF example.
If there's anything else to be done regarding this, please let me know.

@SethTisue SethTisue self-assigned this Jan 31, 2019
@SethTisue SethTisue removed this from the 2.13.0-RC1 milestone Feb 5, 2019
@SethTisue SethTisue added this to the 2.13.1 milestone Feb 5, 2019
@SethTisue
Copy link
Member

@SethTisue SethTisue commented Feb 5, 2019

@sake92 can you rebase and squash this into a smaller number of commits?

I put this on the 2.13.1 milestone since it isn't a release blocker, but it could still make 2.13.0-RC1

@sake92 sake92 force-pushed the specification-pdf-wkhtmltopdf-katex branch from 3c21433 to 60af077 Feb 6, 2019
@sake92
Copy link
Contributor Author

@sake92 sake92 commented Feb 6, 2019

@SethTisue I did the rebase/squash thing. Never done squashing before, I hope everything's ok.. 馃槃

@SethTisue SethTisue removed this from the 2.13.1 milestone Feb 12, 2019
@SethTisue SethTisue added this to the 2.13.0-RC1 milestone Feb 12, 2019
@SethTisue SethTisue force-pushed the specification-pdf-wkhtmltopdf-katex branch from 60af077 to caadc3e Feb 12, 2019
@SethTisue
Copy link
Member

@SethTisue SethTisue commented Feb 12, 2019

I took the liberty of force-pushing to your branch to fix a further merge conflict and to make some changes to the bash script.

note to anyone who wants to try this, on MacOS you can brew cask install wkhtmltopdf (note the cask)

the bash script needs some more work. I need to work on some other things right now, but I'll come back to this

@SethTisue SethTisue removed this from the 2.13.0-RC1 milestone Feb 16, 2019
@SethTisue SethTisue added this to the 2.13.1 milestone Feb 16, 2019
@SethTisue SethTisue force-pushed the specification-pdf-wkhtmltopdf-katex branch 3 times, most recently from 02edced to 95f764e Mar 6, 2019
@SethTisue
Copy link
Member

@SethTisue SethTisue commented Mar 6, 2019

I stabbed it with my steely knives for a while, but we won't be dining on roast beast tonight, it seems.

The bash script now generates a PDF on my Mac (Darwin doesn't do ls -I, and I had to make it ignore some weird http error I don't understand but hopefully doesn't matter, and my shell scripting fu is most definitely stoppable, and ...)

but Travis says:

The switch --print-media-type, is not support using unpatched qt, and will be ignored.
Unknown long argument --window-status

And then later it prints some dusty old Google Code URL, remember Google Code? So apparently brew install wkhtmltopdf on MacOS doesn't give us the same version that sudo apt-get install wkhtmltopdf gets us on Travis, what fun.

It's first and 10 again, at least.

@SethTisue
Copy link
Member

@SethTisue SethTisue commented Mar 19, 2019

in a dusty corner of the internet I found an old thing where someone used:

install:
    - mkdir wkhtmltopdf
    - wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.9.9-static-amd64.tar.bz2
    - tar xvjf wkhtmltopdf-0.9.9-static-amd64.tar.bz2 -C wkhtmltopdf
    - mv wkhtmltopdf/wkhtmltopdf-amd64 wkhtmltopdf/wkhtmltopdf
    - export PATH=$PATH:$PWD/wkhtmltopdf/

seems like I ought to be able to massage this into something that works

@SethTisue SethTisue force-pushed the specification-pdf-wkhtmltopdf-katex branch from 7601356 to f128c02 Dec 9, 2020
@SethTisue
Copy link
Member

@SethTisue SethTisue commented Dec 9, 2020

Squashed. I'll hit "merge" once CI likes it again.

sake92 and others added 2 commits Dec 9, 2020
also includes:
* Fix JS issue in spec PDF building
* Generate HTML for PDF in a separate folder

Co-authored-by: Sakib Hadziavdic <sake.etf@gmail.com>
@SethTisue SethTisue force-pushed the specification-pdf-wkhtmltopdf-katex branch from f128c02 to 6fb0680 Dec 9, 2020
@SethTisue SethTisue merged commit 7035d68 into scala:2.13.x Dec 10, 2020
4 checks passed
@SethTisue
Copy link
Member

@SethTisue SethTisue commented Dec 10, 2020

Leaving scala/bug#10218 open for now since I still need to do the publish-it-to-the-website part. So let's not pop too many champagne corks until I get around to that :-)

Thank you, @sake92, for taking this on and remaining committed to it despite the long wait.

@sake92
Copy link
Contributor Author

@sake92 sake92 commented Dec 10, 2020

Thank you @SethTisue for all the help! Glad that this is finally over. 馃槃

@SethTisue
Copy link
Member

@SethTisue SethTisue commented Dec 12, 2020

as for the publishing part: at e.g. https://travis-ci.org/github/scala/scala/jobs/749278429 we see scripts/travis-publish-spec.sh running and appearing to rsync files up to chara including a spec.pdf, but https://scala-lang.org/files/archive/spec/2.13/spec.pdf is 404

https://github.com/scala/scala-dev/blob/scala-dev/.github/ISSUE_TEMPLATE/release.md has instructions for ssh'ing to chara (I can never remember). if I ssh there and look in /home/linuxsoft/archives/scala/spec/2.13/, spec.pdf is there

so, okay, now this is familiar from doing releases actually, you have to wait for https://scala-webapps.epfl.ch/jenkins/view/All/job/production_scala-lang.org-scala-dist-archive-sync/ to sync the files into production

but this PR was merged two days ago and that job has run a number of times since then...

ah but then there's also https://scala-webapps.epfl.ch/jenkins/view/All/job/production_scala-lang.org-builder/ ... hmm no, that doesn't seem relevant

I triggered https://scala-webapps.epfl.ch/jenkins/view/All/job/production_scala-lang.org-scala-dist-archive-sync/154699/ manually just now on the off chance it might help

interesting, it's been running for 12 minutes now... recent past runs seem to be in the 3鈥10 minute range (why such variation?)

OMG THAT DID IT, we're live! https://scala-lang.org/files/archive/spec/2.13/spec.pdf

champagne consumption may commence 馃

@smarter
Copy link
Contributor

@smarter smarter commented Dec 12, 2020

Awesome! By the way, if I type "scala language specification" on Google the first result is the venerable spec of Scala 2.6 (https://www.scala-lang.org/docu/files/doc/scala-documentation/ScalaReference.pdf), any chance that could redirect to the latest version?

@SethTisue
Copy link
Member

@SethTisue SethTisue commented Dec 14, 2020

@fsalvi could you add that redirect, please?

@fsalvi
Copy link
Contributor

@fsalvi fsalvi commented Dec 15, 2020

Ok, I added a redirect to the new spec page.

@SethTisue SethTisue changed the title Generate Specification PDF Make language specification available as a PDF Feb 19, 2021
@SethTisue SethTisue mentioned this pull request Feb 22, 2021
69 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
10 participants