[SPARK-13300][Documentation] Added pygments.rb dependancy

[amitdev]

Looks like pygments.rb gem is also required for jekyll build to work. At least on Ubuntu/RHEL I could not do build without this dependency. So added this to steps.

[mengxr]

@amitdev The PR also contains ml-guide changes. Could you remove that commit from your branch?

[amitdev]

@mengxr, Sorry I committed that by mistake - removed now. Thanks!

[srowen]

@amitdev can you connect this to https://issues.apache.org/jira/browse/SPARK-13300 in the title? it might be the same thing?

It looks like lots of the examples aren't rendered in the current live site?
http://spark.apache.org/examples.html

[amitdev]

@srowen, it looks related - is there a way to verify it? I couldn't locate where the spark example code is. Things on the main documentation site is working fine. Anyway I'll change the title so that someone with access can take a look.

[srowen]

@amitdev just build the documentation. Do you see the code snippets correctly in the page http://spark.apache.org/examples.html in your local build? instead of the pygments error?

[srowen]

Aha, one significant update. Yes, the change here is a good one, as I also had to install this gem to build the docs. In fact we can simplify:

sudo gem install jekyll jekyll-redirect-from pygments.rb
sudo pip install Pygments sphinx

Maybe it's also worth noting that you'd have to have Ruby and pip installed (and I assume gcc, Python already), which may not be obvious. For Ubuntu that's:

sudo apt-get install ruby ruby-dev pip-python

I don't feel strongly about adding that, but hey.

Actually I was incorrect in connecting this to SPARK-13300 though, as the page in question is not generated from source in docs/. It's part of the ASF site. This also uses pygments though. So maybe it turns out to be the same problem.

While we can just add this change (with a possible simplification) I'm also looking at whether I can fix up the ASF site with something similar.

[srowen]

I fixed the ASF site at http://spark.apache.org/examples.html and indeed it required the same gem to be installed to work. So this is related.

[amitdev]

Thanks @srowen. I've mentioned dependencies on Ruby, Python and cleaned up the doc a little.

[srowen]

Jenkins test this please

[SparkQA]

Test build #51265 has finished for PR 11180 at commit 044b0d8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Merged to master/1.6. @amitdev if you'll tell me your JIRA handle I'll assign you.

[amitdev]

@srowen, my JIRA handle is amitdev.

[srowen]

@amitdev I don't see anyone by your name or "Amit Dev.." in ASF JIRA. Are you sure? maybe link me to your profile page

[amitdev]

@srowen, See https://issues.apache.org/jira/secure/ViewProfile.jspa?name=amitdev