New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rouge as an alternative to pygments (Jekyll 2.x) #79

Closed
wants to merge 3 commits into
from

Conversation

Projects
None yet
@kostrse

kostrse commented Jul 16, 2014

Jekyll 2 supports https://github.com/jneen/rouge hightlighter as an alternative to pygments.
This change adds current version of rouge (1.5.1) to github-pages.

Backlog and known issues:

@lkuczborski

This comment has been minimized.

Show comment
Hide comment
@lkuczborski

lkuczborski Jul 21, 2014

When can we expect Rouge to be included? It is currently the only one to support Swift :(

When can we expect Rouge to be included? It is currently the only one to support Swift :(

@kostrse

This comment has been minimized.

Show comment
Hide comment
@kostrse

kostrse Jul 21, 2014

Only after integration of Jekyll 2.x:
#75 Upgrade to Jekyll 2.1

kostrse commented Jul 21, 2014

Only after integration of Jekyll 2.x:
#75 Upgrade to Jekyll 2.1

@benbalter

This comment has been minimized.

Show comment
Hide comment
@benbalter

benbalter Jul 21, 2014

Contributor

What's the advantage to Rouge over Pygments? From what I understand, it's pure Ruby (👍), but less feature complete (👎).

I'm hesitant to add Yet Another Syntax Highlighter (TM), lest we end up with Markdown interpreter-style code sprawl. It's crazy confusing for new users. (e.g., how do I know which one's "better"?)

As of right now I'm 👎 on including Rogue, unless A) it gains feature parity with Pygments, and B) Jekyll drops support for Pygments in favor of Rogue as a transparent upgrade from a user perspective.

Contributor

benbalter commented Jul 21, 2014

What's the advantage to Rouge over Pygments? From what I understand, it's pure Ruby (👍), but less feature complete (👎).

I'm hesitant to add Yet Another Syntax Highlighter (TM), lest we end up with Markdown interpreter-style code sprawl. It's crazy confusing for new users. (e.g., how do I know which one's "better"?)

As of right now I'm 👎 on including Rogue, unless A) it gains feature parity with Pygments, and B) Jekyll drops support for Pygments in favor of Rogue as a transparent upgrade from a user perspective.

@lkuczborski

This comment has been minimized.

Show comment
Hide comment
@lkuczborski

lkuczborski Jul 21, 2014

Rouge is supporting Swift already as I understand. This is important for many Swift bloggers out there. Currently no way to do this natively in Pygments, unless you create custom lexer yourself.

Rouge is supporting Swift already as I understand. This is important for many Swift bloggers out there. Currently no way to do this natively in Pygments, unless you create custom lexer yourself.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 22, 2014

Contributor

@lkuczborski Have you upgraded to pygments.rb 0.6.x? It is supposed to support rouge.

Contributor

parkr commented Jul 22, 2014

@lkuczborski Have you upgraded to pygments.rb 0.6.x? It is supposed to support rouge.

@lkuczborski

This comment has been minimized.

Show comment
Hide comment
@lkuczborski

lkuczborski Jul 22, 2014

@parkr How would I do that? I am using Bundler to see what is currently available on Github Pages and it uses pygments.rb 0.5.4.

@parkr How would I do that? I am using Bundler to see what is currently available on Github Pages and it uses pygments.rb 0.5.4.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 22, 2014

Contributor

@lkuczborski It will be bumped with the Jekyll 2.1 bump. Jekyll 1.x does not support Rouge.

Contributor

parkr commented Jul 22, 2014

@lkuczborski It will be bumped with the Jekyll 2.1 bump. Jekyll 1.x does not support Rouge.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jul 31, 2014

Contributor

So far GitHub Security is 👎 on Rouge, and @benbalter is likewise per the "Yet Another X" argument.

I see Jekyll eventually switching over to Rouge exclusively but until then, we will want to close this. Thanks!

Contributor

parkr commented Jul 31, 2014

So far GitHub Security is 👎 on Rouge, and @benbalter is likewise per the "Yet Another X" argument.

I see Jekyll eventually switching over to Rouge exclusively but until then, we will want to close this. Thanks!

@parkr parkr closed this Jul 31, 2014

@swanson

This comment has been minimized.

Show comment
Hide comment
@swanson

swanson Aug 4, 2014

Just a footnote - it was not clear to me that Rouge is not supported on GH Pages yet. I tried using it, only to find that the build would fail on GitHub - with a pretty unhelpful message (generic markdown syntax error). Reverting to pygments fixed the issue.

Could/should the jekyll proper docs be updated to be more explicit about what GH Pages supports?

swanson commented Aug 4, 2014

Just a footnote - it was not clear to me that Rouge is not supported on GH Pages yet. I tried using it, only to find that the build would fail on GitHub - with a pretty unhelpful message (generic markdown syntax error). Reverting to pygments fixed the issue.

Could/should the jekyll proper docs be updated to be more explicit about what GH Pages supports?

@kleinfreund

This comment has been minimized.

Show comment
Hide comment
@kleinfreund

kleinfreund Aug 4, 2014

Basically, GitHub Pages and Jekyll are not the same thing, but having a note for so thightly coupled things would be helpful, yes.

I have bookmarked https://pages.github.com/versions/ to check what Pages is actually using currently.

Basically, GitHub Pages and Jekyll are not the same thing, but having a note for so thightly coupled things would be helpful, yes.

I have bookmarked https://pages.github.com/versions/ to check what Pages is actually using currently.

MikeChristianson added a commit to MikeChristianson/MikeChristianson.github.io that referenced this pull request Oct 10, 2014

@ronan-mch

This comment has been minimized.

Show comment
Hide comment
@ronan-mch

ronan-mch Jan 1, 2015

Hi, Just to note that this would be a useful change for those of us developing on the Windows platform. Jekyll cannot build successfully on Windows when pygments is used for highlighting (see this issue for details). The accepted solution is to use rouge instead, but this caused a build error when my site was pushed to GitHub presumably because Pages doesn't support rouge. This means that I will have to use rouge locally while ensuring that it isn't pushed to GitHub. A minor inconvenience but I thought you might like to hear about it. Cheers!

Hi, Just to note that this would be a useful change for those of us developing on the Windows platform. Jekyll cannot build successfully on Windows when pygments is used for highlighting (see this issue for details). The accepted solution is to use rouge instead, but this caused a build error when my site was pushed to GitHub presumably because Pages doesn't support rouge. This means that I will have to use rouge locally while ensuring that it isn't pushed to GitHub. A minor inconvenience but I thought you might like to hear about it. Cheers!

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Jan 1, 2015

Contributor

@ronan-mch FWIW, we plan to set Rouge as the default in Jekyll 3. Not sure about Pages adoption, but I'm going to ask for it very nicely. It being purely Ruby, Rouge provides a better experience for all, including those who run JRuby.

Contributor

parkr commented Jan 1, 2015

@ronan-mch FWIW, we plan to set Rouge as the default in Jekyll 3. Not sure about Pages adoption, but I'm going to ask for it very nicely. It being purely Ruby, Rouge provides a better experience for all, including those who run JRuby.

@mofosyne

This comment has been minimized.

Show comment
Hide comment
@mofosyne

mofosyne Feb 4, 2015

github pages default highlighter pygments just plain don't work well on windows.

My current process is quite cumbersome, in having to switch between rouge/pygments. So getting rouge enabled by default would be awesome. (at least until pygments get their build fixed for windows)

And yea I agree with you parkr that rouge seems to be a much less painful experience installing it.

mofosyne commented Feb 4, 2015

github pages default highlighter pygments just plain don't work well on windows.

My current process is quite cumbersome, in having to switch between rouge/pygments. So getting rouge enabled by default would be awesome. (at least until pygments get their build fixed for windows)

And yea I agree with you parkr that rouge seems to be a much less painful experience installing it.

@NickCraver

This comment has been minimized.

Show comment
Hide comment
@NickCraver

NickCraver Mar 25, 2015

Has there been any change on this? I'm currently maintaining my site on multiple OSes (Windows and OS X) and don't need python at all except to get syntax highlighting. It's a rather large pain to install an entire runtime (with security concerns of its own) for such a simple purpose. It's tremendously more complicated to setup and maintain (especially on Windows) whereas rouge took no time at all.

It's a negative enough experience on Windows that I've currently given up on syntax highlighting on GitHub itself. I'd love for it to work, but it's simply not worth the trouble to get pygments running smooth on Windows.

Has there been any change on this? I'm currently maintaining my site on multiple OSes (Windows and OS X) and don't need python at all except to get syntax highlighting. It's a rather large pain to install an entire runtime (with security concerns of its own) for such a simple purpose. It's tremendously more complicated to setup and maintain (especially on Windows) whereas rouge took no time at all.

It's a negative enough experience on Windows that I've currently given up on syntax highlighting on GitHub itself. I'd love for it to work, but it's simply not worth the trouble to get pygments running smooth on Windows.

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Mar 25, 2015

Contributor

Hey Nick,

As you may know, Jekyll 3.0 will ship with Rouge as its default, rather than pygments.rb. As Pygments and Rouge are not 1-1 compatible, I'll be asking the Pages team to support Rouge instead of Pygments to provide the best experience to Pages users. That said, it's entirely their call so it's possible they may not choose to support Rouge.

Contributor

parkr commented Mar 25, 2015

Hey Nick,

As you may know, Jekyll 3.0 will ship with Rouge as its default, rather than pygments.rb. As Pygments and Rouge are not 1-1 compatible, I'll be asking the Pages team to support Rouge instead of Pygments to provide the best experience to Pages users. That said, it's entirely their call so it's possible they may not choose to support Rouge.

@NickCraver

This comment has been minimized.

Show comment
Hide comment
@NickCraver

NickCraver Mar 25, 2015

@parkr Thanks. It'll be unfortunate if they don't even support the defaults from you guys. A note here about the security concerns would be helpful so they can at least be addressed and hopefully resolved.

I do think it would be worth nothing in the Jekyll docs that Rouge isn't a viable option for such a large use case.

Please do let us know what they say about Rouge support with 3.0. If they don't intend to support the defaults or even (IMO) reasonable setup, then I'd personally much rather do my compilation and pushing to hosting elsewhere.

@parkr Thanks. It'll be unfortunate if they don't even support the defaults from you guys. A note here about the security concerns would be helpful so they can at least be addressed and hopefully resolved.

I do think it would be worth nothing in the Jekyll docs that Rouge isn't a viable option for such a large use case.

Please do let us know what they say about Rouge support with 3.0. If they don't intend to support the defaults or even (IMO) reasonable setup, then I'd personally much rather do my compilation and pushing to hosting elsewhere.

@vmg

This comment has been minimized.

Show comment
Hide comment
@vmg

vmg Mar 25, 2015

Member

Hi! Unfortunately we're not able to support Rouge in production right now (for the reasons previously stated in the thread). We're looking to migrate to another syntax highlighter, the one we use in production and which runs natively and without Python, but I have no ETA on that at the moment. Sorry for the hassle!

Member

vmg commented Mar 25, 2015

Hi! Unfortunately we're not able to support Rouge in production right now (for the reasons previously stated in the thread). We're looking to migrate to another syntax highlighter, the one we use in production and which runs natively and without Python, but I have no ETA on that at the moment. Sorry for the hassle!

@NickCraver

This comment has been minimized.

Show comment
Hide comment
@NickCraver

NickCraver Mar 25, 2015

@vmg can you actually elaborate on the security reason? Unless I'm missing something, only 2 reasons were given in this thread:

  1. It's "yet another highlighter"
  2. Unknown security issue

Number 1 is going away as it will be the default. That means a default install of Jekyll will not behave as expected when pushed. I'd say that's also far more confusing for new users, which was another portion of the point.

Number 2 has no detail at all, it's just a relay of "github security doesn't like it". I'm not saying it's unfounded, but why not tell us what's wrong? We're software developers here, if security is the only reason left, let us actually fix it.

There's currently no decent way for Windows users to use syntax highlighting on GitHub, that's really the bottom line here. Given the source of the hosting, the audience using it, and he mechanism in which it works (a git push), it's far more than likely someone wants to use this with GitHub pages.

All I'm asking is this: Please let us help.

@vmg can you actually elaborate on the security reason? Unless I'm missing something, only 2 reasons were given in this thread:

  1. It's "yet another highlighter"
  2. Unknown security issue

Number 1 is going away as it will be the default. That means a default install of Jekyll will not behave as expected when pushed. I'd say that's also far more confusing for new users, which was another portion of the point.

Number 2 has no detail at all, it's just a relay of "github security doesn't like it". I'm not saying it's unfounded, but why not tell us what's wrong? We're software developers here, if security is the only reason left, let us actually fix it.

There's currently no decent way for Windows users to use syntax highlighting on GitHub, that's really the bottom line here. Given the source of the hosting, the audience using it, and he mechanism in which it works (a git push), it's far more than likely someone wants to use this with GitHub pages.

All I'm asking is this: Please let us help.

@vmg

This comment has been minimized.

Show comment
Hide comment
@vmg

vmg Mar 25, 2015

Member

Hi again! We don't have a specific security issue in mind when looking at Rouge, but our concern is with the fact that it's a third party library which we've never used, audited nor reviewed before.

We've used Pygments in production for 5 years with solid results and audited it extensively, and we really trust it. We've just recently switched to a homegrown syntax highlighter which we've also audited inhouse and paid a (very expensive) third party audit for. This is software that runs under a sandbox but that it's used by millions of users every day; we take security very seriously, specially for the parts of our infrastructure that process arbitrary amounts of user content every day, and we cannot throw in new libraries in our stack just because Jekyll upstream decided to switch to them.

We could certainly spend our time reviewing and preparing Rouge to be run in production, but we'd rather work on open-sourcing our own syntax highlighter so people on all platforms can enjoy syntax highlighting at native speed without having to install other language runtimes. I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll, so wasting our time with a full audit and integration of Rouge can hardly be justified.

Again, I definitely feel your pain (syntax highlighting in Ruby is one of the many things that are inconvenient to do in Windows right now), and I appreciate your willingness to help, but there's not much you can do here -- it's just a matter of us not having enough hours on the day. But you can have my word we're prioritizing the issue, and that we really want to share our new highlighter with the world.

Member

vmg commented Mar 25, 2015

Hi again! We don't have a specific security issue in mind when looking at Rouge, but our concern is with the fact that it's a third party library which we've never used, audited nor reviewed before.

We've used Pygments in production for 5 years with solid results and audited it extensively, and we really trust it. We've just recently switched to a homegrown syntax highlighter which we've also audited inhouse and paid a (very expensive) third party audit for. This is software that runs under a sandbox but that it's used by millions of users every day; we take security very seriously, specially for the parts of our infrastructure that process arbitrary amounts of user content every day, and we cannot throw in new libraries in our stack just because Jekyll upstream decided to switch to them.

We could certainly spend our time reviewing and preparing Rouge to be run in production, but we'd rather work on open-sourcing our own syntax highlighter so people on all platforms can enjoy syntax highlighting at native speed without having to install other language runtimes. I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll, so wasting our time with a full audit and integration of Rouge can hardly be justified.

Again, I definitely feel your pain (syntax highlighting in Ruby is one of the many things that are inconvenient to do in Windows right now), and I appreciate your willingness to help, but there's not much you can do here -- it's just a matter of us not having enough hours on the day. But you can have my word we're prioritizing the issue, and that we really want to share our new highlighter with the world.

@thomasjo

This comment has been minimized.

Show comment
Hide comment
@thomasjo

thomasjo Mar 25, 2015

I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll

Awesome!

I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll

Awesome!

@parkr

This comment has been minimized.

Show comment
Hide comment
@parkr

parkr Mar 25, 2015

Contributor

I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll

Man, this must be the best syntax highlighter out there!

Whatever we support by default must be 100% compatible for Windows and keep C ext to an absolute minimum. The headaches we have had to deal with as a result of pygments and posix spawn on all sorts of systems make a pure-Ruby syntax highlighter far more appealing as a default. Hope it's that!

Contributor

parkr commented Mar 25, 2015

I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll

Man, this must be the best syntax highlighter out there!

Whatever we support by default must be 100% compatible for Windows and keep C ext to an absolute minimum. The headaches we have had to deal with as a result of pygments and posix spawn on all sorts of systems make a pure-Ruby syntax highlighter far more appealing as a default. Hope it's that!

@NickCraver

This comment has been minimized.

Show comment
Hide comment
@NickCraver

NickCraver Mar 25, 2015

@vmg Thank you! That's a much better picture of the GitHub side I was looking for and it sounds like a great solution all around. Do you have any idea if it'll be released in time to be in Jekyll 3.0?

It seems like changing the default would be best in a major release on their side, and it'd be good if you guys reached out (if you haven't already!) as they may want to coordinate the release with it in mind.

@vmg Thank you! That's a much better picture of the GitHub side I was looking for and it sounds like a great solution all around. Do you have any idea if it'll be released in time to be in Jekyll 3.0?

It seems like changing the default would be best in a major release on their side, and it'd be good if you guys reached out (if you haven't already!) as they may want to coordinate the release with it in mind.

@donmccurdy

This comment has been minimized.

Show comment
Hide comment
@donmccurdy

donmccurdy Feb 6, 2016

The recommendation here seems to be to keep using Pygments until PrettyLights is released, but I've started getting notices that redcarpet support is going away and I need to switch to Kramdown.

Kramdown supports Rouge or Coderay, but GH Pages does not. Pygment doesn't seem to work with Kramdown unless I rewrite everything in Liquid {% highlight %} tags. Is there another way to keep syntax highlighting after I switch to Kramdown?


EDIT: Apparently I misunderstood, or this thread is out of date. GH Pages seems to work fine with Rouge, according to both the Jekyll 3 announcement post and a quick test on my blog.

The recommendation here seems to be to keep using Pygments until PrettyLights is released, but I've started getting notices that redcarpet support is going away and I need to switch to Kramdown.

Kramdown supports Rouge or Coderay, but GH Pages does not. Pygment doesn't seem to work with Kramdown unless I rewrite everything in Liquid {% highlight %} tags. Is there another way to keep syntax highlighting after I switch to Kramdown?


EDIT: Apparently I misunderstood, or this thread is out of date. GH Pages seems to work fine with Rouge, according to both the Jekyll 3 announcement post and a quick test on my blog.

donmccurdy added a commit to donmccurdy/donmccurdycom-2015 that referenced this pull request Feb 6, 2016

Update to kramdown.
GitHub Pages will drop support for redcarpet after May 1, 2016.

Currently this requires converting all markdown codeblocks to Liquid
tags, because of lack of overlap between syntax highlighters supported
by both GH and kramdown. 😞

See: github/pages-gem#79 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment