Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Pygments.rb #5

Merged
merged 1 commit into from

3 participants

Simon Gate Zachary Scott Jon
Simon Gate

I have changed the pygments wrapper from Albino to pygments.rb. This makes the highlighting about 10-15 times faster (according to pygments.rb README). I also changed the settings variable to remote_coloring to make it easier to change syntax highlighters in the feature. BUT this will also break backwards compatibility and a major version bump should be recommended.

Zachary Scott
Owner

Thanks, I'll have to checkout pygments.rb. Can you use it on heroku?

Simon Gate

Albino used to power GitHubs syntax highlighting. But a long time ago they built and changed to pygments.rb to gain more speed. Reloading the python env on each request as Albino does slows down things some what.

Haven't tried pygments.rb on heroku, does albino work?

Zachary Scott
Owner

Albino won't work on Heroku due to lack of pygmentize, so we fall back to the pygments api

Simon Gate

Pygments.rb have pygments included. So if there is Python there is pygments if you use pygments.rb. Found a test implementation for deploying pygments.rb to Heroku. One thing to get it to work is that you need to specify python2.6 or else it will crash on Heroku it seems.

Simon Gate

Removed remote pygments generation since pygments.rb works with Heroku.

Zachary Scott
Owner

Looks good, think you could provide a test or two?

I'm hesitant on the python2.6 dependency, anyway to test this?

Jon namelessjon commented on the diff
lib/glorify/renderer.rb
((10 lines not shown))
def initialize(options={})
- @use_albino = options.fetch(:use_albino, true)
+ python = ENV['GLORIFY_PYTHON'] || 'python'
+ RubyPython.configure :python_exe => python

Do you need to configure every time you make a Renderer? Could this be moved somewhere more globally and just done once?

Simon Gate
smgt added a note

No, default you don't need to do anything. But when deploying to heroku you can set a environment variable to overwrite which version of python to use. Since you need to run python 2.6 on Heroku to make this work.

$ heroku config:add GLORIFY_PYTHON=python2.6

That was meant more in the sense of "Does this need to be here, run every time you make a renderer? i.e. once per page view" Though I guess configuration won't be that expensive anyway.

Simon Gate
smgt added a note

You could move it out of Glorify::Renderer if you like and set the configuration when you initialize a Glorify::Renderer instead. But I don't know how much the gain would be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/glorify/renderer.rb
((15 lines not shown))
super
end
def block_code(code, lang)
- if use_albino
- Albino.colorize(code, lang)
- else
- Net::HTTP.post_form(URI.parse('http://pygments.appspot.com/'),
- {'code'=>code, 'lang'=>lang}).body
- end
+ Pygments.highlight(code, :lexar => lang, :options => {:encoding => "utf-8"})

I'm pretty sure this should be :lexer not :lexar

Simon Gate
smgt added a note

Me to!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Simon Gate

You can try the changes out with this gist. https://gist.github.com/2665138

Simon Gate

I will remove the readme fix when I get home. It should not be in this PR.

Zachary Scott
Owner

@simon I think this is good. Could you squash down the commits and maybe add some tests?

@namelessjon Any objections before I merge this and release a 0.1.0?

Simon Gate

I can squash it but I will not be able to add tests in the near future.

Simon Gate smgt Change dep from albino to pygmentize
I also changed the settings option to something more general. If
you would like to change syntax highlighter in the feature there
will be no need to change options variable.

 - Oneline options set
 - Remove remote pygments generation.
 - Make it possible to configure python version with env vars
 - Catch lexer errors
 - Remove Gemfile.lock and add it to gitignore
 - Remove albino reference from README.md
e93938f
Zachary Scott
Owner

@simon Looks good, thanks!

I can add in the tests before the release, ping me if you want to help.

Zachary Scott zzak merged commit 96c7ef2 into from
Simon Gate

That would be awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 15, 2012
  1. Simon Gate

    Change dep from albino to pygmentize

    smgt authored
    I also changed the settings option to something more general. If
    you would like to change syntax highlighter in the feature there
    will be no need to change options variable.
    
     - Oneline options set
     - Remove remote pygments generation.
     - Make it possible to configure python version with env vars
     - Catch lexer errors
     - Remove Gemfile.lock and add it to gitignore
     - Remove albino reference from README.md
This page is out of date. Refresh to see the latest.
2  .gitignore
View
@@ -1,3 +1,5 @@
.rvmrc
pkg/*
*.swp
+.rbx
+Gemfile.lock
2  README.md
View
@@ -2,7 +2,7 @@
Sinatra helper to parse markdown with syntax highlighting like the pros
-Renders via redcarpet with syntax highlighting thanks to albino. Able to use
+Renders via redcarpet with syntax highlighting thanks to [pygments.rb](https://github.com/tmm1/pygments.rb). Able to use
fenced code blocks like github, and includes a default pygments stylesheet.
## install
2  glorify.gemspec
View
@@ -16,6 +16,6 @@ Gem::Specification.new do |s|
s.add_runtime_dependency "sinatra"
s.add_runtime_dependency "redcarpet", "~> 2.0"
- s.add_runtime_dependency "albino"
+ s.add_runtime_dependency "pygments.rb"
s.add_runtime_dependency "nokogiri"
end
3  lib/glorify.rb
View
@@ -6,7 +6,7 @@ module Sinatra
module Glorify
module Helpers
def glorify text
- rndr = Glorify::Renderer.new(:use_albino => settings.use_albino)
+ rndr = Glorify::Renderer.new
Redcarpet::Markdown.new(rndr, settings.glorify_extensions).render(text)
end
@@ -77,7 +77,6 @@ def glorify_css
end
def self.registered(app)
- app.set :use_albino, proc { system 'pygmentize -V' }
app.set :glorify_extensions, { :filter_html => true,
:autolink => true,
:no_intra_emphasis => true,
17 lib/glorify/renderer.rb
View
@@ -1,25 +1,24 @@
require "redcarpet"
-require "albino"
-require "net/http"
+require "pygments.rb"
module Sinatra
module Glorify
class Renderer < Redcarpet::Render::HTML
- attr_reader :use_albino
def initialize(options={})
- @use_albino = options.fetch(:use_albino, true)
+ python = ENV['GLORIFY_PYTHON'] || 'python'
+ RubyPython.configure :python_exe => python

Do you need to configure every time you make a Renderer? Could this be moved somewhere more globally and just done once?

Simon Gate
smgt added a note

No, default you don't need to do anything. But when deploying to heroku you can set a environment variable to overwrite which version of python to use. Since you need to run python 2.6 on Heroku to make this work.

$ heroku config:add GLORIFY_PYTHON=python2.6

That was meant more in the sense of "Does this need to be here, run every time you make a renderer? i.e. once per page view" Though I guess configuration won't be that expensive anyway.

Simon Gate
smgt added a note

You could move it out of Glorify::Renderer if you like and set the configuration when you initialize a Glorify::Renderer instead. But I don't know how much the gain would be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
super
end
def block_code(code, lang)
- if use_albino
- Albino.colorize(code, lang)
- else
- Net::HTTP.post_form(URI.parse('http://pygments.appspot.com/'),
- {'code'=>code, 'lang'=>lang}).body
+ begin
+ Pygments.highlight(code, :lexer => lang, :options => {:encoding => "utf-8"})
+ rescue RubyPython::PythonError
+ Pygments.highlight(code, :options => {:encoding => "utf-8"})
end
end
+
end
end
end
Something went wrong with that request. Please try again.