Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add HTML::Pipeline to core #1760

Closed
wants to merge 4 commits into from

3 participants

@gjtorikian
Collaborator

HTML::Pipeline is the :bee:'s knees. One day @benbalter was like

what are your thoughts on submitting html-pipeline to Jekyll core as a markdown renderer

and I was all

i am very much :+1: for it

So here we go.

Much to my surprise this actually works pretty well. I took a cue from how nanoc accepts filters, wherein you can basically pass in an array of filter names, and The System will figure out what you mean.

I can't figure out how to properly test for failures (they're written in the test), so help on that would be much appreciated.

Todo:

  • Write test for mentions
  • Actually test for failing items
  • Support custom filters Will probably need to be handled by a plugin, not core.
  • Update docs to list html_pipeline defaults

/cc @parkr

lib/jekyll/converters/markdown.rb
@@ -19,7 +21,7 @@ def setup
MarukuParser.new @config
else
STDERR.puts "Invalid Markdown processor: #{@config['markdown']}"
- STDERR.puts " Valid options are [ maruku | rdiscount | kramdown | redcarpet ]"
+ STDERR.puts " Valid options are [ html_pipeline | maruku | rdiscount | kramdown | redcarpet ]"
@mattr- Owner
mattr- added a note

I don't believe there was any sort of ordering in the processor list here, but let's leave maruku as the first one since it's still the default.

@gjtorikian Collaborator

This is spat out of an STDERR message, I don't know how it'd affect any processing. But I'll change it.

@mattr- Owner
mattr- added a note

You're right. This doesn't affect any processing. My only concern is if it's listed first then it will be assumed that it's the default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/jekyll/converters/markdown/html_pipeline_parser.rb
((10 lines not shown))
+
+ def filter_key(s)
+ s.to_s.downcase.to_sym
+ end
+
+ def is_filter?(f)
+ f < HTML::Pipeline::Filter
+ rescue LoadError, ArgumentError
+ false
+ end
+
+ def ensure_default_opts
+ @config['html_pipeline']['filters'] ||= ['markdownfilter']
+ @config['html_pipeline']['context'] ||= {'gfm' => true}
+ # symbolize strings as keys, which is what HTML::Pipeline wants
+ @config['html_pipeline']['context'] = @config['html_pipeline']['context'].inject({}){|memo,(k,v)| memo[k.to_sym] = v; memo}
@mattr- Owner
mattr- added a note

We have an existing method called symbolize_keys for this. Could you use that instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@mattr-
Owner

Looks great so far. Thanks! :heart:

jekyll.gemspec
@@ -26,6 +26,7 @@ Gem::Specification.new do |s|
s.add_runtime_dependency('liquid', "~> 2.5.2")
s.add_runtime_dependency('classifier', "~> 1.3")
s.add_runtime_dependency('listen', "~> 1.3")
+ s.add_runtime_dependency('html-pipeline', "~> 1.0.0")
@parkr Owner
parkr added a note

If it's not a default, I'd prefer to keep it as a specify-it-if-you-want-it gem.

@gjtorikian Collaborator

Fair enough. Changing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@parkr
Owner

Yay!

@mattr-
Owner

@gjtorikian Were you planning on finishing this out or should we just consider it as it currently is and work on the remaining items in the task list later?

@parkr
Owner

@mattr- We're going to get custom markdown processors working such that this could be a plugin for the future. Thoughts?

@gjtorikian
Collaborator

@mattr- Yeah, I'm kind of waiting for #1213 to land first. I may just cherry-pick those commits if @envygeeks doesn't get to it first!

@mattr-
Owner

:+1: all around for custom markdown processors, etc. :smiley:

@gjtorikian
Collaborator

Shut down in favor of #1871

@gjtorikian gjtorikian closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
4 jekyll.gemspec
@@ -41,8 +41,12 @@ Gem::Specification.new do |s|
s.add_development_dependency('cucumber', "~> 1.3")
s.add_development_dependency('RedCloth', "~> 4.2")
s.add_development_dependency('kramdown', "~> 1.2")
+ s.add_development_dependency('html-pipeline', "~> 1.0.0")
+ s.add_development_dependency('github-markdown', "~> 0.6.3")
s.add_development_dependency('rdiscount', "~> 1.6")
s.add_development_dependency('launchy', "~> 2.3")
+ s.add_development_dependency('sanitize', "~> 2.0.6")
+ s.add_development_dependency('gemoji', "~> 1.5.0")
s.add_development_dependency('simplecov', "~> 0.7")
s.add_development_dependency('simplecov-gem-adapter', "~> 1.0.1")
s.add_development_dependency('coveralls', "~> 0.7.0")
View
4 lib/jekyll/converters/markdown.rb
@@ -9,6 +9,8 @@ class Markdown < Converter
def setup
return if @setup
@parser = case @config['markdown']
+ when 'html_pipeline'
+ HTMLPipelineParser.new @config
when 'redcarpet'
RedcarpetParser.new @config
when 'kramdown'
@@ -19,7 +21,7 @@ def setup
MarukuParser.new @config
else
STDERR.puts "Invalid Markdown processor: #{@config['markdown']}"
- STDERR.puts " Valid options are [ maruku | rdiscount | kramdown | redcarpet ]"
+ STDERR.puts " Valid options are [ maruku | rdiscount | kramdown | redcarpet | html_pipeline ]"
raise FatalException.new("Invalid Markdown process: #{@config['markdown']}")
end
@setup = true
View
58 lib/jekyll/converters/markdown/html_pipeline_parser.rb
@@ -0,0 +1,58 @@
+module Jekyll
+ module Converters
+ class Markdown
+ class HTMLPipelineParser
+ def initialize(config)
+ require 'html/pipeline'
+ @config = config
+ @errors = []
+ end
+
+ def filter_key(s)
+ s.to_s.downcase.to_sym
+ end
+
+ def is_filter?(f)
+ f < HTML::Pipeline::Filter
+ rescue LoadError, ArgumentError
+ false
+ end
+
+ def ensure_default_opts
+ @config['html_pipeline']['filters'] ||= ['markdownfilter']
+ @config['html_pipeline']['context'] ||= {'gfm' => true}
+ # symbolize strings as keys, which is what HTML::Pipeline wants
+ @config['html_pipeline']['context'] = @config['html_pipeline']['context'].symbolize_keys
+ end
+
+ def setup
+ unless @setup
+ ensure_default_opts
+
+ filters = @config['html_pipeline']['filters'].map do |f|
+ if is_filter?(f)
+ f
+ else
+ key = filter_key(f)
+ begin
+ filter = HTML::Pipeline.constants.find { |c| c.downcase == key }
+ HTML::Pipeline.const_get(filter)
+ rescue Exception => e
+ raise FatalException.new(e)
+ end
+ end
+ end
+
+ @parser = HTML::Pipeline.new(filters, @config['html_pipeline']['context'])
+ @setup = true
+ end
+ end
+
+ def convert(content)
+ setup
+ @parser.to_html(content)
+ end
+ end
+ end
+ end
+end
View
47 test/test_html_pipeline.rb
@@ -0,0 +1,47 @@
+# encoding: UTF-8
+
+require 'helper'
+
+class TestHTMLPipeline < Test::Unit::TestCase
+ context "HTMLPipeline" do
+ setup do
+ @config = {
+ 'markdown' => 'html_pipeline',
+ 'html_pipeline' => {
+ 'filters' => ['markdownfilter', 'sanitizationfilter', 'emojifilter', 'mentionfilter'],
+ 'context' => {
+ 'asset_root' => "http://foo.com/icons",
+ 'base_url' => "https://github.com/",
+ }
+ }
+ }
+
+ @config = Jekyll.configuration(@config)
+ @markdown = Converters::Markdown.new(@config)
+ end
+
+ should "pass regular options" do
+ assert_equal "<h1>Some Header</h1>", @markdown.convert('# Some Header #').strip
+ end
+
+ should "pass rendering emoji" do
+ assert_equal "<p><img class=\"emoji\" title=\":trollface:\" alt=\":trollface:\" src=\"http://foo.com/icons/emoji/trollface.png\" height=\"20\" width=\"20\" align=\"absmiddle\"></p>", @markdown.convert(':trollface:').strip
+ end
+
+ should "pass rendering mentions" do
+ assert_equal "<p><strong>Hey, <a href=\"https://github.com/mojombo\" class=\"user-mention\">@mojombo</a></strong>!</p>", @markdown.convert('**Hey, @mojombo**!').strip
+ end
+
+ should "fail when a library dependency is not met" do
+ override = { 'html_pipeline' => { 'filters' => ['markdownfilter, AutolinkFilter'] } }
+ markdown = Converters::Markdown.new(@config.deep_merge(override))
+ # assert_fail markdown.convert('http://www.github.com')
+ end
+
+ should "fail when a context dependency is not met" do
+ override = { 'html_pipeline' => { 'context' => {} } }
+ markdown = Converters::Markdown.new(@config.deep_merge(override))
+ # assert_fail markdown.convert(':trollface:')
+ end
+ end
+end
Something went wrong with that request. Please try again.