Refactor highlighter to allow for configuration

[hexid]

I have been working on adding highlighter settings to _config.yml for the past few days. So far I have been able to set default configurations for both Liquid and RedCarpet highlight blocks.

Currently, I am using the following structure in my _config.yml file to add inline line numbers to all highlight blocks:

tags:
  highlight:
    linenos: true

All of the following are affected by the prior configuration (assuming RedCarpet as the markdown engine for the first two):

```ruby
puts "Hello world!"
```

~~~ruby
puts "Hello world!"
~~~

{% highlight ruby %}
puts "Hello world!"
{% endhighlight %}

This, however, is not limited to the linenos options and can also be used with other options that affect the highlight tag, such as hl_lines and linenostart. Granted, this is assuming that you are using pygments as your highlighter.

If anyone could chime in about the below points, along with any concerns you might have, that would be greatly appreciated.

• Moved highlight code to lib/jekyll/highlighter.rb
• Allow configuration of Liquid HighlightBlock
• Allow configuration of RedCarpet markdown highlighting (ref. The linenos support while using ``` #1887)
• Add tests to make sure everything works
• Add documentation to the site for these changes
• Get feedback on configuration structure

[parkr]

Hey! Thanks for the pull request. Always great to see new avatars around 👊

adding highlighter settings to _config.yml

I'm not sure I entirely understand the motivations behind this nor agree with those I can infer. Would you care to expand on your reasoning behind this?

/cc @benbalter, the configuration 🎄 cop.

[hexid]

@parkr What I meant by this was that you can provide default configuration values that will be passed to pygments or rouge.

Currently, there is no way to override these options when using RedCarpet's fenced_code_block because they do not allow for options beyond the language for the codeblock.

When using the Liquid HighlightBlock, it is possible to override these options, so that, in the case provided above, all codeblocks will be assigned line numbers and can then be disabled individually.

An example of this would be the following, in which the first code block will not have line numbers, while the rest will:

_config.yml

markdown: redcarpet
highlighter: pygments
tags:
  highlight:
    linenos: true

index.md

{% highlight ruby linenos=false %}
puts "No line numbers!"
{% endhighlight %}

{% highlight ruby %}
puts "Has line numbers!"
{% endhighlight %}

{% highlight ruby linenos=table %}
puts "Tabled line numbers!"
{% endhighlight %}

~~~ ruby
puts "Has line numbers!"
puts "They can't be individually disabled :("
~~~

[doktorbro]

@hexid Does the refactoring tackle this issue rouge-ruby/rouge#118 too?

[hexid]

@penibelst I was able to add support for both Pygments and Rouge while using Kramdown

@parkr Should I include these changes in this PR or prep another with those changes?

[jneen]

👏 👏 👏

[jneen]

My only concern would be the Makefile hack that we had to add in the redcarpet plugin. I'm not sure if Kramdown has the same behavior, but redcarpet turns all leading tabs into 4 spaces, which completely breaks Makefile syntax. So if that happens (i.e. lexer.tag == 'make'), we simply did a gsub(/^[ ][ ][ ][ ]/, "\t") on the input text.

[ghost]

I'm not sure if Kramdown has the same behavior, but redcarpet turns all
leading tabs into 4 spaces, which completely breaks Makefile syntax.

@jneen not necessarily

Makefiles without tab characters

[ghost]

@parkr was this fixed?

[parkr]

No. Having read through the comments again, I'm not convinced it's worth the trouble/maintenance costs/configuration overhead.

[jneen]

No worries. It turns out the makefile hack isn't necessary in Kramdown anyways, and they have rouge support now.

[parkr]

they have rouge support now.

Ok cool. What version added that in? We doing the right things to make this available?

[jneen]

I'm not sure... let me dig it up

[jneen]

gettalong/kramdown#141 (comment)