A general purpose syntax highlighter in pure Go
Clone or download
Jos512 and alecthomas List lexers in README in table (#176)
This makes it easier to scan through the line and add new lexers in the future.
Latest commit a24ed6d Sep 21, 2018
Permalink
Failed to load latest commit information.
_tools Fix bug with nested newlines. Mar 2, 2018
cmd/chroma Add option to prevent surroundign pre tags Sep 18, 2018
formatters Add option to prevent surroundign pre tags Sep 18, 2018
lexers Add test for Bash lexer (#172) Sep 18, 2018
quick quick: add an example (#70) Oct 22, 2017
styles Assign .gl class to GenericUnderline; add CSS rules for the same Aug 1, 2018
.gitignore Remove binary file and add to gitignore (#72) Oct 22, 2017
.goreleaser.yml Fix goreleaser config (hopefully). Sep 25, 2017
.travis.yml Use goreleaser. Sep 25, 2017
COPYING Switch to github.com/dlclark/regexp2. Sep 18, 2017
README.md List lexers in README in table (#176) Sep 21, 2018
coalesce.go Fix DelegatingLexer. Mar 18, 2018
coalesce_test.go Invert default "ensure newline" behaviour so that it is opt-in. Sep 30, 2017
colour.go Fix nearest colour matching. Jun 25, 2018
colour_test.go Ensure a newline exists at the end of files. Sep 29, 2017
delegate.go Fix DelegatingLexer. Mar 18, 2018
delegate_test.go Implemented delegating lexer. Mar 18, 2018
doc.go Fix typo: translater → translator (#79) Oct 26, 2017
formatter.go Document and add iterator panic recovery. Sep 20, 2017
iterator.go Document and add iterator panic recovery. Sep 20, 2017
lexer.go Add DelegatingLexer. Mar 17, 2018
lexer_test.go Ensure a newline exists at the end of files. Sep 29, 2017
mutators.go Fix DelegatingLexer. Mar 18, 2018
mutators_test.go Invert default "ensure newline" behaviour so that it is opt-in. Sep 30, 2017
pygments-lexers.txt Rename the Nimrod lexer to Nim Sep 25, 2017
regexp.go Fix Dollar-Quoted Strings (postgres + cql) Jun 12, 2018
regexp_test.go Add test for EnsureNL. Oct 12, 2017
remap.go Add support for Go templates. Mar 18, 2018
remap_test.go Invert default "ensure newline" behaviour so that it is opt-in. Sep 30, 2017
style.go Big changes to the style and colour APIs. Sep 23, 2017
style_test.go Ensure a newline exists at the end of files. Sep 29, 2017
tokentype_string.go Regenerate TokenType stringer (#60) Oct 13, 2017
types.go Assign .gl class to GenericUnderline; add CSS rules for the same Aug 1, 2018

README.md

Chroma - A general purpose syntax highlighter in pure Go Build Status Gitter chat

NOTE: As Chroma has just been released, its API is still in flux. That said, the high-level interface should not change significantly.

Chroma takes source code and other structured text and converts it into syntax highlighted HTML, ANSI-coloured text, etc.

Chroma is based heavily on Pygments, and includes translators for Pygments lexers and styles.

Table of Contents

  1. Supported languages
  2. Using the library
    1. Quick start
    2. Identifying the language
    3. Formatting the output
    4. The HTML formatter
  3. More detail
    1. Lexers
    2. Formatters
    3. Styles
  4. Command-line interface
  5. What's missing compared to Pygments?

Supported languages

Prefix Language
A ABNF, ANTLR, APL, ActionScript, ActionScript 3, Ada, Angular2, ApacheConf, AppleScript, Awk
B BNF, Ballerina, Base Makefile, Bash, Batchfile, BlitzBasic, Brainfuck
C C, C#, C++, CFEngine3, cfstatement, CMake, COBOL, CSS, Cap'n Proto, Ceylon, ChaiScript, Cheetah, Clojure, CoffeeScript, Common Lisp, Coq, Crystal, Cython
D DTD, Dart, Diff, Django/Jinja, Docker
E EBNF, Elixir, Elm, EmacsLisp, Erlang
F FSharp, Factor, Fish, Forth, Fortran
G GAS, GDScript, GLSL, Genshi, Genshi HTML, Genshi Text, Gnuplot, Go, Go HTML Template, Go Text Template, Groovy
H HTML, HTTP, Handlebars, Haskell, Haxe, Hexdump, Hy
I INI, Idris, Io
J JSON, JSX, Java, JavaScript, Julia
K Kotlin
L LLVM, Lighttpd configuration file, Lua
M Mako, Markdown, Mason, Mathematica, MiniZinc, Modula-2, MorrowindScript, MySQL, Myghty
N NASM, Newspeak, Nginx configuration file, Nim, Nix
O OCaml, Objective-C, Octave, Org Mode
P PHP, PL/pgSQL, POVRay, PacmanConf, Perl, Pig, PkgConfig, PostScript, PostgreSQL SQL dialect, PowerShell, Prolog, Protocol Buffer, Puppet, Python, Python 3
Q QBasic
R R, Racket, Ragel, reg, reStructuredText, Rexx, Ruby, Rust
S SCSS, SPARQL, SQL, Sass, Scala, Scheme, Scilab, Smalltalk, Smarty, Snobol, Solidity, SquidConf, Swift, Systemverilog
T TASM, TOML, Tcl, Tcsh, TeX, Termcap, Terminfo, Terraform, Thrift, TradingView, Transact-SQL, Turtle, Twig, TypeScript, TypoScript, TypoScriptCssData, TypoScriptHtmlData
V verilog, VHDL, VimL
W WDTE
X XML, Xorg
Y YAML

I will attempt to keep this section up to date, but an authoritative list can be displayed with chroma --list.

Using the library

Chroma, like Pygments, has the concepts of lexers, formatters and styles.

Lexers convert source text into a stream of tokens, styles specify how token types are mapped to colours, and formatters convert tokens and styles into formatted output.

A package exists for each of these, containing a global Registry variable with all of the registered implementations. There are also helper functions for using the registry in each package, such as looking up lexers by name or matching filenames, etc.

In all cases, if a lexer, formatter or style can not be determined, nil will be returned. In this situation you may want to default to the Fallback value in each respective package, which provides sane defaults.

Quick start

A convenience function exists that can be used to simply format some source text, without any effort:

err := quick.Highlight(os.Stdout, someSourceCode, "go", "html", "monokai")

Identifying the language

To highlight code, you'll first have to identify what language the code is written in. There are three primary ways to do that:

  1. Detect the language from its filename.

    lexer := lexers.Match("foo.go")
  2. Explicitly specify the language by its Chroma syntax ID (a full list is available from lexers.Names()).

    lexer := lexers.Get("go")
  3. Detect the language from its content.

    lexer := lexers.Analyse("package main\n\nfunc main()\n{\n}\n")

In all cases, nil will be returned if the language can not be identified.

if lexer == nil {
  lexer = lexers.Fallback
}

At this point, it should be noted that some lexers can be extremely chatty. To mitigate this, you can use the coalescing lexer to coalesce runs of identical token types into a single token:

lexer = chroma.Coalesce(lexer)

Formatting the output

Once a language is identified you will need to pick a formatter and a style (theme).

style := styles.Get("swapoff")
if style == nil {
  style = styles.Fallback
}
formatter := formatters.Get("html")
if formatter == nil {
  formatter = formatters.Fallback
}

Then obtain an iterator over the tokens:

contents, err := ioutil.ReadAll(r)
iterator, err := lexer.Tokenise(nil, string(contents))

And finally, format the tokens from the iterator:

err := formatter.Format(w, style, iterator)

The HTML formatter

By default the html registered formatter generates standalone HTML with embedded CSS. More flexibility is available through the formatters/html package.

Firstly, the output generated by the formatter can be customised with the following constructor options:

  • Standalone() - generate standalone HTML with embedded CSS.
  • WithClasses() - use classes rather than inlined style attributes.
  • ClassPrefix(prefix) - prefix each generated CSS class.
  • TabWidth(width) - Set the rendered tab width, in characters.
  • WithLineNumbers() - Render line numbers (style with LineNumbers).
  • HighlightLines(ranges) - Highlight lines in these ranges (style with LineHighlight).
  • LineNumbersInTable() - Use a table for formatting line numbers and code, rather than spans.

If WithClasses() is used, the corresponding CSS can be obtained from the formatter with:

formatter := html.New(html.WithClasses())
err := formatter.WriteCSS(w, style)

More detail

Lexers

See the Pygments documentation for details on implementing lexers. Most concepts apply directly to Chroma, but see existing lexer implementations for real examples.

In many cases lexers can be automatically converted directly from Pygments by using the included Python 3 script pygments2chroma.py. I use something like the following:

python3 ~/Projects/chroma/_tools/pygments2chroma.py \
  pygments.lexers.jvm.KotlinLexer \
  > ~/Projects/chroma/lexers/kotlin.go \
  && gofmt -s -w ~/Projects/chroma/lexers/*.go

See notes in pygments-lexers.go for a list of lexers, and notes on some of the issues importing them.

Formatters

Chroma supports HTML output, as well as terminal output in 8 colour, 256 colour, and true-colour.

A noop formatter is included that outputs the token text only, and a tokens formatter outputs raw tokens. The latter is useful for debugging lexers.

Styles

Chroma styles use the same syntax as Pygments.

All Pygments styles have been converted to Chroma using the _tools/style.py script.

For a quick overview of the available styles and how they look, check out the Chroma Style Gallery.

Command-line interface

A command-line interface to Chroma is included. It can be installed with:

go get -u github.com/alecthomas/chroma/cmd/chroma

What's missing compared to Pygments?

  • Quite a few lexers, for various reasons (pull-requests welcome):
    • Pygments lexers for complex languages often include custom code to handle certain aspects, such as Perl6's ability to nest code inside regular expressions. These require time and effort to convert.
    • I mostly only converted languages I had heard of, to reduce the porting cost.
  • Some more esoteric features of Pygments are omitted for simplicity.
  • Though the Chroma API supports content detection, very few languages support them. I have plans to implement a statistical analyser at some point, but not enough time.