Fast, extensible, server-side code highlighting

TODO:

• Add JS support
• Add SQL support
• Add Twig support
• Add YAML support
• Add JSON support

Quickstart

composer require tempest/highlight:dev-main

Highlight code like this:

$highlighter = new \Tempest\Highlight\Highlighter();

$code = $highlighter->parse($escapedCode, 'php');

Note: you should always pass the escaped version of your code:

$code = $highlighter->parse(htmlentities($raw), 'php');

Next, you can use one of the provided themes via a CSS import:

@import "../vendor/tempest/highlight/src/Themes/highlight-light-lite.css";

Or you can build your own with just a couple of classes:

.hl-keyword {
    color: #4F95D1;
}

.hl-property {
    color: #46b98d;
}

.hl-attribute {
    font-style: italic;
}

.hl-type {
    color: #D14F57;
}

.hl-generic {
    color: #9D3AF6;
}

.hl-comment {
    color: #888888;
}

.hl-blur {
    filter: blur(2px);
}

.hl-strong {
    font-weight: bold;
}

.hl-em {
    font-style: italic;
}

You should style <pre> tags yourself.

Special highlight tags

You can add these tags within your code to emphasize or blur parts:

• {_ content _} adds the .hl-em class
• {* content *} adds the .hl-strong class
• {~ content ~} adds the .hl-blur class

{~public function parse(string $content, Highlighter $highlighter): string
{
    $pattern = '/\{\~(?<match>(.|\n)*)\~\}/';
    
    preg_match($pattern, $content, $matches);

    if ($matches === []) {
        return $content;
    }~} // This part is blurred

    {*$content = preg_replace_callback(*} // This line is bold
        $pattern,
        function (array $matches) use ($highlighter) {
            $parsed = $highlighter->parse($matches['match'], $highlighter->getCurrentLanguage());
            
            return '<span class="hl-blur">' . $parsed . '</span>';
        },
        {_$content_} // This line is cursive
    );
    
    {~return $highlighter->parse($content, $highlighter->getCurrentLanguage());
}~}

This is the end result:

CommonMark integration

If you're using league/commonmark, you can add highlight support to codeblocks like so:

use League\CommonMark\Environment\Environment;
use League\CommonMark\Extension\CommonMark\CommonMarkCoreExtension;
use League\CommonMark\Extension\CommonMark\Node\Block\FencedCode;
use League\CommonMark\MarkdownConverter;
use Tempest\Highlight\CommonMark\HighlightCodeBlockRenderer;

$environment = new Environment();

$environment
    ->addExtension(new CommonMarkCoreExtension())
    ->addRenderer(FencedCode::class, new HighlightCodeBlockRenderer());

$markdown = new MarkdownConverter($environment);

Keep in mind that you need to manually install league/commonmark:

composer require league/commonmark;

Language support

This package makes it easy for developers to add new languages or extend existing languages. Right now, these languages are supported: php, html, css, and blade. More will be added.

In order to build your own highlighter functionality, you need to understand three concepts of how code is highlighted: patterns, injections, and languages.

1. Patterns

A pattern represents part of your code that should be highlighted. A pattern can target a single keyword like return or class, or it could be any part of your code, like for example a comment: /* this is a comment */ or an attribute: #[Get(uri: '/')].

Each pattern is represented by a simple class that provides a regex pattern, and a TokenType. The regex pattern is used to match relevant content to this specific pattern, while the TokenType is an enum value that will determine how that specific pattern is colored.

Here's an example of a simple pattern to match the namespace of a PHP file:

use Tempest\Highlight\IsPattern;
use Tempest\Highlight\Pattern;
use Tempest\Highlight\Tokens\TokenType;

final readonly class NamespacePattern implements Pattern
{
    use IsPattern;

    public function getPattern(): string
    {
        return 'namespace (?<match>[\w\\\\]+)';
    }

    public function getTokenType(): TokenType
    {
        return TokenType::TYPE;
    }
}

Note that each pattern must include a regex capture group that's named match. The content that matched within this group will be highlighted.

For example, this regex namespace (?<match>[\w\\\\]+) says that every line starting with namespace should be taken into account, but only the part within the named group (?<match>…) will actually be colored. In practice that means that the namespace name matching [\w\\\\]+, will be colored.

Yes, you'll need some basic knowledge of regex. Head over to https://regexr.com/ if you need help, or take a look at the existing patterns in this repository.

In summary:

• Patterns provide a regex that matches parts of your code
• Those regexes should contain a group named match, which is written like so (?<match>…)
• Finally, a pattern provides a TokenType, which is used to determine the highlight style for the specific match

2. Injections

Once you've understood patterns, the next step is to understand injections. Injections are used to highlight different languages within one code block. For example: HTML could contain CSS, which should be styled properly as well.

An injection class will tell the highlighter that it should treat a block of code as a different language. For example:

<div>
    <x-slot name="styles">
        <style>
            body {
                background-color: red;
            }
        </style>
    </x-slot>
</div>

Everything within these <style></style> tags should be treated as CSS. That's done by this class:

use Tempest\Highlight\Highlighter;
use Tempest\Highlight\Injection;
use Tempest\Highlight\IsInjection;

final readonly class CssInjection implements Injection
{
    use IsInjection;

    public function getPattern(): string
    {
        return '&lt;style&gt;(?<match>(.|\n)*)&lt;\/style&gt;';
    }

    public function parseContent(string $content, Highlighter $highlighter): string
    {
        return $highlighter->parse($content, 'css');
    }
}

Just like patterns, an injection must provide a pattern. This pattern, for example, will match anything between style tags: &lt;style&gt;(?<match>(.|\n)*)&lt;\/style&gt;.

Keep in mind that we're always dealing with escaped code!

The second step in providing an injection is to parse the matched content into another language. That's what the parseContent method is for. In this case, we'll get all the code between the style tags that was matched with the named (?<match>…) group, and parse that content as CSS instead of whatever language we're currently dealing with.

In summary:

• Injections provide a regex that matches a blob of code of language A, while in language B
• Just like patterns, injection regexes should contain a group named match, which is written like so (?<match>…)
• Finally, an injection will use the highlighter to parse its matched content into another language

3. Languages

The last concept to understand, although it doesn't mean much. Languages are classes that bring these two concepts together. They are nothing more than a collection of patterns and injections. Take a look at the HtmlLanguage, for example:

class HtmlLanguage implements Language
{
    public function getInjections(): array
    {
        return [
            new PhpInjection(),
            new PhpShortEchoInjection(),
            new CssInjection(),
        ];
    }

    public function getPatterns(): array
    {
        return [
            new OpenTagPattern(),
            new CloseTagPattern(),
            new TagAttributePattern(),
            new HtmlCommentPattern(),
        ];
    }
}

This HtmlLanguage class specifies the following things:

• PHP can be injected within HTML, both with the short echo tag <?= and longer <?php tags
• CSS can be injected as well, JavaScript support is still work in progress
• There are a bunch of patterns to highlight HTML tags properly

So, let's bring everything together to explain how you can add your own languages.

Adding custom languages

Let's say you're adding Blade support. You could create a plain language file and start from there, but it'd probably be easier to extend an existing language, HtmlLanguage is probably the best. Let create a new BladeLanguage class that extends from HtmlLanguage:

class BladeLanguage extends HtmlLanguage
{
    public function getInjections(): array
    {
        return [
            ...parent::getInjections(),
        ];
    }

    public function getPatterns(): array
    {
        return [
            ...parent::getPatterns(),
        ];
    }
}

With this class in place, we can start adding our own patterns and injections. Let's start with adding a pattern that matches all Blade keywords, which are always prepended with the @ sign. Let's add it:

final readonly class BladeKeywordPattern implements Pattern
{
    use IsPattern;

    public function getPattern(): string
    {
        return '(?<match>\@[\w]+)\b';
    }

    public function getTokenType(): TokenType
    {
        return TokenType::KEYWORD;
    }
}

And register it in our BladeLanguage class:

    public function getPatterns(): array
    {
        return [
            ...parent::getPatterns(),
            new BladeKeywordPattern(),
        ];
    }

Next, there are a couple of places within Blade where you can write PHP code: within the @php keyword, as well as within keyword brackets: @if (count(…)). Let's write two injections for that:

final readonly class BladeKeywordInjection implements Injection
{
    use IsInjection;

    public function getPattern(): string
    {
        return '(\@[\w]+)\s?\((?<match>.*)\)';
    }

    public function parseContent(string $content, Highlighter $highlighter): string
    {
        return $highlighter->parse($content, 'php');
    }
}

final readonly class BladePhpInjection implements Injection
{
    use IsInjection;

    public function getPattern(): string
    {
        return '\@php(?<match>(.|\n)*?)\@endphp';
    }

    public function parseContent(string $content, Highlighter $highlighter): string
    {
        return $highlighter->parse($content, 'php');
    }
}

Let's add these to our BladeLanguage class as well:

    public function getInjections(): array
    {
        return [
            ...parent::getInjections(),
            new BladePhpInjection(),
            new BladeKeywordInjection(),
        ];
    }

And, finally, you can write {{ … }} and {!! … !!} to echo output. Whatever is between these brackets is also considered PHP, so, one more injection:

final readonly class BladeEchoInjection implements Injection
{
    use IsInjection;

    public function getPattern(): string
    {
        return '({{|{!!)(?<match>.*)(}}|!!})';
    }

    public function parseContent(string $content, Highlighter $highlighter): string
    {
        return $highlighter->parse($content, 'php');
    }
}

With all of that in place, the only thing left to do is to add our language to the highlighter:

$highlighter->addLanguage('blade', new BladeLanguage());

And you're done! Blade support with just a handful of patterns and injections.

You're free to send pull requests with additional language support! Take a look at the tests to learn how to write tests for patterns and injections.