Add support for hiding lines in other languages

[thecodewarrior]

Does what it says on the tin. I managed to figure out an elegant and flexible solution using only regular expressions and a little bit of code, so it should be highly configurable. I documented the entire system in the guide, so I'll copy that here for ease of access.

Hiding code lines

There is a feature in mdBook that lets you hide code lines by prepending them with a # in the same way that Rustdoc does.

```rust
# fn main() {
    let x = 5;
    let y = 6;

    println!("{}", x + y);
# }
```

Will render as

# fn main() {
    let x = 5;
    let y = 7;

    println!("{}", x + y);
# }

By default, this only works for code examples that are annotated with rust. However, you can define custom prefixes for other languages by adding a new line-hiding prefix in your book.toml with the language name and prefix character (you can even do multi-character prefixes if you really want to):

[output.html.playground.line-hiding-prefixes]
python = "~"

The prefix will hide any lines that begin with the given prefix, but the prefix can be escaped using a backslash. With the python prefix shown above, this:

```python
~def fib():
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()
    ~# hide me!
    \~# leave me be!
~fib(1000)
```

will render as

~def fib():
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()
    ~# hide me!
    \~# leave me be!
~fib(1000)

If you need something more advanced than a simple prefix (e.g. rust uses # but doesn't hide #[...] lines), you can define a custom regular expression using line-hiding patterns.

Configuration

...

• line-hiding-prefixes A map of simple to use prefixes for hiding lines in various languages. (e.g. { python = "~" } will hide lines in python blocks that start with ~, but that ~ can be escaped with a backslash.)
• line-hiding-patterns A map of complex patterns to use for hiding lines in various languages.

Line-hiding patterns

Line-hiding patterns define what lines in a code block should be hidden with togglable visibility. There is already a built-in pattern for rust, which should behave identically to rustdoc. In rust, a hidden line starts with a #, however that can be escaped using a ##, and lines that start with #! (e.g. #![...]) or #[ (e.g. #[...]) won't be hidden.

[output.html.playground.line-hiding-prefixes]
python = "~"
[output.html.playground.line-hiding-patterns]
somelanguage = "<custom pattern>"

The simplest way to add a line-hiding pattern is to add a line-hiding prefix, which is really an auto-generated pattern. The generated pattern will hide any lines that begin with the specified prefix, unless it's preceded by a backslash.

While a simple prefix will almost always work, sometimes a more complex pattern is necessary (e.g. rust won't hide #[...] lines), and this is where fully-fledged line-hiding patterns kick in. These patterns are regular expressions and have to follow a few rules.

• Each pattern should match an entire line.
• The pattern should have a group named escape, and this group should be optional. Note the difference between
  the contents of the group being optional (?P<escape>#?) and the group itself being optional (?P<escape>#)?.
• Everything else that you care about should be in unnamed groups.

mdBook will then test the regex on each line and follow this process:

• If the line doesn't match at all, the line is left unchanged and isn't hidden
• If the line matches and the escape group matches, the escape group is cut out of the middle and the line isn't hidden
• If the line matches and the escape group doesn't match, the result is the combined contents of every match group.
  (This is why you add unnamed groups around everything you care about.)

Here is the pattern generated for a line-hiding prefix (the {} in replaced with the prefix string):

^(\s*)(?P<escape>\\)?{}(.*)$

Breaking it down, we have

• ^(\s*)
  Match the indentation, and put it in a group to preserve it in the hidden output.
• (?P<escape>\\)?
  If we find a backslash this group will match and trigger the escape mechanism.
• {}
  Match the prefix. Note how this isn't in a group, meaning it won't be included in the hidden line.
• (.*)$
  Match the rest of the line, and put it in a group to preserve it in the output.

A more complex example would be the Rust pattern, which is beyond the scope of this guide. Most of the changes are in the block after the # prefix, and are dedicated to ignoring #[...] and #![...] lines.

^(\s*)(?P<escape>#)?#(?: (.*)|([^#!\[ ].*))?$

[thecodewarrior]

I removed the automatic swallowing of the space immediately following basic prefixes since that's not a great assumption to make in the general case.

[dtolnay]

I find this business with regex patterns way more complicated than necessary. A much simpler approach with sane defaults would be more appealing.

Counterproposal:

Only implement prefix matching. Do not expose any escaping mechanism. Instead of an escaping mechanism or complicated regex-based thing, providing a simple local override is just as general, and easier to explain.

# book.toml defines a prefix string per language (one or more chars).
# Any line starting with whitespace+prefix is boring.
# There is no escaping.

[output.html.playground.hidelines]
python = "~"

# index.md

```python
~hidden()
~    hidden()
    ~hidden()
    nothidden()
```

Instead of escaping, there is a local override. Anything the author can solve with escaping they can solve with an override with less thinking and a clearer result for readers.

```python,hidelines=$
~nothidden()
$hidden()
    $hidden()
    nothidden()
```

[thecodewarrior]

From what you've described, your system is virtually identical to my "boring prefixes" except that your system doesn't allow escaping. Plus, your system will require special-casing rust code, whereas mine is able to handle everything with a single, elegant, system.

Boiling it down, my solution is a set of just three rules that solve the problem in an extensible, flexible, simple way:

• If the line doesn't match, the line is left unchanged
• If the line matches and the escape group matches, the output is the entire line with the escape group cut out of the middle.
• If the line matched and the escape group doesn't match, the output is the combined contents of every match group.

I get the sense that you probably don't understand regular expressions well, if at all. If that is the case then yes, my solution would probably seem complicated and confusing, but that's from lack of understanding of regular expressions, not my solution being over complicated.

These regular expressions never need to or will be seen by most users. The boring prefixes will do exactly what you're aiming for, but better.

If it would make the documentation clearer I can avoid talking about the patterns in the general documentation and just mention patterns as an aside.

One thing that might be worth doing is switching the terminology over to "hidelines" as opposed to "boring" (e.g. "hidelines prefix"). I mostly just continued using the internal terminology, but "boring prefix" is pretty ambiguous and unclear.

[thecodewarrior]

Any status update on this? I just don't want it to inadvertently fall through the cracks.

[thecodewarrior]

@ehuss You mentioned in #1370 not having much time to review, but I see you've been a bit more active in the repo lately, so if you have the time now I'd appreciate you looking this over. :)

[ehuss]

I agree with @dtolnay that this seems quite complex for such a niche feature. Can this be simplified?

I think a simple prefix character would be sufficient. Rustdoc's actual rules are:

• ##... remove the leading #, show the rest of the line.
• # is hidden (assumes a blank line)
• # ... removes the leading # and hides the line.

Why would a similar system not work for other languages?

I get the sense that you probably don't understand regular expressions well, if at all.

I would encourage you not to assume the skill level of other people. David is one of the top Rust developers in the world, and I presume understands regular expressions very well.

[thecodewarrior]

Firstly, my apologies to David. I tend to design things with a priority on flexibility, and I didn't really understand why they took issue with my solution, so I assumed their complaint was one of understanding.

Based on your description of the Rust hiding rules, I seem to have misunderstood them. I was not aware that the space was required, which makes things significantly simpler. Using a local prefix override in lieu of escaping, as David suggested, seems like an awkward solution to the problem, but I think using those rust conventions you listed as the standard would work well:

• <prefix><prefix>... remove the leading <prefix>, show the rest of the line.
• <prefix> is hidden (assumes a blank line)
• <prefix> ... removes the leading <prefix> and hides the line.

[thecodewarrior]

Closing this in favor of #1761