Doesn't work well with Obsidian's tags

[kankaristo]

Description

Some of mdl's rules don't work well with the tags that are used in Obsidian.

Environment

MDL Version: 0.11.0

Expected Behavior

Obsidian uses # for tags (which is a bit dangerous, since # is also used for headings, but that's what they've chosen to use). Tags look like this:

• #tag
• #some-other-tag

The tags are # followed by anything alphanumeric (but can't be all numbers), with - or _ as a separator.

If I have a Markdown file like this:

# Heading 1

## Heading 2

- #tag Something!

### Heading 3

- Something else!

I would expect it to pass (it's legal Markdown and uses a tag).

Actual Behavior

However, I get the following rules failing:

test.md:7: MD001 Header levels should only increment by one level at a time
test.md:5: MD026 Trailing punctuation in header

So, #tag is parsed as a heading, even though it's not one. If the tag is removed, the file passes fine.

I know that tags aren't a "standard" Markdown thing, but as far as I know, they're not illegal Markdown either, and Obsidian has a growing user base (it's still pre-v1).

Tags like this could also appear in "regular" text outside of their use in Obsidian, if you're mentioning a channel on IRC/Slack/Discord, or a hashtag on social media.

I'm aware of MD018, but I've never used a Markdown parser/editor that actually allows headings like that.

Replication Case

Create a Markdown file, name it test.md:

# Heading 1

## Heading 2

- #tag Something!

### Heading 3

- Something else!

Then run: mdl test.md

[ajaust]

This might be something to report to kramdown the markdown parser used by markdownlint.

When I parse your markdown file by kramdown and print it to the screen as HTML via a ruby script

#!/usr/bin/env ruby

require 'kramdown'

puts Kramdown::Document.new("# Heading 1

## Heading 2

- #tag Something!

### Heading 3

- Something else!").to_html

I get the following output:

<h1 id="heading-1">Heading 1</h1>

<h2 id="heading-2">Heading 2</h2>

<ul>
  <li>
    <h1 id="tag-something">tag Something!</h1>
  </li>
</ul>

<h3 id="heading-3">Heading 3</h3>

<ul>
  <li>Something else!</li>
</ul>

This means that kramdown incorrectly identifies this tag as a headline.

The parsing seems to work normally if #tag is not the first work in the line.

When I add an empty HTML comment

- <!---->#tag Something!

the warning goes away, but the generated HTML content contains an extra paragraph

<ul>
  <li>
    <!---->
    <p>#tag Something!</p>
  </li>
</ul>

After adding another word in the list entry in front of the tag

- text #tag Something!

generates the expected HTML list

<ul>
  <li>Text #tag Something!</li>
</ul>

In both cases, markdownlint does not report any errors.

[kankaristo]

I submitted an issue for Kramdown.

But after reading about Kramdown's syntax (which is a bit different from "regular" Markdown), especially the part about lists, which apparently allow nesting headers inside list items, I'm afraid that this might be "intended behavior", so it might not be fixed. I'm happy with "standard" Markdown, any "additions" on top of that just complicate things, IMO...

Unfortunately, if this won't be fixed, it would mean that I can't really use Markdownlint, because I very often start list items with a tag (which makes the tag much more visible for long list items, and aligns them nicely if multiple list items use the same tag).

I guess I could write a wrapper script, which would replace all occurrences of - # with - \# before passing the input to Markdownlint (maybe I'll write an Obsidian plugin for Markdownlint while I'm at it, right now I'm linting "manually").

EDIT:
Since I mostly care about linting Obsidian's flavor of Markdown (which is pretty close to "standard" Markdown), and I generally dislike atx style headers without a space after the hash character (i.e. #Header in place of # Header), I guess it would be better if I were to add \ in front of all occurrences of #+[^ ] before giving the input to Markdownlint. But I'll cross this bridge after I hear back from the Kramdown folks...

[jaymzh]

Currently we use a "custom" flavor of markdown with Kramdown, it's the default with a few monkeypatches. But Kramdown supports GH-flavorered markdown, and I've long wanted to move us to that, as I think it's effectively the dominant one now and more flexible. DUnno if that would help in this case or not... but figured I'd mention it.

Anyway, it'd be a backward-incompatible change, so we'd have to bump a major version and it's a fair amount of surgery to undo a bunch of other things, but I think it'd clean up the code base significantly.

[kankaristo]

As I was expecting, the Kramdown issue was closed... I'm "mixing Markdown libraries" by using Markdownlint with Obsidian (which is technically true). 🤷

For now, instead of using mdl "$filename" directly, I'm doing sed -r 's/(#+[^# ])/\\\1/g' "$filename" | mdl.
It's basically escaping all atx style headers without a space after the hash(es) - which includes Obsidian's tags - before passing the input to mdl.

My current linting script, in case anyone reading is interested:

#!/bin/bash

trap 'trap - INT; exit $((128 + $(kill -l INT)))' INT

if ! command -v mdl > /dev/null 2>&1; then
    echo "Installing mdl (Markdown linter)..."
    if ! command -v gem > /dev/null 2>&1; then
        >&2 echo "ERROR: gem not found (please install Ruby, e.g. run install-rvm)"
        exit 64
    fi
    
    gem install mdl || exit 65
fi

find . -type f -name '*.md' \
    -not -path './templates/*' \
    -not -name 'UNLINKED FILES.md' \
    -not -name 'UNRESOLVED LINKS.md' \
    | while read -r filename; \
    do \
        output="$(sed -r 's/(#+[^# ])/\\\1/g' "$filename" | mdl | grep -v -e 'docs/RULES.md' | sort -h)"; \
        [ -n "$output" ] && echo "$filename: $output" && echo; \
    done

I'm also removing the line with A detailed description of the rules is available at [...]/docs/RULES.md, and I'm sorting the output by line number (instead of by rule). It would be nice to have options for both of those things.

The above works well enough for my purposes, but I'll leave this issue open, if you're planning to move to GitHub Flavored Markdown at some point (where the list item case isn't a problem, which is mostly what I care about).

[pjkaufman]

@kankaristo , there is a plugin that is working on the linting and correction of inconsistencies in an Obsidian vault. It is called https://github.com/platers/obsidian-linter. As a disclaimer, I am helping maintain it.

It is not entirely feature comparable with markdownlint but it is trying to get to the point where it includes all fixable rules.

[kankaristo]

@pjkaufman, looks interesting, I'll check it out!

Although, based on the GIF in the readme, it looks like it automatically formats, instead of "just" linting? I would prefer for a linter to only show me what's wrong (like markdownlint), instead of also automatically formatting the files (like obsidian-linter seems to do). Without automatic formatting, it's much easier to see which rules I want and don't want, and the automatic "fix" might not always be the correct one (even if the warning/error might be identified correctly).

[pjkaufman]

Gotcha. That makes sense. It may be something to add in the future would be a setting to allow it to use a yellow underline in a warning mode and have another option to allow for automatically fixing the warnings.