Check for broken full reference links in Markdown

[norswap]

Currently lychee does not check for bad "link references" (not sure if this is the proper name), e.g.

This is a [link text][link-ref].

[link-ref-with-typo]: https://nyan.cat

It would be great if that was caught!

[lebensterben]

First, this is not the correct syntax for link reference. See
https://github.github.com/gfm/#link-reference-definitions

It should be

[foo]: /url "title"

[foo]

Second, we relies on pulldown-cmark to parse hyperlinks in markdown documents.
Thankfully, it's able to find broken links via https://docs.rs/pulldown-cmark/latest/pulldown_cmark/struct.Parser.html#method.new_with_broken_link_callback

But if you want to identify possible 'typo's, the job would be harder as we may need to calculate some text distances.

[mre]

Oh, I did not know about new_with_broken_link_callback! Thanks for mentioning it.
We could start by printing a warning for broken links? That would already be a step into the right direction.

[norswap]

@lebensterben These types of links have been in Markdown since the very start (https://daringfireball.net/projects/markdown/syntax#link) and have always been supported by Github. They are in fact specified in the document you linked (https://github.github.com/gfm/#example-535). What you linked is the reference for the link definition (the [ref]: link part).

@mre A warning meaning it does not cause the program to return a non-zero code?

[lebensterben]

@norswap
Thanks for pointing that out. It's a valid full reference link. https://github.github.com/gfm/#full-reference-link

I suggest to change the title of this issue accordingly.

[mre]

@mre A warning meaning it does not cause the program to return a non-zero code?

Good point. We can actually treat it as an error. After all, the link is broken.

[nuke-web3]

Would love to see this supported https://spec.commonmark.org/0.31.2/#full-reference-link has the full variety of syntax that should/must be supported.

Example of another link checker that uses new_with_broken_link_callback for reference: https://github.com/becheran/mlc/blob/b0cb310fda856cf4a7734bfa6bca20029ffcf89b/src/link_extractors/markdown_link_extractor.rs#L12-L20 (incomplete impl though )

In minimal testing so far, I can see that any [text][id] in the main text without a matching [id]: ... in the input will hit the callback, but will completely skip any unreferenced [not-used-in-doc]: ... reference. Not sure if that is the behavior we want... as a lint I would love to get a warning if that is the case as I probably meant to use the link or deleted where it was used. Checking if the link works would be good too (perhaps as a CLI/config option)