Don't require <input>; Find all supported files in cwd automatically

[fregante]

Lychee supports a lot of files, I'd rather not have to maintain my own glob list. Can it just find and check all the supported filetypes like Prettier does for example?

[mre]

Basic support for this will be merged together with #330. It provides directory support and will check html and markdown by default. We want to be a bit conservative in the beginning as we're not sure if everyone likes to have their txt files checked with the fuzzy parser. There will probably be an option to overwrite this. Something like --extensions md,html,txt or so.

# Check all Markdown and HTML files recursively
lychee .
# Check Text, JSON, YAML as well
lychee . --extensions md,html,txt,json,yaml

[mre]

The question is if we also want to test the current directory when omitting the ..
So

lychee

would be the same as

lychee .

I can see good arguments for both. Opinions?

[mre]

I thought about it and decided not to test the current directory when omitting the . and instead show the usage instructions.

[mre]

With #330 being merged, lychee . will check all markdown and html files recursively by default. The missing piece to close this issue is to allow overwriting the extensions with --extensions md,html,txt.

[fregante]

Including all the supported extensions by default makes more sense to me, with the possibility to override the list with the flag if someone wants fewer extensions. This is because

• Unless I continuously keep track of lychee's supported filetypes I'll miss out on them
• I'd exclusively have use the flag to disable some extensions
• Choosing fewer extensions means the flag will always be shorter than "enabling more extensions", because it's more likely to be used with 1-2 extensions instead of "the whole list"

[mre]

Lychee can extract links from plaintext documents, so technically it could work with files like YAML or XML as well. Even files without an extension might be fine (e.g. README). I wonder if these files should be included by default and if not, where do we draw the line?

[lebensterben]

@mre
ripgrep has an algorithm to detect whether a file is binary or text.
lychee should use similar algorithm and support all text by default.

[mre]

Yeah, doing what ripgrep does sounds like a sane default. In the end, lychee is like a tiny ripgrep with a different objective. 😉 On the other hand I'm not sure if people would be interested in broken links inside SVG and XLS files. If you have a link to the algorithm I'd love to have a look.

[lebensterben]

@mre
I agree that we should skip images, videos, and audios.
That is to say, we should skip all binary files plus certain categories. In terminology of XDG shared-mimeinfo, we should filter out files with image/, audio/, video/*.

[mre]

To summarize:

• Support overwriting extensions (lychee . --extensions md,html,txt,json,yaml would only check these extensions)
• Check all text files by default. Do what ripgrep does to detect them.
• Exclude media files (images, videos, and audios)