Intelligent support for LESSOPEN and LESSCLOSE

[lilyball]

macOS has 3 different plist formats, two of them textual (NeXTStep and XML) and one binary. The textual plists are human-readable already, but the binary one isn't, and trying to run bat on it just warns me that it's a binary file.

What I'd really like is a way to configure bat to run plists through plutil -p to get a human-readable textual output. It could note that the file has been preprocessed this way in the header. And there should be an easy way to disable this (such as a flag that says "disable file preprocessors"). This would be similar to LESSOPEN (which has the flag -L to disable it), except defined on a per–file extension basis rather than being a global option that requires the script to decide what to do (this way bat can know and display whether the file has been preprocessed).

Optionally, this plutil -p preprocessing could be done only for binary files, this way running bat on an XML plist would show the XML. I'm not sure whether this is a good idea. This approach could then mean the flag to disable preprocessing could look like --preprocess=always|binary|never, though I suppose having that as a global option is unfortunate if I want to only preprocess binary plists but always preprocess some other file type. I'm not aware of any precedent for setting per-language options though (just for mapping file globs to syntax names). Perhaps the syntax to define a preprocessor could allow for selecting whether it should only apply to binary files. Or maybe the preprocessor itself should just be a script that makes that call, though in this case bat doesn't know whether the file has been modified (which seems like a very useful thing to know for displaying in the header). Or maybe this is just overcomplicating it and a preprocessor should just be defined for a file pattern and I can use the appropriate flag to disable it if I want to see the original file.

Side note: if bat ever becomes a pager in its own right instead of relying on less it could then have an interactive option to toggle the preprocessor off or on.

Also, while I've been talking about plists, I could imagine this ability being useful for e.g. displaying metadata about images or videos, or displaying some sort of summary of sqlite files, or other similar preprocessors.

Alternatives

While a generic preprocessor feature seems useful, as an alternative bat could simply bake in knowledge of plists on macOS only (along with a macOS-only flag to disable plist transformations). It could either still just run them through plutil -p, or it could call into CoreFoundation to parse the plist directly, though I'm not aware offhand of any way to get the plutil -p output format that way without reimplementing it by hand.

[lilyball]

Another preprocessor use-case I just found: teaching bat foo.md to run mdless foo.md instead. This does mean the preprocessor definition has to declare whether bat should syntax-colorize the output (or perhaps more flexibly, what syntax it should use to colorize) or whether it should just pass the preprocessor output verbatim.

Basically, the preprocessor should act something like $cmd $args $file | bat --file-name "$file ($cmd)" by default, with an optional -l $lang arg too.

At a first pass, the syntax could be something like --map-preprocessor <glob-pattern[:syntax-name]:preprocessor>, where preprocessor is the command to use, ideally using shell word-splitting rules. If the preprocessor includes some defined token (such as %, or {}) that would be replaced with the filename, otherwise the filename would be given as an additional argument.

I could then add something like the following to my config: --map-preprocessor "*.plist:CoffeeScript:plutil -p" (the use of CoffeeScript here is because bat has no built-in plist syntax and CoffeeScript colors the => tokens, though perhaps there's a better language to use here). And then for markdown I can do --map-preprocessor '*.md:mdless" as it will provide the color codes already.

Question

For preprocessors that provide colored output, should bat strip the color codes when piping (using the same rules as its own built-in syntax coloring)?

Should we distinguish between preprocessors that only run interactively vs preprocessors that always run? I'm thinking here in the context of using bat as a drop-in replacement for cat. In this case, bat README.md would preprocess with mdless but should foo=$(bat README.md) also preprocess? Do we need a per-preprocessor config, or just a global --preprocess=auto|always|never?

Do we want to define any env vars when executing the preprocessor, so users can write preprocessor scripts that act intelligently? For example, informing the preprocessor about whether bat would otherwise show colors or decorations.

[sharkdp]

Possibly related? #1597

[Enselic]

Let's turn this into a generic issue for some kind of support for LESSOPEN and LESSCLOSE.

[lilyball]

My concern with LESSOPEN and LESSCLOSE is I might be using those to teach less about stuff that bat already handles. For example, lesspipe can do syntax highlighting. That can be useful with less, but I wouldn't want to do that with bat.

Given that, my initial thoughts are:

• Have a way to map a file extension to LESSOPEN/LESSCLOSE commands without actually using the env var. Perhaps by having a separate "preprocessor" definition group and you map file patterns to those group entries like you do for languages. This should be separate from the language definitions so you don't have to redefine the same preprocessor for a whole slew of languages.
• However this is defined, it could have a syntax that means "use the actual LESSOPEN and LESSCLOSE env vars".
• It might need to muck with the LESS env var exposed to the preprocessor based on whether the user has enabled colors in bat, so that way things like lesspipe that check LESS will behave correctly.
• Possibly separate BATOPEN and BATCLOSE env vars could be defined to enable a preprocessor regardless of filename, in case the user has a valid reason to force a preprocessor on all invocations of bat within a given environment.
• BATOPEN='$LESSOPEN' and BATCLOSE='$LESSCLOSE' could be a way to say "use LESSOPEN/LESSCLOSE for all files". This would have to be special-cased syntax of course.

The ability to configure this per file pattern is something I really want. I tend not to use LESSOPEN/LESSCLOSE because it adds a preprocessing step to every file, which I may not want and which just adds latency to viewing files (it also adds another failure mode). Being able to have an input preprocessor is great, but I should be able to opt in to it based on file pattern. This way I can not only control which files get preprocessed, but also it would then be easy to specify a different preprocessor per file instead of having to write a wrapper script to do that.

[Anomalocaridid]

Being able to have an input preprocessor is great, but I should be able to opt in to it based on file pattern. This way I can not only control which files get preprocessed, but also it would then be easy to specify a different preprocessor per file instead of having to write a wrapper script to do that.

@lilyball The batpipe script from bat-extras provides something like that, although it's still pretty much just a wrapper script. batpipe lets you run scripts for different kinds of files that you want to preprocess. It even works with less as well as bat. Although I have not actually been able to use batpipe with bat since it relies on $LESSOPEN. In fact, that's what led me to this issue.

[ImportTaste]

I noticed this issue linked in the patch notes, should this still be open?

[Enselic]

I think we can close, thanks for the reminder!

[juanmirocks]

Probably the new functionality should be documented under "Integration with other tools" ? https://github.com/sharkdp/bat?tab=readme-ov-file#integration-with-other-tools