UX suggestion: Write a go based previewer that will be used as a library / executable

[doronbehar]

I've been using lf for a while now and I'm eager to make it a better alternative to ranger and add all the missing features / getting used to them.

While thinking about some of these features, one thing has always perplexed me with both in ranger and lf but I think getting my idea inside upstream would be more idle here: What if, instead of relying on the famous ugly scope.sh script with all those exit statements and handle_ functions, we would rely on some kind of dumb-simple configuration file that will be parsed by lf on startup and be used as a previewer guideline? The scheme I was thinking about has the following key points:

• The first word in every line of this file will define a handler for every mime type.
• The rest of the line parsed as the command that executes when this mimetype is encountered.
• %s should be expanded as the file name in preview in the command.
• The first word in every line would be treated as a regex.
• Most importantly, this will not depend on the file extension, the external file *nix command, but on a pure Go library such as https://github.com/h2non/filetype .

For example:

text/* cat %s
video/* mediainfo %s
archive/* atool %s

As a bonus, lf could provide an executable that can be useful when users write their fzf based commands for lf or any other program. This previewer will use of course the configuration file I described above and it will print the contents of a directory if needed (this specific case won't / shouldn't be handled by the configuration with inode/directory and ls IMO).

I think this could bring an undoubtable performance enhancement and convinince for lf which no other terminal file browser has.

What do you think? I'd love to work on this feature and send a PR.

[TeddyDD]

As a bonus, lf could provide an executable that can be useful when users write their fzf based commands for lf or any other program

I like an idea of fast preview 'script' written in Go. I don't see why this should be part of lf tho. lf is focused on Unix philosophy - do one thing well. You could write such previewer as separate project so It could be used with lf, fzf and other programs. If you are interested in such project feel free to ping me - I'd love to help.

[doronbehar]

Thanks for your comment. Since my plan is to write it in Go, I was thinking lf could use it embedded so it will be even faster within lf and not only fast enough because it's not scope.sh.

[gokcehan]

@doronbehar Sorry for late reply. I'm glad to hear about your enthusiasm for contributing but I'm not sure about this idea. I think this might require a significant amount of additional code and a new dependency with little or no benefit. I agree scope.sh is kind of complicated with seemingly unnecessary details but it does not have to be. For example, in the simple example you give:

text/* cat %s
video/* mediainfo %s
archive/* atool %s

You can simply have a previewer script as such:

#!/bin/sh
case $(file --mime-type "$1" -b) in
    text/*) cat "$1";;
    video/*) mediainfo "$1";;
    archive/*) atool "$1";;
esac

Besides wrapping everything with a case and adding a few additional punctuations, I don't think these are too different. And since the latter is a shell script, you can for example check the existence of an installed program within the script and provide a fallback program and such. In your example, we need to provide a syntax for this use case which is an extra learning for users. We currently don't use exit codes but it is possible that we might make use of them at some point to control caching. I don't think it would be possible to provide such flexibility in your example.

Of course, the advantage of such an integrated approach would be to avoid spawning an extra shell and file process. Extra performance is always nice, but spawning a few additional processes may not always be significant since you already need to use external programs to process and preview the file. Sure, it can be faster in some cases, but not exponentially faster.

[doronbehar]

O.K, you have a point. Anyway, I'll write my previewer in a separate project and I'll leave a function that would be exposed for other packages to use. When it'll be ready I'll try to integrate it in lf either as the executable that will preview the files and perhaps I'll even try to patch lf to use it as a library and I'll report how significant were the performance improvements.

For @TeddyDD or anyone else interested. I have a design proposal for the main handle function, if you wish to influence, you're invited to comment.

Cheers.

[doronbehar]

@gokcehan, I've had a few more thoughts after some experience in the development of the my previewer which I call now pistol:

We currently don't use exit codes but it is possible that we might make use of them at some point to control caching.

Since this previewer may be useful for other programs as well, I thought that maybe the caching mechanism could be part of it and not handled only in lf so every program using pistol could benefit from it as well.

And since the latter is a shell script, you can for example check the existence of an installed program within the script and provide a fallback program and such. In your example, we need to provide a syntax for this use case which is an extra learning for users.

I agree as for the learning difficulty and I'm exploring ways to make pistol's configuration file more immune to situations like this while keep it's simplicity.

Additionally, I was thinking maybe this internal previewer, shouldn't be enforced on users and we can decide that it will be used only if one of the following is used in lfrc:

• set previewer pistol
• set previewer internal

To further favor pistol's speed (and especially if it'll be integrated), I'm exploring ways to teach it to preview common files with an internal previewer (unless something else is specified in the configuration file). For example, for normal text files, I've used github.com/alecthomas/chroma to provide built-in syntax highlighting. Also, I plan to use github.com/mholt/archiver to preview any archive file without any external dependencies.

I think the benefit of this design, (especially if it'll be also incorporated into lf) will be significant not only because it's faster but because it'll make lf a self-contained (as always go executables are) file manager useful for chrooted environments and it will have all of those nice basic features out of the box.

Anyway, these are my updates for the meantime. I hope you will share your opinions about my decisions.

[gokcehan]

@doronbehar I'm glad to hear that you started working on pistol. I think it is an interesting project on its own. I know about bat tool which is an improved version of cat with syntax highlighting and git integration. From what we discuss here, I think pistol aims to be a much more improved version of bat using external tools to preview files and such. People are quite fond of these tools.

On the other hand, I don't want to create false expectations. I try to follow the unix philosophy for the development of lf to provide as little as possible so that it can become more stable as time passes. That means excluding features when there is enough functionality for a separate standalone tool. For example we don't provide a builtin file opener functionality so that people can use whatever they like. I remember ranger was also trying to do something similar to separate rifle file opener from ranger. I think file previewing is no less complex than file opening and it deserves to have a standalone tool on its own. Therefore, I'm not sure if it is a good idea to directly integrate pistol in lf. We could as you say not enforce users to use pistol and make it optional but still it needs to be a dependency for lf. The functionality will be baked into the binary even for non-users which some people may see as bloat.

Of course, this is only my opinion about this. I don't want to be the only person to reject such an integration. If majority of users support adding such an integration and your project matures in time I think we can rethink this integration. In the meantime, I advise you to work on this only if you won't be disappointed if the integration with lf is rejected. As I said, it looks like an interesting project on its own and surely it will benefit the lf ecosystem in the long term. I would like to hear about your progress so feel free to leave comments here from time to time.

[doronbehar]

@gokcehan Thanks for showing your support and for kindly presenting your opinion. I've had more work done on Pistol lately and I've pushed a few updates which make it rather stable as far as I can see along with a README.

I've been thinking more about what you said on the Unix philosophy conformism and so lately I've been tending to think that integrating Pistol as a library in Lf would certainly violate these ideals. Although Go would allow us to do that rather easily with an io.Pipe (I've tested that on lf's source), it doesn't mean we should jump right away and implement that as it would certainly introduce at least a bit of maintenance burden; and this is exactly what the Unix philosophy tries to protect us from.

However, although Pistol should be ready to use, I've encountered an issue with it when I tried to set previewer /home/doron/.go/bin/pistol in lfrc so I'll be addressing this in a new issue.

As for this one, I hope no body minds me closing it.