lfcd improvement proposal

[nicolasdumitru]

Goal

Simplifying lfcd and making it more POSIX compliant.

Issue

Currently, lfcd uses temporary files that it creates with mktemp in order to store the path of the directory that was last open in lf and into it. This poses multiple problems:

1. mktemp is not POSIX. This means that lfcd is also not POSIX (here, by POSIX I mean it only requires utilities/commands found in the POSIX specification and the obvious lf). While mktemp is pretty widely available as a GNU and BSD utility, it could be a problem on some POSIX compliant operating systems (e.g. non-GNU Linux distros).
2. Usage of temporary files is messy, overcomplicated and potentially unsafe (in some cases, not necessarily here).

Solution

Add a flag/option to lf that is similar to -last-dir-path, but instead of writing to a file, outputs to stdout. Why?

1. Lf can already write to a file, it would be trivial to instead print something to stdout.
2. the lfcd function body becomes a trivial, much simpler, more POSIX-compliant and less prone to breakage one-liner: \cd $(\lf -new-flag-to-print-path-to-stdout "$@")
3. lfcd doesn't have to check for the existence of a temporary file and then make sure it gets deleted.

The inspiration for this is fzf. By default, fzf prints the path to what you search for to stdout. Consequently, a user can do something like vim $(fzf) to edit a file that they don't know the exact path to or cd $(dirname $(fzf)) to do some fuzzy searching for a file and cd into the directory of said file. If you are interested and would like to try something similar without having to write a single line of code, try the above mentioned cd $(dirname $(fzf)) and search for a file or a directory where you have files.

[ilyagr]

I'm worried this would get messed up if the user calls any program that prints to stdout before quitting lf. This could be explicit (!cat $f) or a side effect of some previewer, editor, etc.

[gokcehan]

@nicolasdumitru Thank you for the proposal. @ilyagr is right in that it can get messy when shell commands are used. Infact, this idea has been discussed before in #140 and #472. If you want to see this in action, you can try one of the workarounds mentioned in these discussions (e.g. lf -last-dir-path /dev/stdout).

[nicolasdumitru]

@ilyagr valid concern. Do you think that forcing \lf -new-flag-to-print-path-to-stdout to print on a new line and using \cd $(\lf -new-flag-to-print-path-to-stdout "$@" | tail -n -1) could solve the problem?

[ilyagr]

No, it wouldn't. The user might need the output from the commands run from inside lf, and it'd be lost in your setup.

Try running lf|tail -n 1 and then !cat $f inside lf; it's not a great experience.

[joelim-work]

This is probably a crazy idea, but would this be possible if the shell commands were connected directly to the TTY instead of stdout?

diff --git a/app.go b/app.go
index fc9bd11..3057c00 100644
--- a/app.go
+++ b/app.go
@@ -327,20 +327,6 @@ func (app *app) loop() {
 			app.nav.dirPreviewChan <- nil
 
 			log.Print("bye!")
-
-			if gLastDirPath != "" {
-				f, err := os.Create(gLastDirPath)
-				if err != nil {
-					log.Printf("opening last dir file: %s", err)
-				}
-				defer f.Close()
-
-				_, err = f.WriteString(app.nav.currDir().path)
-				if err != nil {
-					log.Printf("writing last dir file: %s", err)
-				}
-			}
-
 			return
 		case n := <-app.nav.copyBytesChan:
 			app.nav.copyBytes += n
@@ -518,9 +504,15 @@ func (app *app) runShell(s string, args []string, prefix string) {
 	var err error
 	switch prefix {
 	case "$", "!":
-		cmd.Stdin = os.Stdin
-		cmd.Stdout = os.Stdout
-		cmd.Stderr = os.Stderr
+		tty, err := os.OpenFile("/dev/tty", os.O_RDWR, 0)
+		if err != nil {
+			log.Printf("getting tty: %s", err)
+			return
+		}
+
+		cmd.Stdin = tty
+		cmd.Stdout = tty
+		cmd.Stderr = tty
 
 		app.runCmdSync(cmd, prefix == "!")
 		return
diff --git a/client.go b/client.go
index 0fab7d5..7fbe32f 100644
--- a/client.go
+++ b/client.go
@@ -61,6 +61,10 @@ func run() {
 	app.loop()
 
 	app.ui.screen.Fini()
+
+	if gLastDirPath != "" {
+		fmt.Println(app.nav.currDir().path)
+	}
 }
 
 func readExpr() <-chan expr {

Even if it does work, I still have some concerns about this idea in general, as it involves breaking changes and involves a fair amount of risk when considering other platforms like Windows (which will require a different implementation).

I imagine that @gokcehan might have tried this already, perhaps there's a good reason this wasn't done.

[nicolasdumitru]

@ilyagr indeed, not a nice experience.
@joelim-work very interesting idea. Provided that it works as far as lf goes (at least on Linux), how would lfcd look after leveraging this change?
in the meantime, I have changed my lfcd to:

lfcd () {
	\umask 077
	tmp="$(command mktemp)"
	command lf -last-dir-path="$tmp" "$@"
	if [ -f "$tmp" ]; then
		dir="$(command cat "$tmp")"
		command rm -f "$tmp"
	else
		\return 1
	fi
	if [ -d "$dir" ] && [ "$dir" != "$(pwd)" ]; then
		\cd "$dir" || \return 1
	fi
}
```.
It has some additional safety measures and, in my opinion, it's slightly more readable.

[joelim-work]

Pretty much how you described above: cd $(lf ...).

That being said, the patch above is just a quick and dirty POC and takes a number of shortcuts. I haven't looked into it much further as I still have reservations about changing the design (i.e. having the shell commands bypass stdout and write to the console directly, just so it won't mess with last directory output).

[ilyagr]

I don't think @joelim-work 's idea is crazy, but I'm not sure what bugs might lurk if we do that.

I'm especially wary of issues on unusual OSes. We'd at least need a separate codepath for Windows, unless Go stdlib somehow provides a tty file object.

[gokcehan]

@joelim-work Thank you for coming up with a solution. I have not experimented with using /dev/tty for shell commands before. It seems like an interesting idea. However, I'm not yet sure about this change in general. I don't think it is a good idea to assume that /dev/tty is a portable interface. Windows is the obvious outlier, but even besides Windows I think there are slight differences among different unix systems (e.g. tty file should be opened as read/write/readwrite, tty file reads are blocking/nonblocking, tty file should be dup/open, tty file should be closed after use or not). As far as I'm aware, there is not a standard /dev/tty behavior described in the POSIX standard. Besides, isn't that the case that there are also different alternatives for tty nowadays (e.g. pty, pts, ptmx)? Personally, I still assume these things are magic devices, so I often prefer to stay at an abstraction level above these things.

Historically, we have had issues before when there was a change in tcell from /dev/tty to stdin/stdout (see #480 on our side and gdamore/tcell#394 on tcell side). It seems that tcell has changed back to /dev/tty since then. There seems to be a stdin/stdout implementation (i.e. stdIoTty) but it does not seem to be actively used currently. To be honest, I don't know the rationale for this choice. I'm aware these issues are not directly related to this discussion, but they can give an idea about the kind of problems that can result with the change discussed here. To be clear, I would be equally worried if we currently had used /dev/tty and someone had suggested changing it to stdin/stdout/stderr.

A slightly safer alternative might be to direct all command output to stderr with something like following:

cmd.Stdin = os.Stdin
cmd.Stdout = os.Stderr
cmd.Stderr = os.Stderr

Although, I'm not sure about this change either. There are just too many different kinds of commands that users can come up with, so it is difficult to tell if such a change would break anything.

All in all, I'm still not convinced that this change is worth the risks involved. I don't think we get any new functionality with this change. I can see the appeal of a simple command like cd $(lf ...), but I don't think there would be many users manually typing such commands, and if someone defines a custom alias or function for lfcd, it should not matter much how complicated the command looks like. If there are issues with the lfcd commands (e.g. leftover temporary files, security considerations), then it is worth a separate discussion.

I don't want to disregard this suggestion altogether, but if we go down this route, I would personally like to see this change tested extensively, preferably on all platforms that we support.

[joelim-work]

@gokcehan I had not thought of piping the shell output to stderr, although I'm not sure if it is much better than the current behavior, for example if the user wants to redirect stderr to a file to analyze stack traces during crashes.

The problem of obtaining a handle to the TTY for reading/writing in a portable manner is a serious concern to me. I would prefer to not handle it directly inside the lf (e.g. tty.go, tty_windows.go, etc.), and instead import such functionality from a library, although I am not sure that one exists because in most cases programs will just use the standard stdin/stdout/stderr streams and never need to have direct access to the TTY.

I agree that the benefit this feature provides is not really enough to warrant the risks involved. The problems described above seem to be theoretical in nature as opposed to actual issues that a user might encounter in practice. From my opinion, the sample lfcd scripts are just provided for convenience - there is no obligation to use them at all, and in the worst case scenario a user can just write their own lfcd script to suit their requirements.

That being said, I can see the appeal of using lf in a pipeline. For instance, the entire discussion in here can equally be applied to the -selection-path option. I think it's worth keeping this issue open just in case the situation changes and this feature ends up being easier to implement.

[nicolasdumitru]

@gokcehan @ilyagr @joelim-work what about setting a shell/enviroment variable (with a unique name that can be configured in lfrc) that holds the path to the last directory that was open in lf? I don't know how Window's shell works (and consequently if it could work on Windows), but lfcd is already different for Windows and this solution would work on every POSIX-compliant system and would circumvent stdout and the problems it poses. lfcd just becomes:
lfcd -new-flag && cd "$NEW_VARIABLE".

[joelim-work]

@nicolasdumitru Using a pre-defined environment variable as the argument for -last-dir-path would certainly be possible, but the point of the sample lfcd script is that the user can simply use it as is without any prior configuration.

That being said, after looking into this problem some more, I have found a library mattn/go-tty that can handle the task of portably obtaining a handle to the TTY. The usage would look like this:

tty, err := tty.Open()
if err != nil {
	log.Printf("getting tty: %s", err)
}

cmd.Stdin = tty.Input()
cmd.Stdout = tty.Output()
cmd.Stderr = tty.Output()

I have done some very brief testing on both Linux and Windows and it appears to work fine.

There is still some risk involved given that running shell commands is a core feature of lf which is widely used, and it is very unlikely that I am going to be able to test every single platform and usage scenario on my own. Perhaps it might be safer if this was hidden behind some option, or maybe this change could be added right after a new release to allow plenty of time for other users to test.

[nicolasdumitru]

@joelim-work it doesn't necessarily require prior configuration. The shell variable can have a default name (a very descriptive one that is very unlikely to coincide with what is already on the system). And the shell variable's name could just be a one-liner in (default) lfrc (basically set a particular string and then lf names the shell variable like that).

[joelim-work]

@nicolasdumitru I still have some trouble understanding, could you please provide a more detailed description about how this is supposed to work? If the user doesn't specify a path for this file, then we would have to provide one by default, and there isn't really a sane value for that, which is one of the reasons why mktemp is used in the first place.

Also, I'm not sure if this is relevant to the discussion, but AFAIK any environment variables set by lf (or any other process) aren't made available to the parent shell. This is why lf has to write the information to a file (or alternatively stdout) in order to communicate back to the parent shell.

[nicolasdumitru]

@joelim-work let me put it this way:
Say the new flag that I am talking about was "-lfcd". When a user runs lf -lfcd, when hitting the key that quits lf (default q), before lf quits, it sets a shell variable. By default, the shell variable could be called something like "lf_last_open_directory" or some other very descriptive name (so that it is very unlikely that another variable with that name already exists). This default name should be set in lf's source code. There should also be a line in the default lfrc that reflects the default (so that it is very simple to change if for whoever wants/needs to). The "-lfcd" flag would also take an optional argument which would set the name of the specific variable, but only for that specific invocation of lf. This is what I was thinking about in more detail. After this change, a simple version of lfcd could look something like:

lfcd () {
	lf -lfcd "$@" && cd "$lf_last_open_directory"
}

This does not take into account providing the optional argument, but you get the idea.

[gokcehan]

@nicolasdumitru If I'm not mistaken, that requires changing the value of an environment variable that belongs to the parent process (i.e. shell), which is not allowed due to security considerations as @joelim-work mentioned.

To further iterate on the stderr approach, maybe it could only be done when the new command line options are used. So let's say we added two command line options discussed here, -last-dir and -selection, that print the last directory or the selection to stdout on exit respectively. When lf is called with either of these flags, we map cmd.Stdout to os.Stderr, otherwise the behavior remains the same as before. It would still be possible to redirect stderr to a file as before to be able to debug crashes without using these new flags, and the idea discussed here could work with the new flags (e.g. cd $(lf -last-dir ...) or vim $(lf -selection ...)).

[joelim-work]

@gokcehan Although I see the TTY approach as the ideal end goal (especially after discovering go-tty), I think using stderr is an acceptable middle of the road approach which balances between user requirements and risk. Considering the following plan:

1. Map cmd.Stdout to os.Stderr if either of the -last-dir or -selection options are set
2. Replace os.Stderr with the underlying TTY
3. Remove the condition for the -last-dir/-selection options and connect to the TTY unconditionally

Only 1) is required to satisfy the user requirements, while 2) and 3) are nice to have but also optional and not necessary to implement at all.

I am happy to work on this, though as a side note I was thinking of merging the other PRs I have outstanding (#1384 and #1386) soon just so I don't have so many changes sitting in progress, if you have no objections.

P.S. I was thinking of calling the options -print-last-dir and -print-selection instead. I'm not sure if you prefer something shorter or something more descriptive.

[gokcehan]

@joelim-work I was expecting more discussion about the stderr approach. That is why I had not written anything about go-tty yet. I think we can really benefit from more discussion before the execution.

First of all, I did not mean to replace the old flags with the new ones, but instead I meant adding two new flags. Otherwise, this would be a breaking change that could effect the majority of our users. I don't think anything discussed in this thread is worth such a major breaking change.

I suggested this because you mentioned the use case for redirecting stderr to a file in your previous comment. I assume you meant redirecting stderr to a file at all times so one should be able to see stack traces for crashes that are hard to reproduce consistently. Currently it is possible to do this with something like the following:

lfcd () {
    tmp="$(mktemp)"
    err="$(mktemp --tmpdir lf.err.XXXXXXXXXX)"
    command lf -last-dir-path="$tmp" "$@" 2> "$err"
    if [ $? -eq 0 ]; then
        rm -f "$err"
    fi
    ...
}

With the new stderr proposal, this should work exactly as before. However, this would not work well if one tries to use the new flag with an err file like the following:

lfcd () {
    err="$(mktemp --tmpdir lf.err.XXX)"
    cd $(command lf -last-dir "$@" 2> "$err)
}

The assumption here is that if someone is okay to create a temporary err file, then they wouldn't mind creating a temporary file for directory change either. I should note that the current behavior is not perfect for the err file either since it also redirects command errors to the err file. If we want to properly support the err file, an alternative approach might be to use either stdout or stderr at all times like the following:

cmd.Stdin = os.Stdin
if gLastDir || gSelection {
	cmd.Stdout = os.Stderr
	cmd.Stderr = os.Stderr
} else {
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stdout
}

But then this is technically a breaking change, so we should consider more carefully, though I don't expect many problems with this either. In any case, I still see such changes much safer than anything that make use of tty. go-tty library seems nice, but it is still an additional library which has its own lifecyle with issues and updates, so I think it is better to avoid it if possible. Looking through its current and past issues, I get the impression that we might need to be extra careful about signal handling when using this library.

With these in mind, is there any disadvantage of the stderr approach that I'm missing? Why do you still see the tty approach as the ideal end goal? What does it bring to the table that stderr approach is missing?

[joelim-work]

@gokcehan Thanks for your feedback, and also my apologies for causing any misunderstandings and confusion with the PR. I probably should clarify a few things:

First of all, I did not mean to replace the old flags with the new ones, but instead I meant adding two new flags. Otherwise, this would be a breaking change that could effect the majority of our users. I don't think anything discussed in this thread is worth such a major breaking change.

I admit that I jumped the gun here and made the assumption that you wanted to replace the existing flags. I wasn't sure because providing two sets of flags could be confusing for users, however I am happy to add the old flags back in if you wish. Does this mean that going forward, you intend to maintain both sets of flags, or are you planning to eventually deprecate the old flags, or perhaps just leave the decision for later? Also if both sets of flags are being maintained, I would assume that you prefer to leave the various lfcd scripts unchanged?

I should note that the current behavior is not perfect for the err file either since it also redirects command errors to the err file. If we want to properly support the err file, an alternative approach might be to use either stdout or stderr at all times

While the stderr redirection is technically an issue, after thinking about it some more I think it doesn't actually matter too much in practice since it doesn't represent a regular use case. I see this technique as just a tool for debugging purposes, and I'm not sure whether there is any need to officially support it. I am inclined to keep the logic that you suggested originally:

cmd.Stdin = os.Stdin
if gLastDir || gSelection {
	cmd.Stdout = os.Stderr
} else {
	cmd.Stdout = os.Stdout
}
cmd.Stderr = os.Stderr

With these in mind, is there any disadvantage of the stderr approach that I'm missing? Why do you still see the tty approach as the ideal end goal? What does it bring to the table that stderr approach is missing?

The reason I see writing to the TTY as the ideal end goal is because there is no need to worry about this kind of conditional stream mapping, and that shell commands would still work regardless of whether the user redirects stdout, stderr or both. However I should point out that such a goal might also not be realistic and not worth pursuing, especially given the number of users - if lf was still in its early development stages, this might be different story. Also, I understand that the decision to include another library as a dependency is not to be taken lightly, which is why I have not done so and just used the stderr approach. For me personally, I don't intend to make any changes after #1397, and anything further than that can be left as a consideration.

Having said all of that, regarding #1397, did you have any other issues apart from replacing the old flags, and is there anything else you would like to discuss?

[gokcehan]

@joelim-work Thank you for your understanding. I hope you forgive me for dragging this discussion a little further. I just think it is important to find the right trade-off here that enables all use cases if possible and causes the least amount of disruption.

If err redirection is not on the table anymore, I should remind again that the original stderr approach (i.e. send all command output to stderr) might be a better option. This has the advantage of getting rid of conditional behavior, which I'm not a fan of either. When we don't need conditional behavior, additional flags may also not be necessary, since cd $(lf -last-dir-path /dev/stdout) should work without a problem. When we don't need conditional behavior, we could also consider adding an option to control this behavior as you previously mentioned. For example, we can add an option output that takes the value of either stdout or stderr with a default value of stderr. This would also enable the use of err redirection with lf -last-dir-path="$tmp" -command='set output stdout' 2> "$err". I also wouldn't mind to see set output tty as an opt-in alternative using go-tty library.

I should say a few words about breaking changes in general. I use many open-source tools myself, and I rarely follow the development of these tools myself. This is why I assume there are lf users that do not read update notes and/or interact with issues/discussions in any way. Some of these users simply use the program that comes with their package manager without being aware of the version of the program. I think it is important to keep the core functionality stable for such type of users. It should be okay to make a breaking change such as err redirection since it involves users that are actively engaged in the development. It should also be okayish to make a breaking change for a feature that is recently introduced like ruler/rulerfmt since we can assume these are mostly users that follows the development and likes to tinker with their setups. However, removing -last-dir-path/-selection-path flags are different in my mind since these are core features from day one or so. So I don't know if/how/when we can get rid of these options if we want to replace them with possible new ones. Having said that, I think it is okay to make changes to lfcd scripts. As I mentioned in other discussions, I'm not a fan of temporary files either, so using the new method, whichever we decide here, would be nice. We should keep in mind though that such a change should be close to the new release, since we have some installation instructions in the doc/wiki that mention downloading lfcd scripts from the repository. It might also be an option to delay lfcd script updates one or two releases further.

[joelim-work]

@gokcehan Thanks once again for your input. In my mind I had not considered using lf -last-dir-path /dev/stdout - I had somewhat rejected the idea because it seemed quite unnatural compared to using lf -last-dir > foo.txt. But I think it is actually better like this, given the requirement to maintain compatibility. I also feel uneasy when making breaking changes, and I don't expect users to follow the development of every program they use as it is far too much work to the point of being unreasonable.

I have submitted a new draft PR #1399 which incorporates your suggestions. There are no changes to command line options or any other breaking changes, which simplifies a lot of things. As for the lfcd scripts, I will leave them untouched for now since I don't see any reason to change them - unless they are actively causing issues which I think is not the case.

[joelim-work]

@nicolasdumitru I guess we can reopen this ticket. For now I haven't made any changes to the lfcd script, but we can add it in before the next release.

I do agree that the the syntax lf -last-dir-path /dev/stdout is not ideal compared to lf -last-dir, but I'm not sure whether it's worth introducing an additional flag because this means having to maintain two sets of flags and/or deprecate the old flags (i.e. breaking changes). Does this actually cause issues for you practically, or is this more about being idealistic?

[ilyagr]

I think we could mark "-last-dir-path" as deprecated and keep it (either forever or for a few versions). I'd then name the other flag to "-print-last-dir" or "-echo-last-dir".

[ilyagr]

I'm also wondering if -echo-last-dir should print a newline after the dir to match what other command-line programs (e.g. ls) do.

[joelim-work]

I do not mind adding new flags but if so I would prefer to deprecate the old ones, since to me it doesn't make sense to expose two similar sets of flags (which could be confusing for new users, as they might wonder which one is better to use).

I also noticed that a newline character is missing at the end for both the -last-dir-path and -selection-path options. I don't think there should be any problems adding it in, but I didn't want to change anything as it is a separate issue.

[nicolasdumitru]

@joelim-work if you were asking me whether it causes a practical issue, no, it doesn't cause me an issue in my current situation, but it may cause people running e.g BDSs or other unixes some problems (I don't know exactly what os does or doesn't have /dev/stdout); idk, maybe one day I'll try one of those. Also, if it's POSIX and as simple as possible, you don't have to worry about lfcd (os, other programs' implementations etc).
To reiterate: for me, it's more philosophical than practical (I really like the UNIX philosophy), but it might just be important for a bunch of people AND it would save a lot of people future headaches (so it is pragmatic).

[ilyagr]

I do not mind adding new flags but if so I would prefer to deprecate the old ones,

I think marking one of the flags as "deprecated" in the help text and the docs is good enough. I think we definitely should keep the old flags for a version or two, just in case the new ones have unexpected bugs on some systems. After that, I don't have a strong preference. I personally would probably keep them unless it actually causes confusion, but it'll probably be clearer what's best at that time.

I also noticed that a newline character is missing at the end for both the -last-dir-path and -selection-path options. I don't think there should be any problems adding it in, but I didn't want to change anything as it is a separate issue.

Yes, I don't think we should change those flags. I'm suggesting maybe inserting a newline if we make a new flag. I don't insist on it if people think that slight difference in behavior is confusing.

[joelim-work]

OK, given the way this conversation is going, I will consider doing the following:

• Add the -print-last-dir /-print-selection flags (and update the corresponding shell completions).
• Fix the trailing newline issue for these new flags.
• Mark the original -last-dir-path/-selection-path flags as deprecated. They actually aren't mentioned in doc.go but maybe it's sufficient to just add an echoerr call when lf is started if either of these two flags are used.

However I would like some feedback from @gokcehan before taking any action.

[gokcehan]

I agree it is a good idea to add new flags for stdout, especially considering Windows, but I'm still not convinced that we should deprecate the old flags. What exactly does deprecation mean in this context? If we are going to keep them forever, then what is the point of deprecation? If we plan to remove them at some point, then how long should we wait? One simple heuristic to determine a deprecation duration might be to wait as long as the lifetime of the feature. So if a feature was implemented 1 month/release ago, we should at least deprecate it for 1 month/release. Clearly, this would not work for these flags since they are almost 7 years old at this point. Ubuntu pushes LTS releases every 2 years. Would that be an appropriate duration for this deprecation? Otherwise, system upgrades might result in a broken setup without any deprecation warning. Do we really want to plan a 2 year deprecation ahead so that we will have a "perfect" software after all this time?

I also don't see the point of removing the old flags as they provide two different functionalities in a way. There are many programs which has an -o flag for an output file even though you can always manually redirect the output instead. If we consider hypothetical unix systems without /dev/stdout, shouldn't we also need to consider niche cases where redirection is not available? I could imagine there might be some terminal init commands or obscure editor scripting languages where redirection is not available or difficult. Do we want to explicitly prevent such possible use cases? Note, this would also put a nail in the coffin for a possible set output stdout feature in the future.

I commonly see the argument about unix philosophy in these proposals. As I said previously, there are many terminal applications including the text editors in the original unix that do not write to stdout. Most compilers in unix systems do not read from stdin and print to stdout. There are many other unix tools that skip the use of stdin, stdout, or both. The use of stdin/stdout/stderr as a design principle is mostly useful for tools that are meant to be used in pipes to process and filter data, though not every tool is meant to be used in such a manner. Technically speaking, we don't even consider any examples with pipes in this discussion. Note, we also don't use stdin in lf. Besides, most unix tools have a default behavior when no flags are given. We can't do that in lf since we don't know whether the user wants the last directory or the selection. Why? Because, we implement two different functionalities in lf, which is against the unix philosophy. So shall we fork the project to separate -last-dir-path and -selection as separate tools?

Regarding the POSIX compatibility, we try to remain compatible in our shell examples, though this compatibility does not include our command line options. POSIX standard has an utility convention (https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html) that defines options as single characters with a preceding dash character. We only have long form options in lf, and we don't even follow the double dash convention (e.g. GNU). After all, lf is not a program that is part of the POSIX standard, so it does not make much sense to follow the POSIX compatibility.

In general, we should keep in mind that lf is a cross platform tool that also works in non-unix and non-POSIX systems (e.g. Windows), so technically we neither follow the unix philosophy nor aim for the POSIX compatibility.

@nicolasdumitru Regarding the portability of /dev/stdout, I find it a little inconsistent that you were not worried about the portability when we were discussing the use of /dev/tty. Do we expect, there are unix systems that have /dev/tty and not /dev/stdout? To be honest, I don't know the answer to this question, though it is also not too hard to answer either. Go language has a defined set of operating systems that it can run and among these we can only build tcell on a subset of them. So we actually have the full list of systems that we support in our build script. Let me know if there is a system in this list that you worry might be lacking /dev/stdout and we can try googling around to see if it is missing or not. To be clear, I wasn't worried about the existence of /dev/tty on unix systems, but instead I was worried about the portability of its behavior. Looking at go-tty, it seems that /dev/tty is used for all unix systems, and Windows and Plan9 (not relevant for lf since tcell does not work on Plan9) are implemented separately.

All in all, I'm in favor of getting rid of our attitudes of perfectionism, unix philosophy, POSIX compatibility, or whatever it is that drives us to make a breaking change and keep the old flags existing as before without any deprecation. I don't think it is too difficult to maintain the extra code. I don't remember any major issues with these flags before and I don't expect anything much in the future. If an issue comes up regarding these flags, I'm ready to take the blame and responsibility of fixing the issue. If I'm not around, then feel free to remove these flags.

Having said that, I don't mind adding new flags for stdout, and it is probably a good idea to do so. If new flags are added, it could also be nice to use them in our lfcd scripts. In a way, that could be our way of deprecation, without actually deprecating and removing anything.

I currently have no opinions about the names for the new flags, and the existence of a trailing newline in the output.

[joelim-work]

I don't have much more to add to this discussion. TBH I did not expect that I would have to reopen this issue after implementing #1399.

Regarding the various topics discussed in this issue (e.g. deprecation strategies, POSIX compatibility, potential use of lf in pipelines, software minimalism, etc.), I think opinions will inevitably vary among different people including myself and everyone else here, and that there is no one single correct answer. As a collaborator I will respect the wishes of @gokcehan who is the owner of this project.

In any case I think we are all in agreement to add new flags, which I don't mind working on. In addition, I will update lfcd.sh to make use of this new flag to serve as an example, though I will leave the lfcd implementations for other shells untouched, and someone who actually does use those shells can update them some time later.