How is cwd determined?

[qbedard]

I'll open an issue if it seems like it's a problem with kitty, but there are a lot of moving pieces with the problem I'm having, so I just wanted to get an answer to this question first.

Basically, when I have Neovim running and there is a language server running via the built-in LSP client, opening a new window in kitty results in the new shell starting in the directory where the LSP server is located. pwd in Neovim still shows the correct working directory, as does the tab title when the Neovim window is selected.

I'm kinda baffled, so I figured finding out how cwd is determined by kitty is the place to start to track down what the heck is going on.

[kovidgoyal]

The CWD comes from the process attached to the tty device that has the highest PID number. Your LSP server process should be calling setsid() so that it is not associated with the original tty. Indeed neovim should in general be calling setsid() after forking when execing processes it does not expect to output to/input from the tty.

[gotgenes]

Would you be willing to say more about this? It sounds like you believe Neovim and/or the language servers are doing something unexpected. I'm unfamiliar with setsid so maybe you could explain why they ought to be calling that. Then perhaps we could figure out why they don't, currently.

From my experience, Kitty 0.24.2 was the most recent release that I felt did the "intuitive" thing of using the CWD from which Neovim was launched, meaning, if I launched nvim from $HOME/myproject, new_tab_with_cwd would spawn a new tab with the shell at $HOME/myproject, regardless of subprocesses spawned under Neovim.

Starting with Kitty 0.24.3, if, for example, Neovim launches an instance of pyright language server, via pyright-langserver --stdio (relevant code), then when creating a new tab with new_tab_with_cwd, Kitty will start a new session using the pwd of the pyright process — /opt/homebrew/lib/node_modules/pyright/dist on my system — instead of $HOME/myproject, which is where I expected the new shell to start.

The changelog provides confirmation that something about the CWD behavior of Kitty changed starting with the 0.24.3 release, although it does not link to any PRs or commits:

Improve CWD detection when there are multiple foreground processes in the TTY process group

Edit: It looks like 7448789 is the commit where the behavior changed and went out with the 0.24.3 release.

I'll note that in both Kitty 0.24.2 and 0.24.3 and newer, if I launch pyright straight from the shell with pyright-langserver --stdio, then create a new tab with new_tab_with_cwd, they both start a new shell session at /opt/homebrew/lib/node_modules/pyright/dist instead of $HOME/myproject.

Many of these language servers communicate over STDIN/STDOUT. I'm not familiar enough with TTY to know whether that's the same thing as expecting "output to/input from the tty".

I continue to enjoy using Kitty as my terminal emulator daily, however, the new CWD behavior surprised me once it started, and it may surprise other users. Maybe another heuristic exists that would lead to more intuitive behavior for users like me? I did prefer the old behavior, for example, even if it wasn't "right" by Unix/POSIX philosophy, or whatever setsid and the TTY rules belong to.

[hansemschnokeloch]

I found a workaround by controlling kitty with this command in neovim

:silent !kcwd=`pwd`;kitty @ launch --cwd=$kcwd 

Following configuration is needed in kitty.conf

allow_remote_control yes
listen_on unix:@mykitty

[brightpuddle]

Another option. This is the first kitten I've ever written so just kind of cobbled this together, but this will iterate through the parent processes until it finds the shell:

import sys

PYTHON_SITE_PACKAGES = "/usr/local/lib/python3.9/site-packages"
SHELL_NAME = "zsh"

sys.path.append(PYTHON_SITE_PACKAGES)

import psutil
from kittens.tui.handler import result_handler


def get_cwd(pid: int) -> str:
    p = psutil.Process(pid)
    if p is None:
        return "~"
    while p.name() != SHELL_NAME:
        p = p.parent()
        if p is None:
            return "~"
    return p.cwd()


def main(*args):
    pass


@result_handler(no_ui=True)
def handle_result(args, result, target_window_id, boss):
    window = boss.active_window_for_cwd
    pid = window.as_dict().get("pid")
    cwd = window.cwd_of_child
    if pid is not None:
        cwd = get_cwd(pid)
    boss.new_window(cwd)

Put this in ~/.config/kitty/cwd_split.py and call this with e.g.: map cmd-d kitten cwd_launch.py

I'm not immediately sure what the argument is to new_window to control the split path, or what other issues this highly simplified version might cause, but it does work as expected.

[kovidgoyal]

stdin/stdout have nothing to do with the tty. A language server's stdin/stdout will be connected to a pipe from which the process that launches it reads/writes data to control it. They are not connected to the tty, otherwise you would see output from the lsp in your terminal. When you launch a daemon (background) process in unix you should detach it from its environment, including the controlling tty. An editor running in a terminal needs exclusive control over the tty, otherwise other processes can change tty state behind its back, leading to corruption. Therefore, a well designed editor should call setsid before execing binaries that it does not expect to write to the *TTY*. Read man 7 credentials and man 2 setsid and google how to write a unix daemon process for more information. kitty used to use the heuristic that the *first* process associated with the tty was used to determine the process to take the cwd from. That was not well defined, thought it worked around this bug in neovim. Now it has switched to using the processes with the highest PID, which is more likely to be the actual foreground process in situations such as running a script like: ``` #!/bin/bash cd /something vim ``` Here both bash and vim will be associated with the tty, but the user will expect the cwd to come from vim not bash. Using the highest PID achieves that.

[gotgenes]

Thanks for your reply! I have some more questions around "what is the expected CWD", but first, I want to get a better idea of what you're explaining.

I'll start by express what I want, as a user. I want Neovim to terminate if its parent shell terminates. I want Neovim to start up language servers for me when editing appropriate files. I also want all language servers spawned by Neovim to terminate when when Neovim terminates. And, transitively, I want Neovim and all its child processes terminate when the parent shell terminates.

I tried to research more what you meant by having Neovim and/or the language servers use setsid, and I came across these resources:

From this post on the Unix & Linux Stack Exchange

Normally, a shell will be a session leader, and every pipeline executed by that shell will be a process group. This is to make it easy to kill the children of a shell when it exits. (See exit(3) for the gory details.)

And also this one

Some operations within UNIX/Linux operate on process sessions, for example, negating the process id when sending to the kill system call or command. The most common use for this is when logging out of a shell. The OS will send kill -HUP -$$, which will send a SIGHUP (hangup) signal to all the processes with the same session id as the shell. When you disown a process, the session id of the process is changed from the shell, so it will not respond to the hangup signal. This is one part of the process to become a daemon process.

Lastly, regarding setsid itself, from this post

setsid is useful when you want to start a new session, because you have started to be connected to a new terminal -- such as when starting a shell inside a terminal emulator -- or you want a daemon (which you don't want to be associated with a controlling terminal).

It's not clear to me, after reading these, what would happen if Neovim was run with setsid, or if the language servers under it were run with setsid, when the shell session that started Neovim closes or is killed. It sounds to me like they would continue running. Does my understanding match yours, or have I missed some important things?

Thanks for your patience. I'm learning parts of the Unix system I hadn't had a chance to learn yet.

[kovidgoyal]

Neovim should not run with setsid(). When it runs background processes that have nothing to do with the terminal, it should run those with setsid(). And when it is exiting, it should cleanup those background processes that it does not want to outlive it. A language server is very much a daemon, it should be run as one, which involves using setsid().

[kovidgoyal]

Oh and if neovim for some reason doesnt want to cleanup its child processes itself, preferring to rely on SIGHUP, then rather than using setsid() it should instead background them (like running bg in a shell) which it can do by using setpgid() in the forked process and tcsetpgrp() in the parent process.

[gotgenes]

I filed an issue with Neovim for this:
neovim/neovim#18475

Thank you again for your help explaining the matter. Hopefully I paraphrased correctly.

[kovidgoyal]

See feea699

So you can use launch --cwd=oldest
to workaround the bug in nvim

[gotgenes]

Thank you! I appreciate you adding this feature!

[qbedard]

Sick. Got this working with nightly. Thank youuuuuuuuuu!

[qbedard]

For Homebrew users who want to use this now, kitty-nightly is now available:
Homebrew/homebrew-cask-versions@5faffe1

brew tap homebrew/cask-versions
brew uninstall kitty
brew install kitty-nightly

[kovidgoyal]

On Sat, May 07, 2022 at 06:13:30PM -0700, Chris Lasher wrote: I filed an issue with Neovim for this: neovim/neovim#18475 Thank you again for your help explaining the matter. Hopefully I paraphrased correctly.

Good, it should be a straightforward change in neovim.

[kovidgoyal]

neovim itself should not run with setsid() it should run background processes it starts with setsid(). And when it is exiting, it should clean up background processes that it does not want to outlive it. Language servers have no business being connected to a terminal, they are not displaying text to the user.

[saidbelkacem]

Very interesting read here Kovid. As a nvim user I just want to say thank you so much for this.