remote ssh plugin (like vscode remote-ssh)

[justinmk]

This is mostly just for tracking. Obviously everyone wants this, but it looks like work... 😏

Problem

vscode has "remote" tools (broadly referred to as vscode-remote, in particular vscode remote-ssh)

• https://code.visualstudio.com/docs/remote/remote-overview
• https://code.visualstudio.com/docs/remote/ssh

... that allow the user to

1. input a ssh URI (hostname + port)
   • or select from a list of hosts discovered from your local ~/.ssh/config
2. vscode connects to the remote ssh endpoint using your local ~/.ssh credentials
   • or prompts for password as needed
3. vscode starts a new local UI
4. vscode installs another vscode on the remote machine automatically
   • it also installs your plugins, on the remote!
5. the new local vscode UI controls the remote vscode server, and you can use it to work on the remote machine very much like a local vscode.

Ideaaaaaa

1. Implement a "universal installer" script (sh, powershell) that pulls from
   https://github.com/neovim/neovim/releases and installs the correct Nvim on the current machine

    curl https://neovim.io/install-hax && bash install-hax
   

   • does something like this already exist?
     • https://github.com/rhysd/action-setup-vim
     • https://github.com/NTBBloodbath/nvenv
     • https://github.com/MordechaiHadad/bob
2. Implement an Nvim plugin that works like vscode remote-ssh (described above)
   • select ssh host from ~/.ssh/config
   • installs Nvim on the remote (if suitable nvim version is not found on the remote)
   • starts remote Nvim server
   • configures ssh tunnel
   • starts local Nvim UI connected to the remote Nvim

[lewis6991]

Relates to https://github.com/chipsenkbeil/distant.nvim

[theHamsta]

More related ideas:

• virtually (or via scp) adding local rtp to the remote environment? Or maybe have a dedicated lua file for it that let's users spin up their preferred environment: a remote equivalent to init.lua when init.lua is not yet present on the remote.
• the possibility to manage such attachments and quickly switch between them

[ObserverOfTime]

Is it really necessary to include this in the core (if that's the plan)?
Remote files can already be edited through sshfs with a local editor.
Using a remote editor should be accomplished with an external plugin instead.

[justinmk]

Is it really necessary to include this in the core (if that's the plan)?

afaik, this should be possible as a shell script + plugin (which is the current proposal described above).

doesn't need to be in core, but until a plugin magically appears, this issue just tracks the general idea. And if a plugin does magically appear, then it might make sense to integrate it as a builtin plugin later.

[jdrouhard]

One major difference with vscode-remote vs a remote nvim UI is that with vscode-remote, all the keyboard input (and thus the actual buffer changes) are done by local client, with the input and server-driven updates being synchronized asynchronously in the background.

This becomes very important when you have a high latency connection--vscode-remote stays feeling "snappy" when typing with characters being displayed to the screen instantly, but a remote nvim will start to be laggy with displaying input (or even just a normal nvim run in an ssh session).

Not saying we should try to make it work like vscode-remote, but just wanted to point out the high latency remote connection use case as food for thought.

[justinmk]

yep, see also emacs TRAMP. It's definitely a better model in many respects, but more complex. vscode has a VFS layer, Nvim would need something like that.

The door is open for some sort of VFS model in Nvim, but that's orthogonal to this tracking issue.

[theHamsta]

What you can do in VS code is also to manage multiple remote sessions. Maybe there could be a command to reattach the TUI client to another server during a running session. Then, it could be implemented via a plugin only in a running session.

[chipsenkbeil]

@justinmk curious if there would be a way to embed the distant library into neovim itself. It's a rust library, so if we exported it in a C-friendly way, it could be consumed.

If so, you could have neovim provide tramp natively and support doing this via ssh and distant's own protocol. And anything else added eventually like Google Drive, Dropbox, etc. for file editing.

Would take a lot of the work out of needing to handle all of the protocol stuff and support something higher level where you can edit files, read files, run programs, etc. on the remote machine. Also supports remote shells and whatnot.

All of these features are in the distant.nvim plugin, but would love to have them be native.

[KFearsoff]

Here's my 2 cents.

I would love to see remote development as a feature (perhaps in a plugin, but I do want it!), but I think following the footsteps of VSCode is the wrong approach.

VSCode approach downsides

1. Running Neovim on the remote takes CPU, RAM and disk space. This can be a complete dealbreaker for some of the more resource-intensive plugins, like rust-analyzer.
2. As noted by @jdrouhard , latencies quickly become an issue, which makes remote development feeling no better than just SSHing into the host and typing manually.
3. We'd have to write a script for installing Neovim. We want it to be reliable enough to not warrant any manual intervention. This is a tall order. Consider the sheer varierty of targets: you have Linux targets that use glibc, musl, ancient kernels, different BSDs, Windows, mingw, MacOS. All of them with their own quirks. Differences in Neovim dependencies might introduce subtle bugs that are basically impossible to debug.
4. You would also have to bootstrap the installation somehow. curl might be not present on the target, and so can bash. The package manager varierty is also quite gigantic. You would have to handle A LOT of platforms to bootstrap the Neovim installation.
5. Plugins are pretty ad-hoc. Some of them depend on some runtime (LSPs are a great example), some require packages to be installed on the host, some require fonts (basically every plugin that wants Nerd Fonts). You are usually expected to handle it yourself, but it doesn't mesh well with automation. And then, a lot of plugin managers support version pinning but don't enforce it by default, so you could easily get a different version of a plugin installed which would cause unexpected problems. And then some plugins also have their internal update mechanism (LSP servers, Treesitter), which can also explode on you.

Better approach

1. We should use local Neovim for everything. That way we are sure that everything works, and it doesn't require extra effort.
2. We mount remote files locally, and open them in Neovim. SSHFS is one way to do it, but it's unmaintained and problematic. I suggest rclone - it supports remote mounting. It does a lot of what we'd like to have itself, it's reasonably efficient, it caches files locally (which means that if you experience a disconnect - you can continue your work) and it comes with a very nice perk of supporting a lot of cloud providers. Maybe you are storing JSONs or JS files on S3 - you can redact them with Neovim seamlessly! This is actually better than what VSCode can offer.
3. We'd have to sync the changes. Rclone has commands and settings for syncing changes: you could set it to sync changes every 5 seconds, and do it in background asynchronously. You would likely have to auto-write changes to local filesystem in Neovim just before that, which should be fine. This can be made reasonably efficient and nice to use if you play around with the settings, you'd be able to handle disconnects too. Rclone also lets you resolve conflicts!

Prior work

https://github.com/ipod825/vim-netranger - implements a file manager that also can use rclone. This is basically what we want.
https://github.com/DanielWeidinger/nvim-sshfs - implementation with SSHFS

[jdrouhard]

The obvious glaring downside to the "local everything" approach is if you actually need to develop in the remote server's environment.

What if I have a work laptop with bare minimum essentials (a terminal emulator, maybe a neovim install, and a VPN), and I can't have the full development environment toolchain? Pick your poison: gcc with lots of of library dependencies, node with tons of npm modules, whatever. It might not even be the same OS. Windows->Linux, Linux->macOS, etc.

With vscode-remote, I can have a development experience that "feels" local whilst actually running in a completely different environment that is equipped appropriately. In this way, vscode-remote and remote nvim UI's approaches are superior to some sort of file synchronization method to pull files locally for editing.

[chaneyzorn]

Mounting remote files locally such as sshfs under poor network is not friendly for local git, and git operations become terribly slow.

[KFearsoff]

The obvious glaring downside to the "local everything" approach is if you actually need to develop in the remote server's environment.

I thought the thread is about the opposite. Doesn't Neovim already support remote server's environment with nvim --headless --listen 0.0.0.0:8080 and nvim --server=neovim.com:8080?

With vscode-remote, I can have a development experience that "feels" local whilst actually running in a completely different environment that is equipped appropriately. In this way, vscode-remote and remote nvim UI's approaches are superior to some sort of file synchronization method to pull files locally for editing.

Sure, that's true. I'm personally a lot more interested in working locally on remote files, but working locally with a remote environment is a cool usecase.

Mounting remote files locally such as sshfs under poor network is not friendly for local git, and git operations become terribly slow.

I have proposed using Rclone, which has functionality of letting you resolve conflicts. I guess that could be used to always prefer the local .git over the remote one, which would introduce its own set of issues, but I don't think git-related issues are avoidable anyway. Git really works quite poorly with file synchronization.

I think that's more of a UI issue, though. I'd imagine you'd be able to make a local checkout if you have remote access to the machine that has the repository, no? In that case, it would be much better to just avoid this remote dance altogether and git clone locally.

[chaneyzorn]

I thought the thread is about the opposite. Doesn't Neovim already support remote server's environment with nvim --headless --listen 0.0.0.0:8080 and nvim --server=neovim.com:8080?

As @jdrouhard mentioned above, remote TUI over ssh/tcp does not provide a local vim file buffer. Every time the cursor is moved (not even an edit action), the terminal needs to be refreshed through ssh, which will cause obvious lag when the network is not good.

Mounting remote files locally do provide a local vim file buffer, but full filesystem-level access over network are unnecessary, and the experience isn't good.

Such as local git or local LSP server, need to analyze the complete context of the project, when over network , the input is huge and the output is small.

[ashwinvis]

I have played around with Neovim's --remote-ui option and some SSH tunneling of via sockets. I made a semi-automated solution here, https://codeberg.org/ashwinvis/ssh.nvim . Someone might find it useful

I agree particularly with @jdrouhard's take [1]. What we have in neovim natively is:

• Communication over scp:// using netrw which gives a local everything approach. Buffer is cached locally making it "snappy" even with high network latency, :Lexplore would show the remote files. The problem is, things like LSP, integrated :terminal and :! commands do not work.
• Using nvim --remote-ui --server or neovide --server option is a remote everything approach. It works with some semi-automation as shown above, until someone makes a plugin to make this truly effortless and cross-platform. It does not have the downsides of netrw (LSP, terminal and commands works) but everything is synchronous. It works only with a good network and is painful to use over network with high latency.

People have mentioned VScode remote-ssh and Emacs TRAMP. Another tool which works well remotely out of the box is JupyterLab because it was always built with a client-server model. The buffers are cached in the client's browser and everything is executed in the server.

[elijahmorg]

curious if there would be a way to embed the distant library into neovim itself. It's a rust library, so if we exported it in a C-friendly way, it could be consumed.

I don't think this is a good idea. SSH is ubiquitous. Distant is very new and not currently supported.

The beauty of VCode remote is how seemless it is and it "just works" most of the time.

The security implications of SSH are also very well understood while distant would be something entirely new that users would have to understand.

I would be for this if
A) It was seamless and the end user does not have to add distant to the remote machine
B) If all the communication could be wrapped in an SSH tunnel

otherwise I worry it would be too niche.

[chipsenkbeil]

I would be for this if
A) It was seamless and the end user does not have to add distant to the remote machine
B) If all the communication could be wrapped in an SSH tunnel

otherwise I worry it would be too niche.

Distant also supports acting as an ssh client, so you get both feature sets. So yes you can! 😄

The security implications of SSH are also very well understood while distant would be something entirely new that users would have to understand.

Fair enough. Distant uses a very modern encryption and authentication standard on par with ssh, but it definitely isn't battle tested or audited, although the crypto libraries have been audited.

But advanced functionality like running a file watcher for remote changes, translating language server file references such that it can work on the remote machine, remote filesystem searching, and persistent connections (like eternal terminal) are implemented using distant.

Regardless, using it as a client library would give you a high level abstraction for file io and process execution, rather than needing to work with ssh and the sftp subsystem.

As distant implements support for other remote platforms, neovim would also get it for free.

So I'm not necessarily sold on limiting to ssh. We've seen the wonders that Lua had over vim script! 😄

[amitds1997]

I created a plugin for this use case: remote-nvim.nvim. If you use devpod, you can also develop inside devcontainers using this plugin.

I deployed the first release of the plugin yesterday (v0.0.1) 🎉. Feel free to reach out to me if you have any questions about the plugin. You can drop your queries about the plugin over in the release discussion.

I have only tested this with MacOS and Linux. Support for Windows is planned but timelines are not decided yet.

[amitds1997]

Does this do something magic or just setup neovim remotely and start a headless nvim client?

No magic. It installs Neovim on the remote server in a custom folder and then starts a headless server over on remote and then launches a client locally. Everything done by the plugin is managed inside that folder.

[justinmk]

Buffers need to be available and edited locally. Swapfiles are local.

Where did that requirement come from?

There should be some APIs to change the location to "local", like :local COMMAND to run a command with a local location,

This is too complicated and outside the scope of this issue. The goal of this issue is not full emacs TRAMP. The goal is just what I described in the main description: auto-download nvim on the remote and connecting to that from a local nvim, with a single command.

Create a separate issue for more complex approach such as emacs TRAMP.

[CGamesPlay]

Sorry for being considered off-topic. I think it's worth pointing out that the subject of this issue says "like vscode remote-ssh", which does all editing locally and has the notion of local and remote execution contexts. That's why I thought it was related!

[fecet]

I personally strongly agree that neovim should have a remote mode as powerful as vscode. This is definitely a killer feature.

[justinmk]

Locking this since comments are now repeating things already addressed above.