Remote Hosts

Open sessions on other machines in your own client, run panes on another machine, and script across hosts.

TUIOS can reach other machines over ssh. Once a machine is added as a host, you can list its sessions and agents, open one of its sessions in your own client, run a single pane on it inside a local session, and send commands and messages to it.

Each host needs TUIOS installed. Your daemon holds one ssh link to each host and talks to the TUIOS daemon on the other side. There is no new listening port and no second login system: ssh decides who can connect.

Tip

This part is interactive on the docs site. Open it on tuios.dev.

Adding a host

tuios hosts add build gaurav@buildbox

The name (build) is what you type later. It takes letters, digits, dot, dash and underscore. The address is anything ssh understands, including an alias from ~/.ssh/config.

The command writes the [hosts] table in your config file. The link is tested at once and the result is printed. A host that does not answer is still added.

From a terminal outside TUIOS, the running daemon opens the new host with no restart. From a pane, the new host waits for tuios config apply from a terminal outside TUIOS. A host that you add with "Add a host" on the settings page applies at once. See Changes that wait.

tuios hosts                  # every host and the state of its link
tuios hosts test build       # dial it now and print what ssh said
tuios hosts test build --start  # start its daemon if it does not run
tuios hosts remove build     # drop it

hosts add also takes:

• --ssh-option ARG: one extra ssh argument. Repeat it, for example --ssh-option -J --ssh-option bastion for a jump host.
• --connect-timeout SECONDS: how long one dial may take (default 10).
• --command PATH: the TUIOS binary to run on the host. Without it, the link looks on the PATH, then in the usual install locations, then in a login shell.
• --repos-root DIR: where the host keeps its checkouts, such as ~/src. fan --host, worktree new --host and start-agent -s HOST:SESSION look there for a repository by its origin URL.

Warning

Accept the host key first

The daemon runs ssh with BatchMode on, so a link never asks for a password or about a host key. Run ssh to the host once by hand to accept its key, and use key-based login.

Link states

Status	Meaning
up	The link is open and the remote daemon answers
no_daemon	The machine is up and no TUIOS daemon runs on it. See Start the daemon on a host
no_tuios	The machine is up and the link cannot find TUIOS on it. tuios hosts test shows where it looked
unreachable	The last attempt failed. The line under the table says why
reconnecting	The link dropped and TUIOS is dialling again
tailscale_check	Tailscale SSH waits for you to approve the login. The line under the table gives the link to open. See Tailscale SSH check mode
incompatible	The two sides speak different control protocol versions. Update one of them, or run tuios hosts sync
connecting	The first attempt has not finished

After sleep

(Added in v0.10.0)

A link finds a dead connection on its own, for example after a laptop sleeps.

• A link that carried nothing for 15 seconds sends a check to its host and waits 5 seconds. Two missed answers in a row end the link, and TUIOS dials again. A link that carries traffic sends no check.
• After this machine wakes from sleep, one missed answer ends the link. A link that waits to dial again dials at once.
• The first dial after a dead link runs ssh with -o ControlPath=none, so an ssh master that did not survive the sleep does not block it. A host that needs a password or a second factor fails that one dial, and the next dial uses the master again.
• The retry-host verb, which a click on a remote session also calls, checks a link that is up at once. On a link that is down, it starts the wait between dials again from one second.

On Windows the clocks do not show a sleep, so only the check applies.

The hosts table

tuios hosts add writes this for you. You can also edit it by hand:

[hosts.build]
addr = "gaurav@buildbox"

[hosts.lab]
addr = "lab-01"
connect_timeout = 5
command = "/opt/tools/tuios"
ssh_options = ["-J", "bastion"]

ssh_options accepts only safe options, such as -J, -p and -o Keyword=value. TUIOS drops a host with a refused option, such as ProxyCommand or a port forward, and tuios hosts names the reason. Put such options in ~/.ssh/config. See Rules for ssh_options.

A machine behind NAT is a reachability problem for ssh. Any tool that works as an ssh ProxyCommand in ~/.ssh/config works here too. See Configuration for an example.

Machines on a tailnet

If this machine is on a Tailscale tailnet, TUIOS can list the machines on it:

tuios hosts tailnet

   arch-btw          arch-btw.example.ts.net          offline
 + ente              ente.example.ts.net
 = forgejo           forgejo.example.ts.net           already the host forgejo
   my-phone          my-phone.example.ts.net          cannot run tuios (iOS)

Add one by its name:

tuios hosts add ente --tailnet

Nothing is added on its own. The list only saves you typing the MagicDNS name. A host added this way is still reached over ssh. TUIOS gets the list by running tailscale status --json, looked up on PATH and then inside the macOS app bundle, so it needs no root and no auth key. Without the tailscale command, the list is empty.

By default the list leaves out machines that are offline, this machine, machines shared from another tailnet, and phones and tablets. The [tailscale] table changes that:

[tailscale]
addr = "dns"            # dns (MagicDNS name), name (short name) or ip (100.x address)
user = "ubuntu"         # ssh login put in front of every address
os = ["linux", "macOS"] # an empty list offers every OS
offline = false
include = ["*"]
exclude = ["*-pad-*"]

[tailscale.users]
build = "root"          # per-machine login, wins over user

tuios hosts tailnet --json gives every machine with offered, and a skipped reason when it is false.

Tailscale SSH check mode

Tailscale SSH can hold a login until you approve it in a browser. This is "check" mode. ssh then waits, and Tailscale prints a link to open:

# Tailscale SSH requires an additional check.
# To authenticate, visit: https://login.tailscale.com/a/l1d9c7e392d3020

TUIOS reads this link and shows it.

• The daemon's link shows tailscale_check in tuios hosts, with the link to open. The link waits up to 10 minutes. It goes on when you approve the login.
• tuios hosts test and tuios hosts add report tailscale_check with the link. Open it, approve the login, then run the command again.
• tuios hosts sync on a terminal lists the link of each host that waits. Then it asks to wait up to 5 minutes. Each host continues when you approve its login. Without a terminal, or with --json, the host fails at once. Its row has "error_kind": "tailscale_check" and approval_url.

All ssh calls of one hosts sync run to a host use one shared connection. Thus one approval covers the whole run. The connection uses a private socket in $XDG_RUNTIME_DIR and stops when the run ends.

If the tailnet policy refuses the login, TUIOS names the user it refused. The row has "error_kind": "tailscale_policy". Change the user in the addr of the host in [hosts], for example addr = "ubuntu@ente".

Opening a session on a host

tuios attach --host build api        # attach the session api on build
tuios new --host build               # create a session on build and attach it
tuios new --host build ci --detach   # create the session ci on build and return
tuios ls --all-hosts                 # sessions on every machine

The session is drawn in your client, with your theme, config and prefix key. Nothing is nested. The connection goes through your local daemon and its link to the host.

In the rail, sessions are grouped by machine. Press Enter on a session under a host to attach it, or on the + beside a host to create a session there. A click opens a session in every link state (Added in v0.10.0). On a host that is not up, TUIOS says "Reconnecting to HOST" and opens the session when the link comes up. The wait ends after one minute, and TUIOS says why. While you are on a host, the rail lists this machine's sessions under local. Press Enter on one to come back.

If the link drops, the session keeps running on the host. Your client keeps the pane on screen, the dock says it is reconnecting, and TUIOS dials again on its own. After three minutes it gives up, says why, and returns you to the session you left on this machine.

--ssh runs ssh to the host and starts the TUIOS there instead. Use it when the host's TUIOS is too old to serve your client. You then see the remote client nested in yours, and you press the prefix key twice to send it to the inner one.

Panes on another machine

A single pane can run on another machine while it stays in a local session:

tuios new-window deploy --host build

The palette has the same thing as New window on another machine. The pane is drawn, laid out and closed here. Only its process runs on build. Its title bar reads build:deploy, so two panes side by side never look alike when they are on different machines.

The session is still an ordinary session: tuios ls, scripting, the mailbox, hooks and resurrection work on it as usual. The host runs only the process and its pty. It keeps no screen or scrollback for the pane, and the pane does not appear in tuios ls on the host.

Agent detection works in such a pane: the local daemon asks the host what the pane is running. An agent inside it can also report its own state and read its own mail with the usual commands and hooks: the report goes to the daemon on the host, which passes it back over the link as that window and nothing else. These panes have TUIOS_PANE_ID, TUIOS_PANE_HOSTED=1, TUIOS_HOST set to the host, and TUIOS_SESSION_REMOTE set to the session name here. They do not have TUIOS_SOCKET or TUIOS_SESSION, since those would name a daemon and a session on the host.

To paste an image from your clipboard into such a pane, press Ctrl+B V. TUIOS sends the image over the link. The host saves it to a file, and TUIOS pastes the path of that file into the pane.

The same works in a session attached with tuios attach --host. It also works in a pane of this machine that runs ssh to a machine in your [hosts] table. (Added in v0.9.3) TUIOS finds the entry that matches the ssh command, and writes the image on that machine. The host must allow files from this machine. When TUIOS cannot tell which machine the pane is on, it pastes nothing and shows a message that says why. See Paste an image.

Limits:

• The pane outlives a dropped link for a while. The host keeps its process running for its hosted_grace (10 minutes unless the host's config says otherwise) and keeps its last 64 KB of output. The title bar reads [reconnecting] build:deploy, and keys are refused rather than queued. When the link comes back, the pane is reattached and what it printed meanwhile is written to it. If the grace runs out first, the pane closes.
• Closing the pane ends its process at once.
• A restart of this daemon does not reattach. The processes on the host wait out their grace and end.
• A resurrected session brings the pane back as a local shell.
• Both machines need a TUIOS new enough to open remote panes. An older one refuses with a message that says to update.

Global sessions

A session with panes from several machines belongs to none of them. Once a second machine is reachable, the rail draws a global group above the machines for such sessions.

tuios new deploy --global

A global session starts with no panes. Every way of making a pane in it (the key, the rail's +, the palette) asks which machine to run on. Turn the group off with appearance.global_session = false.

Scripting across machines

Name the host in the target and a command runs on that host's daemon, through the link:

Target	Means
-s HOST:SESSION	A session on a host
-w HOST:SESSION:WINDOW	A pane in a session on a host
-s local:NAME	A session on this machine whose name contains a colon

tuios list-windows -s build:api
tuios capture-pane -w build:api:0
tuios send-text -s build:api -w 0 'make test'
tuios wait-for window-idle -w build:api:0
tuios list-agents --host build
tuios kill-session build:api

The CLI says which host answered, and --json adds a host field. A host name is matched exactly. An unknown name fails with unknown_host and lists the configured names, and a host that is down fails with host_unreachable. A command is not queued for later.

Messages and files cross links too. Mail to a host whose link is down waits on this machine, under Waiting to send in the Inbox, and goes when the link is back. See Agent Messaging.

Agents on every linked host appear on the rail and in your Inbox, and fan, worktree and start-agent can run there. See Worktrees.

Copy files between machines

(Added in v0.9.3)

tuios cp copies files and folders between this machine and a host, or between two hosts. The daemon runs the copy. It goes on after you exit, after a dropped link and after a daemon restart.

tuios cp ./build.tar build:              # to the receive folder on build
tuios cp build:~/logs ./                 # a folder from build to here
tuios cp build:/var/log/app.log gpu:/tmp/  # from one host to another

A path with HOST: in front is on that host. A path without it is on this machine. HOST: alone is the receive folder of the host. See tuios cp for every flag and exit code.

• TUIOS checks every file with sha256 before it puts the file in place. A file that does not match is copied one more time.
• A file that is there with the same bytes is never sent again. A second run of the same copy is a quick check.
• A folder copies with all that it holds. Small files go in one stream with a sha256 for each file, so a folder of thousands of files is fast on a slow link.
• A copy keeps the permission bits and the modification time of the original. It does not keep the owner.
• Links, named pipes, sockets and devices in a folder are not copied. The copy lists them as skipped.
• At most three copies move bytes at once. The others wait in the queue.
• A copy on a link waits behind your typing, so a large copy does not slow a pane.

When a file is already there

A file there with different bytes follows the conflict policy. On a terminal, tuios cp asks for each file. Elsewhere, it skips the file. Pass --conflict skip, replace or keep-both to choose. Keep both names a copy from another machine report (from build).pdf.

Resume, pause and detach

When a machine goes away, the copy waits. When the machine is back, the copy goes on from where it stopped. A restart of the daemon does the same. --timeout 10m makes the command exit after ten minutes of waiting. The copy keeps waiting in the daemon.

--detach starts the copy and returns. Press Ctrl+C in a running tuios cp and choose k to leave the copy running. Follow, pause, resume, cancel and clear copies with tuios transfers. tuios hosts test build --speed measures how fast a copy can be.

Progress

On a terminal, tuios cp draws a progress block: a bar, the rate and the time left. In a pipe it prints plain lines. With --json it prints one JSON object on each line.

In the TUI, nothing shows while there is no copy. When there is one:

• The dock shows a bar and a percent.
• The rail row of the machine shows a percent mark.
• The Copies panel (Ctrl+B u) lists every copy with pause, cancel and retry.
• A notice shows when a copy ends.

The files list of the rail has Send to machine for a file. A file dropped on a pane on another machine asks whether to send it there.

You can also start a copy with the transfer-start verb. The transfer-list, transfer-pause, transfer-resume and transfer-cancel verbs control the copies. The file verbs also list, read, rename and remove files on a host.

What a host lets you copy

The machine that receives a write decides where it may land. Each machine needs files in the other machine's allow list. A host lets a write land only in its files_roots: by default one receive folder, ~/Downloads/tuios. It never lets a write reach ~/.ssh, credentials or shell start files, and it never lets a read return keys or credentials. When a host refuses a copy, the error names the setting to change on that host. See Files over a link.

Keeping hosts on one version

tuios hosts sync installs the TUIOS version of this machine on each host that runs another version. A release build sends its own release. --dev sends a build of a checkout, and --binary sends a file.

tuios hosts sync --dry-run    # see what would change
tuios hosts sync --dev        # install a build of this checkout on every host
tuios hosts sync build lab    # only these hosts

The host needs no internet: sync sends the binary over ssh. It never uses sudo. It installs over the tuios on the host, or to ~/.local/bin/tuios when it cannot write that folder.

The daemon on a host keeps running the old version, and its sessions keep running. The row for the host gives the command that restarts it. Add --restart to restart the daemons in the same run. sync lists the sessions and the programs that a restart ends, and asks first. Without a terminal, add --yes. The sessions come back with their layouts and new shells.

See hosts sync for every flag.

Start the daemon on a host

A link reaches a host only when a TUIOS daemon runs there. When none runs, tuios hosts test build reports no_daemon and prints the command to run on the host:

build  gaurav@buildbox  no_daemon
The host is up and no tuios daemon is running on it.
To start the daemon, run this command on the host:
  /home/gaurav/.local/bin/tuios start-server
Or start it from this machine:
  tuios hosts test build --start

To start it from this machine, add --start. It runs that command over ssh and tests the host again. --dry-run shows the command and changes nothing.

tuios hosts test build --start    # start the daemon on build if it does not run
tuios hosts sync build --start    # install TUIOS on build if needed, then start it

hosts test --start does not install TUIOS. On a host with no TUIOS, use hosts sync --start.

Your ssh agent on a host

With ssh_agent = "follow" on both machines and agent forwarding on the link, a session that you attach on a host uses the ssh agent of your client here. Use ForwardAgent yes in ~/.ssh/config, or ssh_options = ["-A"] under [hosts.NAME]. TUIOS does not forward the agent by default. Forward it only to a host you trust, because root on the host can use the agent. See SSH Agent on tuios.dev.

What another machine may do here

The [hosts] table on the machine a link arrives at also says what the machine of that name may do there. With no entry, a linked machine may list, send mail, open sessions and panes, type into panes, and read and write files (files). Answering prompts for you (respond) is off unless you allow it:

[hosts.laptop]
allow = ["list", "mail", "open", "write", "respond", "files"]
files_roots = ["~/Downloads/tuios", "~/dev"]   # where it may write
hold_mail = true      # hold its mail to agents here in your Inbox until you pass it on

[hosts."*"]           # every machine with no entry of its own
allow = ["list", "mail"]

Anything not allowed is refused with forbidden. See What another machine may do here for every capability and option.

Related

• Sessions: Daemon sessions, detaching and resurrection
• Session Rail: Sessions grouped by machine
• Configuration: The hosts table and ProxyCommand transports
• Web Terminal: Reach TUIOS from a browser instead
• Phones: Pair a phone with tuios pair