Non-human (bot) interface

[naggie]

I'm sure I saw a bot known as Hubot last time I logged in, amongst the now-frequent spam from various miscreants.

I wonder, is it using the regular human interface?

Perhaps we could have some kind of API to make a bot, remote or local.

[shazow]

What distinguishes a human interface from a bot interface? Are we going to start presenting CAPTCHAs? :)

[naggie]

For example, no ANSI colours and a machine parseable interface -- eg, JSON

[shazow]

Ah, well there is /theme mono which removes all colors. No plans for JSON yet, though. Maybe someday.

[peterhellberg]

The line based format is ok when building simple bots (like my ssh-chat-bot)

But it would be nice to be able to identify as a bot so that human users could opt to filter out potential noise.

[shazow]

There is no way to force bots to identify themselves as such, /mute command will come eventually to let you silence whoever you don't like. We can also add better throttling heuristics if anyone has ideas.

Most likely implementation of a non-human interface is maybe something like TERM=bot env variable which will skip PTY handling and won't set a prompt.

[peterhellberg]

I was thinking that well behaved bots should have an option to identify themselves as bots, since we don’t have control over the clients.

I’ll experiment some more with my little bot.

[shazow]

I was thinking of exposing the TERM variable in the /whois (any objections?), so that may help further in identifying people vs bots.

[peterhellberg]

Ah, that might be a good way to do it. (Unless there are security implications in doing so)

[chexo3]

I think that is fine. It's your fault if you use an insecure version of bash anyways.

[shazow]

@chexo3 This is unrelated to bash, it's your terminal (that you might run bash inside).

No, I don't think there should be security implications. But someone correct me if I'm wrong.

[rkeene]

Perhaps another way forward would be to use the SSH "exec" session request rather than "shell" session request to differentiate the protocol-based interface from the UI, leaving the "UI" with shell.

Then someone could do "ssh server sshchat/1.0" to get the protocol interface v1.0 and a session would be established, or an error returned in the form of an exit code.

[shazow]

IMO setting a TERM variable to bot or readline something would be the cleanest thing to do.

[rkeene]

I'd argue against passing environment variables in a "shell" session because there's no right answer -- setting it to "bot" isn't good enough since someone in the future may define a "bot" terminal (probably not) and we can avoid ambiguity thanks to SSH's existing features.

[shazow]

Since we did a bunch of changes to how we render things in the last release, we probably broke most bots. I'd like to provide a more barebones output for bots for the next release.

My current thinking:

• I'd like to avoid putting this into the exec field (ssh chat.shazow.net /bot or whatever`) because I want to reserve this functionality for other things in the future (e.g. choosing which room to join, if we have rooms).
• I'd like to have an in-chat /bot command to toggle bot-mode which will remove all formatting and escape codes. Ideally all the bot will see is ASCII with \r\n between messages.
• In general, we want to support enabling/disabling features based on what TERM is set. For example, there are terminals out there that support image-inlining, would be a cool future feature to detect that.
• TERM=dumb is a thing too, which seems to indicate support for a handful of escape codes, but we'd like to avoid using these so I'm not sure that makes more sense here.
• I'm still landing on TERM=bot as the API. Yes, there is a risk that somebody someday might write a terminal called bot for some reason and decide to break all compatibility with things that rely on the xterm-256color TERM value like curses-based apps. But I think that's a risk we can take? Worst case, users of that terminal can do TERM=xterm-256color ssh chat.shazow.net if they really wanna override it.
• When no PTY is allocated (e.g. when piping, or using ssh -T), the TERM value is not sent. In this scenario, the server will default it to TERM=bot. You can still override this behaviour by sending the TERM value explicitly TERM=foo ssh -o SendEnv=TERM if for example you want to pipe but you also want to use the normal mode.
• Bonus points: Make ssh-chat work with rlwrap out of the box? (Disable echo when there's no TTY #226)

My only concern right now is whether it'll be too hard to do the mid-session /bot toggle, if the implementation strategy will be to bypass the readline/terminal lib altogether in bot mode. We'd be transferring ownership of the io between two implementations which might prove annoying.

[chexo3]

I think the exec session request is probably the best idea. You can use the TERM variable in shell mode to enable support for extra terminal features.

[shazow]

@chexo3 Why is it the best idea? What about my reasoning do you disagree with?

[chexo3]

I'm sorry, I wasn't trying to impose. I'm not too versed with the ssh protocol, but I thought that using the exec system was an easy way to ask for a specific service and interface for that service. you don't have to worry about environment variable issues, you can do something like ssh <server> query/interfaces OR ssh <server> channel/<desired_channel>, and then, if you're a bot, you can do something like ssh <server> interfaces/binary-verbose/<interface_version>

[chexo3]

To clarify, so you can ask what interfaces are available (first example), join a specific channel/room, (second example), use a special interface (third example, could even have an TTS interface for eyesight-impaired people)

[shazow]

I don't disagree that having exec commands is a thing that is possible--it is in fact something I want to use for other features, I'm just trying to understand why that's better for describing what kind of terminal encoding is supported?

Could you expand on what you're referring to by "worry about environment variable issues"?

[chexo3]

You don’t have to worry about someone making a terminal or having their terminal set up in a way that it conflicts with the bot mode. In that sense the exec commands to enter bot mode are a command/endpoint like any other. Like /bot, but it’s more convenient for bots that need to start working immediately.

[shazow]

You don’t have to worry about someone making a terminal or having their terminal set up in a way that it conflicts with the bot mode.

As I mentioned in #104 (comment) I don't believe this is a problem, so I think we're fine.

In that sense the exec commands to enter bot mode are a command/endpoint like any other.

env commands as much commands as exec commands. It's just another kind of command in the request/response loop.

Like /bot, but it’s more convenient for bots that need to start working immediately.

FWIW I'm not sure interactive /bot mode will be possible easily, but we'll see.

[millerlogic]

It seems that the TERM is not even set if there's no pty (it's set with the pty-req, not env vars), and I personally don't know why a bot (or 3rd party client) would want a pty. Another solution might be to just have a different environment variable such as SSHCHAT_FORMAT, though shazow disagrees. I guess it's not a big deal to use TERM with a pty, I just don't see why.

Also since #333 supports SSHCHAT_THEME=mono a bot mode seems less important, we could potentially reserve a bot mode for something a bit better. I do like the idea of a JSON mode, however.

[shazow]

This was completeld in #341
and #343 makes the pty optional, you can now do both:

# With pty
TERM=bot ssh ssh.chat

# Without pty
ssh -T -o "SetEnv=TERM=bot" ssh.chat

From an ssh protocol perspective, you can use either pty-req sub-channel or env sub-channel to set the TERM.

I like the idea of adding a TERM=json someday but I'm closing this issue for now, we can open a fresh one for that. :)

[Low-power]

Publish the conversion happend on 2019-03-23 for some ideas.

[00:00:30] [2019/03/23]
[20:21:26] ssh-chat-relay: shazow: is there a problem with passing the TERM env value?
[20:22:46] Low-power: TERM environment will only be sent by ssh(1) when pty is enabled
[20:23:41] Low-power: and I think bot ususally will, and should disable the unneeded pty request
[20:23:46] Low-power: bots
[20:23:54] ssh-chat-relay: shazow: well then i can also default it to the bot api when there's no pty
[20:23:57] Low-power: ssh -T ...
...
[20:27:07] Low-power: active bot mode on pty disabled should also be an option, as long as this mode isn't hard for humans
[20:28:18] Low-power: in some cases, pty disabled for some reasons, while the pipe still operated by human
[20:28:29] ssh-chat-relay: shazow: okely
[20:28:39] ssh-chat-relay: shazow: ill make sure you can toggle it with /bot or something, just in case
[20:29:11] ssh-chat-relay: shazow: you could also force send the TERM even without pty with SendEnv, no?
[20:30:50] Low-power: Mar 23 20:30:34 public-chatroom sshd[65966]: Received env from 123.119.212.243:11525: TERM=xterm
[20:30:52] Low-power: yes
[20:31:04] ssh-chat-relay: shazow: :)
...
[21:35:54] Low-power: TERM var is sent as a part of pty request in SSH protocol
[21:36:17] Low-power: so it won't be available without a terminal
[21:36:38] ssh-chat-relay: robert: AFAIK, you can export any environment variable that you would like over ssh.
[21:37:26] ssh-chat-relay: robert: TERM is not special in this respect, except that it has a well-known purpose.
[21:37:31] Low-power: if using of 'SendEnv' is needed, why not choose another var name?
[21:38:12] Low-power: like SSH_CHAT_API_LEVEL=1
[21:38:47] ssh-chat-relay: robert: I would be fine with that, but $TERM is more conventional.
[21:39:27] ssh-chat-relay: robert: I would favor "SSH_CHAT_API_LEVEL=1" over "TERM=dumb", because the first is more explicit & readable.
[21:39:35] ssh-chat-relay: robert: ...and version-safe.
[21:42:29] Low-power: TERM was no longer conventional if pty request is disabled, for example, by 'ssh -T ...'
[21:43:45] ssh-chat-relay: robert: That might be a viable idea too, but then you lose the version number.
...
[22:28:37] Low-power: I'm not suggesting TERM=dumb
[22:28:47] ssh-chat-relay: shazow: Low-power: I mean specifically over TERM=bot
[22:29:08] ssh-chat-relay: shazow: my position re using the TERM var is what robert said, indicating support of escape codes etc is exactly what it's for
[22:30:32] ssh-chat-relay: shazow: i'll write up my thinking in an issue so we can have the record for context :) 
[22:31:09] Low-power: but this env var will losing its point by usual bots, it will be no different from other vars when pty is disabled
[22:31:32] ssh-chat-relay: shazow: Low-power: losing its point how?
[22:31:46] ssh-chat-relay: shazow: you can still pass it even without pty
[22:31:53] ssh-chat-relay: shazow: and without pty, it'll be enabled by default
[22:31:54] Low-power: TERM got specical handling by pty request
[22:32:07] ssh-chat-relay: shazow: Low-power: can you expand on that?
[22:32:11] Low-power: but it is not, if pty is disabled
[22:32:25] ssh-chat-relay: shazow: what part of the special handling presents a problem?
[22:35:17] Low-power: in my understanding, you said re-using TERM var, to indicate support of escape codes etc. is because this var got specical handling by SSH pty request
[22:36:01] ssh-chat-relay: shazow: I'm optimizing for two scenarios
[22:36:27] ssh-chat-relay: shazow: 1. "Hey the bot API is really simple, just type this: TERM=bot ssh chat.shazow.net"
[22:36:34] ssh-chat-relay: shazow: and it'll do the Right Thing
[22:36:52] ssh-chat-relay: shazow: 2. If you use it in a pipe without a pty (like you mentioned), it'll use the bot API by default to make that easier
[22:37:01] ssh-chat-relay: shazow: s/bot API/bot mode/
[22:37:22] ssh-chat-relay: shazow: when people start writing bots, first thing they want to do is try it out interactively
[22:37:38] ssh-chat-relay: shazow: which is #1
[22:37:57] ssh-chat-relay: shazow: and 3. we'll have a /bot command to toggle the mode mid-chat
[22:38:00] Low-power: so the first scenario just to set TERM to something invalid as a terminal, to try out
[22:38:05] ssh-chat-relay: shazow: one thing I'm uncertain about is what will happen with the prompt
[22:38:13] Low-power: but, why not just use '-T' for this
[22:38:15] ssh-chat-relay: shazow: Low-power: what do you mean?
[22:38:30] ssh-chat-relay: shazow: because we want to support custom terminal features too
[22:38:37] ssh-chat-relay: shazow: for example, there are terminals that support image inlining
[22:38:42] Low-power: 1. "Hey the bot API is really simple, just type this: ssh -T chat.shazow.net"
[22:38:50] ssh-chat-relay: shazow: we're going to be messing with and changing behaviour absed on the TERM var either way
[22:39:37] ssh-chat-relay: shazow: Low-power: the problem with that is exactly what you described: what if you want to use nopty but also not use the bot api?
[22:39:37] ssh-chat-relay: shazow: *mode
[22:39:48] ssh-chat-relay: shazow: also tbh TERM=bot ssh ... is more self-explanatory
[22:40:33] ssh-chat-relay: shazow: basically, TERM is going to default to "bot" (or whatever)
[22:40:36] ssh-chat-relay: shazow: if it's not provided, it'll be that
[22:40:41] ssh-chat-relay: shazow: if it's provided as bot, it'll be that
[22:40:55] ssh-chat-relay: shazow: if it's provided as something else, it'll be the normal experience
[22:42:15] ssh-chat-relay: shazow: TERM=dumb is certainly a thing, and i'm open to arguments why that makes more sense than TERM=bot, but I'm not seeing anything in the dumb functionality that we want to provide?
[22:42:52] ssh-chat-relay: shazow: it supports a handful of escape codes that bot will avoid
[22:43:45] Low-power: may be true for a simple try out, and if the bot mode is really different from human mode, explicit activition of it would be needed. consider someone writing a bot, and the server is expecting use 'TERM=bot' to active bot mode; since this var would only be sent with pty enabled, the bot author may no aware that TERM won't be sent to server it he uses pipe to communicate to its SSH client, this may causes confusing.
[22:45:04] Low-power: s/it he uses pipe/, if he uses pipe/
[22:47:15] ssh-chat-relay: shazow: if you pipe to the ssh client, you won't have a pty, and bot will be enabled by default, right?
[22:47:28] ssh-chat-relay: shazow: it's already super common for cli apps to provide different output when piped vs when run interactively
[22:47:47] ssh-chat-relay: shazow: most colorized loggers do this by default already
[22:48:52] Low-power: mostly for color and terminal paging
[22:50:16] ssh-chat-relay: shazow: that is exactly what we're doing here
[22:50:17] Low-power: but I think the output should be as similar as possible
[22:50:23] ssh-chat-relay: shazow: of course, it will be
[22:50:46] ssh-chat-relay: shazow: it'll be the exact same output but without color formatting and escape codes to move the cursor around