Corkscrew in ssh config file won't work #2160

Open
metal3d opened this Issue Jun 30, 2015 · 23 comments

Projects

None yet

8 participants

@metal3d
metal3d commented Jun 30, 2015

Hello,

I'm working behind an HTTP proxy and I need to connect to SSH server via "corkscrew". Appending in ~/.ssh.config:

Host *
    ProxyCommand corkscrew ip.of.my.proxy 8080 %h %p

That works with zsh, bash and sh but not with fish. Is there any chance for me to be able to make it working ?

Thanks a lot

(Fedora 22 x64 - Fish 2.1.1)

@ridiculousfish
Member

What exactly doesn't work?

@metal3d
metal3d commented Jul 1, 2015

I'm not sure but, look at this example:

➜  ~ bash
[pferlet@patrice-laptop ~]$ ssh xxx@metal3d.org
xxx@metal3d.org's password: 
#I'm connected
$ exit
Connection to metal3d.org closed.
[pferlet@patrice-laptop ~]$ exit
exit
# back in fish
➜  ~ ssh xxx@metal3d.org 

The last command is in wait... no password prompt.
With bash, corkscrew is in use, it pass proxy and it works like a charm but not with fish.

EDIT: I try without corkscrew in config to be sure:

➜  ~ ssh xxx@metal3d.org 
ssh: connect to host metal3d.org port 22: Connection refused

So, ssh config is used, but somthing goes wrong with corkscrew.

Note that at home, I remove the corkscrew command from my ssh/config (because I don't use proxy at home) and fish let me connect in ssh without any problem.

Thanks

EDIT: note that while ssh command is in wait, I'm checking with netstat if a connexion is made to proxy or to my ssh server and none of them are found. Maybe the ssh config is not used with fish (That's a hypothesis without any research ;))

@metal3d
metal3d commented Jul 1, 2015

Tried with ssh key appended to my server to check if it's the password prompt, and the problem is still there.

@nitishch
Contributor
nitishch commented Jul 1, 2015

I am not much familiar with fish-shell but, to know if ssh is using the config file or not, run the command

strace ssh xxx@metal3d.org > logFile

Check if the string .ssh/config appears in the logFile

@metal3d
metal3d commented Jul 1, 2015

I edited my last comment to note that corkscrew and .ssh/config seems to be in use (without the corkscrew command in config, the connexion is immediatly refused) but I will do the strace command to be sure

@metal3d
metal3d commented Jul 1, 2015

@nitishch the file config appears in logs, but no trace about corkscrew (same with bash)

@metal3d
metal3d commented Jul 1, 2015

Ha, it seems that corkscrew is in use with fish + config file:

➜  ~ ps ax | grep cork
10780 pts/2    S+     0:00 corkscrew my.proxy.address 8080 metal3d.org 22

So the problem should be the packet redirection...

(note that I change proxy addresses because that's the proxy of my client I'm working for ;) )

@nitishch
Contributor
nitishch commented Jul 1, 2015

This is a long shot but, may be -v option to ssh would give a clue.

@metal3d
metal3d commented Jul 1, 2015

mmmh... with bash:

pferlet@patrice-laptop ~]$ ssh -v xxx@metal3d.org  
OpenSSH_6.8p1, OpenSSL 1.0.1k-fips 8 Jan 2015
debug1: Reading configuration data /home/pferlet/.ssh/config
debug1: /home/pferlet/.ssh/config line 10: Applying options for metal3d.org
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Executing proxy command: exec corkscrew proxy.server.address 8080 metal3d.org 22
debug1: permanently_drop_suid: 1002
debug1: identity file /home/pferlet/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.0p1 Debian-4+deb7u2
debug1: match: OpenSSH_6.0p1 Debian-4+deb7u2 pat OpenSSH* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr umac-64@openssh.com none
debug1: kex: client->server aes128-ctr umac-64@openssh.com none
debug1: kex: ecdh-sha2-nistp256 need=16 dh_need=16
debug1: kex: ecdh-sha2-nistp256 need=16 dh_need=16
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:3SbTF4HtpVhKGNn3XbYIv5UBVIYN0KYqieqn5nKtiBA
debug1: Host 'metal3d.org' is known and matches the ECDSA host key.
debug1: Found key in /home/pferlet/.ssh/known_hosts:5
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/pferlet/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 535
debug1: Authentication succeeded (publickey).
Authenticated to metal3d.org (via proxy).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env XMODIFIERS = @im=ibus
debug1: Sending env LANG = fr_FR.utf8

And with fish:

OpenSSH_6.8p1, OpenSSL 1.0.1k-fips 8 Jan 2015
debug1: Reading configuration data /home/pferlet/.ssh/config
debug1: /home/pferlet/.ssh/config line 10: Applying options for metal3d.org
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug1: Executing proxy command: exec corkscrew proxy.server.address 8080 metal3d.org 22
debug1: permanently_drop_suid: 1002
debug1: identity file /home/pferlet/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/pferlet/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.8
debug1: ssh_exchange_identification: \033]4;0;rgb:18/18/18\033\\\033]4;1;rgb:ab/46/42\033\\\033]4;2;rgb:a1/b5/6c\033\\\033]4;3;rgb:f7/ca/88\033\\\033]4;4;rgb:7c/af/c2\033\\\033]4;5;rgb:ba/8b/af\033\\\033]4;6;rgb:86/c1/b9\033\\\033]4;7;rgb:d8/d8/d8\033\\\033]4;8;rgb:58/58/58\033\\\033]4;9;rgb:ab/46/42\033\\\033]4;10;rgb:a1/b5/6c\033\\\033]4;11;rgb:f7/ca/88\033\\\033]4;12;rgb:7c
debug1: ssh_exchange_identification: /af/c2\033\\\033]4;13;rgb:ba/8b/af\033\\\033]4;14;rgb:86/c1/b9\033\\\033]4;15;rgb:f8/f8/f8\033\\\033]4;16;rgb:dc/96/56\033\\\033]4;17;rgb:a1/69/46\033\\\033]4;18;rgb:28/28/28\033\\\033]4;19;rgb:38/38/38\033\\\033]4;20;rgb:b8/b8/b8\033\\\033]4;21;rgb:e8/e8/e8\033\\\033]10;rgb:d8/d8/d8\033\\\033]11;rgb:18/18/18\033\\\033]12;rgb:d8/d8/d8\033\\S
debug1: ssh_exchange_identification: SH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2


debug1: ssh_exchange_identification: 
debug1: ssh_exchange_identification: 128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug1: ssh_exchange_identification: bc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se

logs seems to not be the same

@zanchey
Member
zanchey commented Jul 1, 2015

There's something strange going on there - the remote identification string starts with nonsense (which looks an awful lot like terminal control characters).

type ssh is definitely /usr/bin/ssh under fish, right? The full output of strace -ff would be handy, but you might want to sanitise the output first.

@metal3d
metal3d commented Jul 6, 2015

I did the strace -ff, I cleanup the output to not flood the issue. I guess that the following log can help:

What I did is that I tried to connect with bash then fish, then I compared both logs.

The begining of the log seems to be very similar, but the 811 line change radically.

[pid 12278] socket(PF_LOCAL, SOCK_STREAM, 0) = 3
[pid 12278] connect(3, {sa_family=AF_LOCAL, sun_path="/run/user/1002/fishd.socket"}, 110) = 0
[pid 12278] fcntl(3, F_GETFL)           = 0x2 (flags O_RDWR)
[pid 12278] fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 12278] fcntl(3, F_SETFD, FD_CLOEXEC) = 0
[pid 12278] write(3, "BARRIER\n", 8)    = 8
[pid 12278] select(4, [3], NULL, NULL, NULL) = 1 (in [3])
[pid 12278] read(3, "# Fish universal variable daemon"..., 1024) = 499
[pid 12278] read(3, "SET fish_color_quote:brown\nSET f"..., 1024) = 587
[pid 12278] read(3, 0x7ffe117bb8b0, 1024) = -1 EAGAIN (Resource temporarily unavailable)
[pid 12278] getcwd("/home/pferlet/src/OFS-BO-Pro", 24576) = 29
[pid 12278] ioctl(0, TCGETS, 0x7ffe117bc210) = -1 ENOTTY (Inappropriate ioctl for device)
[pid 12278] rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], NULL, 8) = 0
[pid 12278] rt_sigprocmask(SIG_UNBLOCK, ~[RTMIN RT_1], NULL, 8) = 0
[pid 12278] open("/usr/share/fish/config.fish", O_RDONLY|O_CLOEXEC) = 4
[pid 12278] fstat(4, {st_mode=S_IFREG|0644, st_size=4247, ...}) = 0
[pid 12278] lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 12278] lstat("/usr/share", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
[pid 12278] lstat("/usr/share/fish", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 12278] lstat("/usr/share/fish/config.fish", {st_mode=S_IFREG|0644, st_size=4247, ...}) = 0
[pid 12278] fcntl(4, F_GETFL)           = 0x8000 (flags O_RDONLY|O_LARGEFILE)
[pid 12278] fstat(4, {st_mode=S_IFREG|0644, st_size=4247, ...}) = 0
[pid 12278] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a0d81d000
[pid 12278] read(4, "#\n# Main file for fish command c"..., 4096) = 4096
[pid 12278] read(4, " \"source: '.' command is depreca"..., 4096) = 151
[pid 12278] read(4, "", 4096)           = 0
[pid 12278] close(4)                    = 0
[pid 12278] munmap(0x7f6a0d81d000, 4096) = 0

Then after a while:

[pid 12278] read(3, "HTTP/1.1 200 Connection Establis"..., 4096) = 65
[pid 12278] write(1, "", 0)             = 0
[pid 12278] select(4, [0 3], [], NULL, {5, 0}) = 1 (in [0], left {4, 999999})
[pid 12278] read(0, "SSH-2.0-OpenSSH_6.8\r\n", 4096) = 21
[pid 12278] write(3, "SSH-2.0-OpenSSH_6.8\r\n", 21) = 21
[pid 12278] select(4, [0 3], [], NULL, {5, 0}) = 1 (in [3], left {4, 970201})
[pid 12278] read(3, "SSH-2.0-OpenSSH_6.0p1 Debian-4+d"..., 4096) = 39
[pid 12278] write(1, "SSH-2.0-OpenSSH_6.0p1 Debian-4+d"..., 39) = 39

That mean, I gues, that proxy responds with the ssh connection. I see the same in the bash log.

But while bash log says that it reads known_host file, the fish log does'nt. This at the end of log.

At the end of log:

[pid 12277] read(5, "\0", 1)            = 1
[pid 12277] read(5, "\0", 1)            = 1
[pid 12277] read(5,  <unfinished ...>
[pid 12278] <... select resumed> )      = 0 (Timeout)
[pid 12278] select(4, [0 3], [], NULL, {5, 0}) = 0 (Timeout)
[pid 12278] select(4, [0 3], [], NULL, {5, 0}) = 0 (Timeout)
[pid 12278] select(4, [0 3], [], NULL, {5, 0}) = 0 (Timeout)
 # ... indefinitly the same line repeated....

That's all I can say... If you really need my complete log, tell me.

@metal3d
metal3d commented Jul 6, 2015

(I edited my last comment, a log was copied twice)

@metal3d
metal3d commented Jul 6, 2015

My god... I found the problem...
Because I saw some "color strings" sent, I wondered if the base16 color script breaks something (https://github.com/chriskempson/base16-shell)
So I commented the eval sh $HOME/.config/base16-shell/base16-default.dark.sh that the author ask us to set in config.fish... and miracle: connection is ok...

Sorry for that, I will open an issue in base 16 repo - referencing this one. Thanks for your help and ideas (strace helped me)

@metal3d metal3d referenced this issue in chriskempson/base16-shell Jul 6, 2015
Open

Base 16 breaks corckscrew connection with Fish #42

@faho
Member
faho commented Jul 15, 2015

Is there an issue in fish here?

@metal3d
metal3d commented Jul 15, 2015

I'm not sure about the problem. Base-16 works like a charm with bash and I've got no problem to connect ssh throught corkscrew. But with fish, a problem appears. So either the problem is that base-16 makes something wrong, or this is fish that makes something wrong after calling the base-16 shell script.

@zanchey
Member
zanchey commented Jul 16, 2015

base-16 is some serious terminal voodoo. I don't think the eval bit is needed - could you try it with just sh $HOME/.config/base16-shell/base16-default.dark.sh in your config.fish.?

@zanchey
Member
zanchey commented Jul 16, 2015

Also, I haven't been able to reproduce this on my system with gnome-terminal, fish 2.2.0 or 2.1.2, base16-devel, corkscrew and TinyHTTPProxy. Which terminal are you using?

@metal3d
metal3d commented Jul 16, 2015

With or without eval, the result is the same.
This is the proxy of my client, it seems to be squid but I'm not sure.

IMHO the problem is that fish inserts control char for color on a way it souldn't. It's particuliary weird that this problem appears only with fish and not with bash, sh or zsh.

That's not a serious problem while base-16 is a gadget that changes colors to be "smooth". But I guess this is interessing to understand what happends and how to fix it.

@zanchey
Member
zanchey commented Jul 16, 2015

I think I've worked it out now. The problem is that base16 gets executed on every shell invocation with the suggested installation instruction. In bash and zsh, .bashrc/.zshrc only get executed for interactive shells, but config.fish is executed for every shell. base16 works by spewing a bunch of control characters to the terminal, but in this case the terminal is stdin of ssh.

Try this instead in your config.fish:

if status --is-interactive
    sh $HOME/.config/base16-shell/base16-default.dark.sh
end

The reason I couldn't reproduce this is that I don't use fish as my login shell - instead I exec it from the bottom of my .zshrc. (ssh's ProxyCommand assumes that your login shell is sh-compatible, which is probably unwise.)

@cycomanic

I just ran into this as well. This was a tough nut to crack, I was actually suspecting a misconfigured http proxy. As stated above this is due to the fact that fish executes config.fish for every shell. This not related to base-16-default.dark.sh (which I don't use), but happens with most adjustments for the prompt I suspect.
When fish.config gets executed control characters get spewed onto the response from the proxy server which prevents ssh to continue the indentification below are the relevant lines from my ssh -vvv:

debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4
debug1: ssh_exchange_identification: \033[30m\033(B\033[mSSH-2.0-libssh-0.7.0

debug1: ssh_exchange_identification:
debug1: ssh_exchange_identification: \024\213{\231\026\211\306&\275e\275\350\225\304\002\255p
debug1: ssh_exchange_identification:

I could fix this by placing if status --is-interactive around my plugins and themes.

I wonder if fish should not invoke fish.config when called not interactively, because I imagine this could trigger some other hard to track bugs.

@krader1961
Member

I wonder if fish should not invoke fish.config when called not interactively, ...

Then we would have to have another mechanism where people can put initialization they want done for every shell (e.g., setting env vars). Too, consider that even in shells like bash and zsh which have a multiplicity of config files that are sourced under various conditions it is still trivial to break a ssh connection by putting something into one of them that injects unexpected characters into the output stream. Having just reread the "INVOCATION" section of man bash I don't think we want to emulate it.

@kopischke

Having just reread the "INVOCATION" section of man bash I don't think we want to emulate it.

Having just recently switched from bash to fish, I could not agree more.

@zanchey
Member
zanchey commented May 21, 2016

Perhaps we should ship an example config.fish that makes this clear.

@krader1961 krader1961 added this to the fish-future milestone May 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment