Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Corrupted MAC on input #2078

Open
3 tasks done
zhanglc opened this issue Jun 9, 2023 · 14 comments
Open
3 tasks done

Corrupted MAC on input #2078

zhanglc opened this issue Jun 9, 2023 · 14 comments
Assignees
@tgauth
Labels
Investigate Resolution - External

Comments

@zhanglc
Copy link

zhanglc commented Jun 9, 2023

Prerequisites

  • Write a descriptive title.
  • Make sure you are able to repro it on the latest version
  • Search the existing issues.

Steps to reproduce

ssh -vvv root@x.x.x.x

sshd_config:

#       $OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $

Include /etc/ssh/sshd_config.d/*.conf

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no

Banner /etc/ssh_login.warn

AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem       sftp    /usr/lib/openssh/sftp-server

sshd_algorithms.conf

MACs umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
kexAlgorithms curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
CASignatureAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
HostbasedAcceptedKeyTypes ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com
HostKeyAlgorithms ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com
PubkeyAcceptedKeyTypes ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com

ssh output:

OpenSSH_for_Windows_9.2p1, LibreSSL 3.7.2
debug3: Failed to open file:C:\\XXX/.ssh/config error:2
debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2
debug2: resolve_canonicalize: hostname x.x.x.x is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> 'C:\\XXX/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> 'C:\\XXX/.ssh/known_hosts2'
debug3: ssh_connect_direct: entering
debug1: Connecting to x.x.x.x [x.x.x.x] port 22.
debug1: Connection established.
debug1: identity file C:\\XXX/.ssh/id_rsa type 0
debug3: Failed to open file:C:\\XXX/.ssh/id_rsa-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_rsa-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_rsa-cert error:2
debug1: identity file C:\\XXX/.ssh/id_rsa-cert type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ecdsa error:2
debug1: identity file C:\\XXX/.ssh/id_ecdsa type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ecdsa-cert error:2
debug1: identity file C:\\XXX/.ssh/id_ecdsa-cert type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa_sk error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa_sk.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ecdsa_sk error:2
debug1: identity file C:\\XXX/.ssh/id_ecdsa_sk type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa_sk-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ecdsa_sk-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ecdsa_sk-cert error:2
debug1: identity file C:\\XXX/.ssh/id_ecdsa_sk-cert type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519 error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ed25519 error:2
debug1: identity file C:\\XXX/.ssh/id_ed25519 type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ed25519-cert error:2
debug1: identity file C:\\XXX/.ssh/id_ed25519-cert type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519_sk error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519_sk.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ed25519_sk error:2
debug1: identity file C:\\XXX/.ssh/id_ed25519_sk type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519_sk-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_ed25519_sk-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_ed25519_sk-cert error:2
debug1: identity file C:\\XXX/.ssh/id_ed25519_sk-cert type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_xmss error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_xmss.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_xmss error:2
debug1: identity file C:\\XXX/.ssh/id_xmss type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_xmss-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_xmss-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_xmss-cert error:2
debug1: identity file C:\\XXX/.ssh/id_xmss-cert type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_dsa error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_dsa.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_dsa error:2
debug1: identity file C:\\XXX/.ssh/id_dsa type -1
debug3: Failed to open file:C:\\XXX/.ssh/id_dsa-cert error:2
debug3: Failed to open file:C:\\XXX/.ssh/id_dsa-cert.pub error:2
debug3: failed to open file:C:\\XXX/.ssh/id_dsa-cert error:2
debug1: identity file C:\\XXX/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_9.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5
debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to x.x.x.x:22 as 'secur1ty'
debug3: record_hostkey: found key type ECDSA in file C:\\XXX/.ssh/known_hosts:5
debug3: load_hostkeys_file: loaded 1 keys from x.x.x.x
debug3: Failed to open file:C:\\XXX/.ssh/known_hosts2 error:2
debug1: load_hostkeys: fopen C:\\XXX/.ssh/known_hosts2: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
debug2: host key algorithms: ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
debug2: MACs stoc: umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes128-ctr MAC: umac-128-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: umac-128-etm@openssh.com compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:JQXEGbJuKdCsTF0b0Jksdzb8ipQYBD5i5toaqGnvKII
debug3: record_hostkey: found key type ECDSA in file C:\\XXX/.ssh/known_hosts:5
debug3: load_hostkeys_file: loaded 1 keys from x.x.x.x
debug3: Failed to open file:C:\\XXX/.ssh/known_hosts2 error:2
debug1: load_hostkeys: fopen C:\\XXX/.ssh/known_hosts2: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'x.x.x.x' is known and matches the ECDSA host key.
debug1: Found key in C:\\XXX/.ssh/known_hosts:5
debug3: send packet: type 21
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug3: ssh_get_authentication_socket_path: path '\\\\.\\pipe\\openssh-ssh-agent'
debug2: get_agent_identities: ssh_agent_bind_hostkey: invalid format
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: C:\\XXX/.ssh/id_rsa RSA SHA256:4HFB+dv5MV9gamIRyXzVFpiGnxyG9buP72FRX/CfGzA
debug1: Will attempt key: C:\\XXX/.ssh/id_ecdsa
debug1: Will attempt key: C:\\XXX/.ssh/id_ecdsa_sk
debug1: Will attempt key: C:\\XXX/.ssh/id_ed25519
debug1: Will attempt key: C:\\XXX/.ssh/id_ed25519_sk
debug1: Will attempt key: C:\\XXX/.ssh/id_xmss
debug1: Will attempt key: C:\\XXX/.ssh/id_dsa
debug2: pubkey_prepare: done
debug3: send packet: type 5
Corrupted MAC on input.
ssh_dispatch_run_fatal: Connection to x.x.x.x port 22: message authentication code incorrect

Expected behavior

ssh login success

Actual behavior

Corrupted MAC on input.
ssh_dispatch_run_fatal: Connection to x.x.x.x port 22: message authentication code incorrect

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      5.1.19041.2673
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.2673
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Version

OpenSSH_for_Windows_9.2p1, LibreSSL 3.7.2

Visuals

No response
@tgauth
Copy link
Collaborator

tgauth commented Jun 9, 2023

Based on https://serverfault.com/questions/994646/ssh-on-windows-corrupted-mac-on-input, run: ssh -Q mac then add -m <one of the macs from the output of ssh-Q mac> to the ssh -vvv root@x.x.x.x command

@zhanglc
Copy link
Author

zhanglc commented Jun 12, 2023

Client and Server had negotiated the mac algorithm, So the way you say it is not feasible @tgauth

debug1: kex: server->client cipher: aes128-ctr MAC: umac-128-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: umac-128-etm@openssh.com compression: none

@tgauth
Copy link
Collaborator

tgauth commented Jun 12, 2023

Could you still try the -m option explicitly? The error message is the same as in the link above, and that was the solution.

hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com are also listed in sshd_algorithms.conf, could you remove umac-128-etm@openssh.com and see if one of the other MACs works?

Also, have you tried updating the server from OpenSSH 8.4 to a more current version?
Or have you seen this issue with any other SSH clients aside from Windows OpenSSH 9.2?

@zhanglc
Copy link
Author

zhanglc commented Jun 13, 2023

@tgauth It works when use ssh -m hmac-sha2-256-etm@openssh.com xxx@x.x.x.x, thanks a lot .

By the way , why the algo umac-128-etm@openssh.com not work ? different implament between OpenSSH 8.4 and OpenSSH_for_Windows_9.2p1?

@tgauth
Copy link
Collaborator

tgauth commented Jun 13, 2023

Great!

It seems like this might be related to the crypto libraries used with each OpenSSH and their implementations of umac-128-etm@openssh.com - see #1359. Do you know if the server has FIPS enabled?

@maertendMSFT maertendMSFT added the Waiting on Author Need more information to diagnose label Jun 19, 2023
@ket000
Copy link

ket000 commented Jan 5, 2024 •
edited

I have installed the latest version OpenSSH_for_Windows_9.5p1, LibreSSL 3.8.2 and issue is still there when below macs are used.

umac-128-etm@openssh.com
umac-128@openssh.com

To workaround, I have added the below to my %USERPROFILE%/.ssh/config file

Host *
MACs=hmac-sha2-256,hmac-sha2-512,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com

or use the below command line option wen invoking ssh

ssh -oMACs=hmac-sha2-256,hmac-sha2-512,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com username@server.com

This has resolved the issue. Same works fine from putty, mac, Red Hat Linux 8 and 7 without any issues, so issue seems to be with Windows OpenSSH client. Hopefully it gets fixed eventually so no workaround is needed.

@maertendMSFT
Copy link
Collaborator

We believe that the issue is coming from LibreSSL, specifically: libressl/portable#603. We will see if we can get any traction on the thread.

@maertendMSFT maertendMSFT removed the Waiting on Author Need more information to diagnose label Jan 8, 2024
@ket000
Copy link

ket000 commented Jan 8, 2024

Thanks for the update. Hopefully libressl version for windows can be fixed. I noticed this issue when i disabled chacha20-poly1305@openssh.com encryption algorithm on linux 8.9 server and windows / linux decided to use umac-128-etm@openssh.com to mitigate terrapin attack.

@ByteEnable
Copy link

ByteEnable commented Jan 10, 2024 •
edited

I have experienced the same issue as @ket000 .

Corrupted MAC on input.
ssh_dispatch_run_fatal: Connection to XXX.XXX.XXX.XXX port 22: message authentication code incorrect

Version:
OpenSSH_for_Windows_8.6p1, LibreSSL 3.4.3

Workaround:
ssh -m hmac-sha2-512 user@host

@pzxocs
Copy link

pzxocs commented Jan 12, 2024 •
edited

Reporting the same issue.
Corrupted MAC on input.
ssh_dispatch_run_fatal: Connection to * port 22: message authentication code incorrect

ssh -V on client:
OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
server:
OpenSSH_8.0p1, OpenSSL 1.1.1k FIPS 25 Mar 2021

@ket000 solution worked for me

@jansohn jansohn mentioned this issue Jan 18, 2024
Closed
3 tasks
@bsteinb
Copy link

bsteinb commented Jan 23, 2024

It seems to be confirmed now that the MAC algorithms umac-128-etm@openssh.com and umac-128@openssh.com are not working at the moment. Instead of relying on user configuration to work around the issue, would it make sense to push a new release that completely disables these algorithms?

@johnwick1
Copy link

johnwick1 commented Feb 5, 2024 •
edited

having the same issue here on Oracle Linux 8 with openssh-server-8.0p1-19.el8_9.2.x86_64 (latest package available)

"Corrupted MAC on input" with putty but fine with the ssh from windows cmd

logs:
Corrupted MAC on input. [preauth]
ssh_dispatch_run_fatal: Connection from 1.2.3.4 port 12345: message authentication code incorrect [preauth]

i downgraded openssh-server to openssh-server-8.0p1-19.el8_8.x86_64 (dnf downgrade openssh-server)

then it works

@ket000
Copy link

ket000 commented Feb 5, 2024

This is a known issue since Redhat updated the openssh-server to newer version to resolve the terrapin vulnerability. The same is true for oracle linux 8. Oracle has pulled the offending version from their repo today Feb 5th 2024.

To resolve the issue,
sudo dnf downgrade openssh-server
sudo dnf clean all

Last command will make sure package cache is clean and no longer install the offending version. Reference links

oracle/oracle-linux#125
https://access.redhat.com/security/cve/cve-2023-48795?cmdf=CVE-2023-48795+redhat

Version that causes the issue : openssh-server-8.0p1-19.el8_9.2.x86_64 which makes below cipher not to work including putty 0.80 and other ubuntu servers

chacha20-poly1305@openssh.com
aes128-ctr
aes192-ctr
aes256-ctr

When you downgrade, all 3 packages will be updated to lower version : 8.0p1-19.el8_9.2
openssh
openssh-clients
openssh-server

@ebenhoehdaniel
Copy link

ebenhoehdaniel commented Apr 26, 2024 •
edited

Error is still there.
No newer Version than v9.5.0.0p1-Beta (LibreSSL 3.8.2)

And Windows Server 2022 is affected "Feature OpenSSH" (OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2)

We need a bugfixed version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tgauth tgauth

Labels
Investigate Resolution - External
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@bsteinb @zhanglc @ByteEnable @tgauth @ket000 @johnwick1 @pzxocs @maertendMSFT @ebenhoehdaniel