FIDO_ERR_RX with openssh 8.3 and Yubikey 5Ci

[ismail]

I promise I read all the other bug reports :-)

First, the key info:

sisyphus ~ > ykman info
Device type: YubiKey 5Ci
Serial number: 14411044
Firmware version: 5.2.6
Form factor: Keychain (USB-C, Lightning)
Enabled USB interfaces: OTP+FIDO+CCID

Applications
OTP     	Enabled	
FIDO U2F	Enabled	
OpenPGP 	Enabled	
PIV     	Enabled	
OATH    	Enabled	
FIDO2   	Enabled

and trying to create an ecdsa-sk key:

sisyphus ~ > ssh-keygen -vvv -t ecdsa-sk -f ~/.ssh/id_ecdsa_sk
Generating public/private ecdsa-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=3184
debug3: ssh_msg_send: type 5
debug3: ssh_msg_recv entering
debug1: start_helper: starting /usr/lib/ssh/ssh-sk-helper 
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: ssh_sk_enroll: using device /dev/hidraw5
debug1: ssh_sk_enroll: fido_dev_open: FIDO_ERR_RX
debug1: sshsk_enroll: provider "internal" returned failure -1
debug1: ssh-sk-helper: Enrollment failed: invalid format
debug1: ssh-sk-helper: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -4
debug3: reap_helper: pid=3184
Key enrollment failed: invalid format
sisyphus ~ > cccccctnugdfjnubhvhfıenhhjıvefbtllvvllıcgnkk
zsh: command not found: cccccctnugdfjnubhvhfıenhhjıvefbtllvvllıcgnkk

This happens with both libfido 1.4.0 and the latest git checkout. I set a breakpoint on fido_dev_open and it fails at https://github.com/Yubico/libfido2/blob/master/src/io.c#L157

rx_frame (d=0x5644ddba6f50, fp=0x7fffeefa2930, ms=-1) at ../src/io.c:157
157		    (unsigned char *)fp, d->rx_len, ms)) < 0 || (size_t)n != d->rx_len)
(gdb) l
152		int n;
153	
154		memset(fp, 0, sizeof(*fp));
155	
156		if (d->rx_len > sizeof(*fp) || (n = d->io.read(d->io_handle,
157		    (unsigned char *)fp, d->rx_len, ms)) < 0 || (size_t)n != d->rx_len)
158			return (-1);
159	
160		return (0);
161	}
(gdb) print d->rx_len
$1 = 64
(gdb) print sizeof(*fp)
$2 = 64
(gdb) print n
$3 = 8

So, n != d->rx_len and the function returns -1.

[martelletto]

Hi, Thank you for the bug report. Could you capture the output of FIDO_DEBUG=1 with examples/cred?

…

-p.

[ismail]

Attached output for FIDO_DEBUG=1 ./examples/cred
cred.txt

[martelletto]

Thank you. examples/cred seems to work as expected (assuming you have configured a PIN). Is the problem reproducible? If so, can you try to get FIDO_DEBUG=1 output from ssh-keygen?

…

-p.

[ismail]

Yes, sadly this is reproducible every time and FIDO_DEBUG=1 ssh-keygen -vvv -t ecdsa-sk -f ~/.ssh/id_ecdsa_sk produces no new debug output. I wondered if ssh-sk-helper not even using libfido2 at all, but that's not the case:

sisyphus ~ > ldd /usr/lib/ssh/ssh-sk-helper | grep fido 
        libfido2.so.1 => /usr/lib64/libfido2.so.1 (0x00007efed846c000)

[martelletto]

My bad; I thought 'FIDO_DEBUG=1 ssh-keygen' would work with OpenSSH 8.3, but now I see that the call to fido_init() was added after 8.3 was tagged. We have two options: a) build OpenSSH from HEAD (it is less work than it sounds) and run 'FIDO_DEBUG=1 ssh-keygen'; b) build libfido2 with debugging enabled by default, and use LD_LIBRARY_PATH to load our modified libfido2 instead of /usr/lib64/libfido2.so.1, with ssh as-is.

…

-p.

[ismail]

Hah! I just built openssh-portable.git and got:

sisyphus ~/t/openssh-portable > FIDO_DEBUG=1 ./dist/bin/ssh-keygen -vvv -t ecdsa-sk -f ~/.ssh/id_ecdsa_sk
Generating public/private ecdsa-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=3960
debug3: ssh_msg_send: type 5
debug3: ssh_msg_recv entering
debug1: start_helper: starting /home/ismail/t/openssh-portable/dist/lib/ssh-sk-helper
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: ssh_sk_enroll: using device /dev/hidraw5
fido_tx: d=0x56072db07040, cmd=0x06, buf=0x56072db07040, count=8
0000: 10 1b 8d 14 2f db bf 91
fido_rx: d=0x56072db07040, cmd=0x06, buf=0x56072db07048, count=17, ms=-1
rx: rx_preamble
fido_dev_open_rx: fido_rx
debug1: ssh_sk_enroll: fido_dev_open: FIDO_ERR_RX
debug1: sshsk_enroll: provider "internal" returned failure -1
debug1: ssh-sk-helper: Enrollment failed: invalid format
debug1: ssh-sk-helper: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -4
debug3: reap_helper: pid=3960
Key enrollment failed: invalid format

[martelletto]

Thanks :) Are you using a HID backend different than hid_linux.c? Do you have more than one security key attached to your machine? If the answer is 'no' to both questions, I am afraid we will need to instrument hid_linux.c's fido_hid_read() to see what it is doing. From a glance, I don't see how it can return 8 when passed a length of 64.

…

-p.

[ismail]

Now, we are getting somewhere, if I compile with -DUSE_HIDAPI=0 it works fine, so this might be a hidapi bug? More importantly, will we lose anything if we disable hidapi support? Except that I'll gain a working YubiKey, asking for with my package maintainer hat on :-)

[martelletto]

I would expect hidapi to do a read(2) on /dev/hidrawX just like hid_linux.c. Does strace reveal anything?

…

-p.

[ismail]

strace shows read(3, "\0\0\6\0\0\0\0\0", 64) = 8 but more importantly, if I disable hidapi, ssh-keygen asks me for a PIN first (as there is a PIN set indeed).

[ismail]

Ok problem is elsewhere, hidapi opens /dev/hidraw5 hid_linux opens /dev/hidraw6

Looking at dmesg hidraw5 is the keyboard interface:

[23127.826739] hid-generic 0003:1050:0407.0010: input,hidraw5: USB HID v1.10 Keyboard [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:00:14.0-2/input0
[23127.827175] hid-generic 0003:1050:0407.0011: hiddev98,hidraw6: USB HID v1.10 Device [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:00:14.0-2/input1

[martelletto]

It looks like https://github.com/Yubico/libfido2/blob/master/src/hid_hidapi.c#L162 is missing '|| defined(__linux__)'.

…

-p.

[ismail]

That doesn't help because usage_page is 0x4800 for both devices.

[ismail]

I think https://github.com/Yubico/libfido2/blob/master/src/hid_linux.c#L260 eliminates the input interface.

[martelletto]

IIRC (it has been a while since I wrote the code), what eliminates the input interface in hid_linux.c is is_fido(). Since hidapi doesn't set usage_page on Linux (as you correctly pointed out), we would need similar code in hid_hidapi.c.

…

-p.

[ismail]

IIRC (it has been a while since I wrote the code), what eliminates the input interface in hid_linux.c is is_fido(). Since hidapi doesn't set usage_page on Linux (as you correctly pointed out), we would need similar code in hid_hidapi.c.

That sounds like the right fix. For now, I disabled hidapi usage for openSUSE Linux and asked someone to test (for them it worked with hidapi too).

[martelletto]

Tentative fix developed in https://github.com/Yubico/libfido2/tree/hidapi_linux. I am planning to test it over the weekend, if I find the time.

[martelletto]

Tested on Ubuntu 20.04; macOS and Windows pending.

[martelletto]

Fixed in 01236dd and 95ae664. Thanks for the report!