ykcs11 performance problem

[dwmw2]

In various usage, ykcs11 seems fairly slow. For a simple openssl pkeyutl -sign operation, ykcs11 seems to take quite a lot longer than OpenSC does...

$ time openssl pkeyutl -engine pkcs11 -keyform engine -sign -in foo -out bar \
        -inkey 'pkcs11:model=YubiKey%20NEO;id=%01;pin-value=1234' 
#engine "pkcs11" set.

real    0m3.771s
user    0m0.033s
sys 0m0.019s

vs. OpenSC:

$ time openssl pkeyutl -engine pkcs11 -keyform engine -sign -in foo -out bar \
        -inkey 'pkcs11:manufacturer=piv_II;id=%01;pin-value=1234'
engine "pkcs11" set.

real    0m1.763s
user    0m0.024s
sys 0m0.020s

[mouse07410]

Semi-on-topic: how do you use OpenSSL with YKCS11?

[dwmw2]

Um, as shown above? ☺

Although first I had to fix:

• ykcs11 doesn't install p11-kit module file #92 so it actually gets loaded into applications
• ykcs11 reports 48 slots #91 so it enumerated its slots properly when probed
• ykcs11: Allow C_Login() with short PIN #95 so it actually worked with my test token's PIN of 1234
• C_OpenSession fails with compressed certificates on device #93 so that it worked with compressed certificates

You might not need the last two, but you probably will need the first two.

[dwmw2]

And now it should fairly much Just Work™ with any application in the system:

• openconnect -c 'pkcs11:model=YubiKey%20NEO;id=%01;pin-value=1234' …
• openvpn --pkcs11-id 'pkcs11:model=YubiKey%20NEO;id=%01;pin-value=1234' …
• curl -E 'pkcs11:model=YubiKey%20NEO;id=%01;pin-value=1234' …
• In wpa_supplicant.conf: client_cert="pkcs11:model=YubiKey%20NEO;id=%01;pin-value=1234"

Note that I need to build curl against GnuTLS for it to work correctly. For curl built with OpenSSL you have to manually tell it to use engine_pkcs11 — although I'm about to fix that because it should work that out for itself if you give it a PKCS#11 URI, of course. And for NSS there are these two bugs with patches attached, after which it works.

If you find any application in Fedora which doesn't Just Work™ then please file bugs. Other distributions are... distinctly less coherent about all this crypto stuff and many don't even have system-wide CA trust sorted out so that you trust a CA just once and it works for everyone. But why not file bugs there too :)

[mouse07410]

Semi-on-topic: how do you use OpenSSL with YKCS11?

Um, as shown above? ☺

No, not quite. In my system that would get opensc-pkcs11.so rather than YKCS11. So what needs to be configured and how, in order for YKCS11 to be picked as the PKCS#11 provider for OpenSSL, GnuTLS, and p11tool?

Also, what does specifying model=YubiKey%20NEO; buy you, besides inability to use other tokens? Or is this the "magic incantation" to use YKCS11? Does it then find the appropriate libraries automatically? Or one still needs to point at it in some config file(s)?

[dwmw2]

This is what ticket #92 is about — on a modern system, p11-kit covers fairly much all of this for you. You install an appropriate module file to either the directory indicated by pkg-config --variable=p11_module_configs p11-kit-1 (which will probably be /usr/share/p11-kit/modules), or to your personal config directory in ~/.config/pkcs11/modules/.

Now your PKCS#11 provider module is visible to well-behaved applications automatically. OpenSC is correctly packaged and does that for itself, of course. Having done the same for YKCS11, I now have both OpenSC and YKCS11 available (as well as the default trust roots and GNOME keyring's PKCS#11 module which are there by default):

 $ p11tool --list-all
warning: no token URL was provided for this operation; the available tokens are:

pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=System%20Trust
pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=Default%20Trust
pkcs11:model=1.0;manufacturer=Gnome%20Keyring;serial=1%3aSSH%3aHOME;token=SSH%20Keys
pkcs11:model=1.0;manufacturer=Gnome%20Keyring;serial=1%3aSECRET%3aMAIN;token=Secret%20Store
pkcs11:model=1.0;manufacturer=Gnome%20Keyring;serial=1%3aUSER%3aDEFAULT;token=Gnome2%20Key%20Storage
pkcs11:model=1.0;manufacturer=Gnome%20Keyring;serial=1%3aXDG%3aDEFAULT;token=User%20Key%20Storage
pkcs11:model=PKCS%2315%20emulated;manufacturer=piv_II;serial=108421384210c3f5;token=PIV_II%20%28PIV%20Card%20Holder%20pin%29
pkcs11:model=YubiKey%20NEO;manufacturer=Yubico;serial=1234;token=YubiKey%20PIV

So the model=Yubikey%20NEO part in my openssl command line was telling it which token to use. If I had only specified id=%01 and not anything about the token, it probably would have found a match in the OpenSC-provided token first. So yes, in a way that is the "inability to use other tokens", as you called it. Deliberately so, just as the id=%01 part was for the inability to use other keys. I was specifying the object I wanted openssl to use for this command.

There's nothing new here; this ought to be exactly how you use OpenSC or any other provider, surely?

[dwmw2]

Getting back on-topic for this ticket, it looks like even for producing the output above, it's iterating over all the objects in the card twice. If I enable the log-calls option in the module file, I get the following output...

pkcs11:model=PKCS%2315%20emulated;manufacturer=piv_II;serial=108421384210c3f5;token=PIV_II%20%28PIV%20Card%20Holder%20pin%29
C_GetSlotList
  IN: tokenPresent = CK_TRUE
  IN: pulCount = 0x7FFCF3DDAA08 = 48
debug: ykcs11.c:194 (C_GetSlotList): In
debug: ykcs11.c:231 (C_GetSlotList): token present is 1
debug: ykcs11.c:232 (C_GetSlotList): number of slot(s) is 1
debug: ykcs11.c:234 (C_GetSlotList): Out
 OUT: pSlotList = (1) [ SL0 ]
C_GetSlotList = CKR_OK
C_GetTokenInfo
  IN: slotID = SL0
debug: ykcs11.c:266 (C_GetTokenInfo): In
debug: ykcs11.c:307 (C_GetTokenInfo): Out
 OUT: pInfo = {
    label: "YubiKey PIV"
    manufacturerID: "Yubico"
    model: "YubiKey NEO"
    serialNumber: "1234"
    flags: 1037 = CKF_RNG | CKF_LOGIN_REQUIRED | CKF_USER_PIN_INITIALIZED | CKF_TOKEN_INITIALIZED
    ulMaxSessionCount: CK_EFFECTIVELY_INFINITE
    ulSessionCount: 18446744073709551615
    ulMaxRwSessionCount: CK_EFFECTIVELY_INFINITE
    ulRwSessionCount: 18446744073709551615
    ulMaxPinLen: 8
    ulMinPinLen: 4
    ulTotalPublicMemory: CK_EFFECTIVELY_INFINITE
    ulFreePublicMemory: CK_EFFECTIVELY_INFINITE
    ulTotalPrivateMemory: CK_EFFECTIVELY_INFINITE
    ulFreePrivateMemory: CK_EFFECTIVELY_INFINITE
    ulFreePrivateMemory: CK_EFFECTIVELY_INFINITE
    hardwareVersion: 109.112
    firmwareVersion: 0.101
    utcTime: 
      }
C_GetTokenInfo = CKR_OK
C_GetSlotInfo
  IN: slotID = SL0
debug: ykcs11.c:243 (C_GetSlotInfo): In
debug: ykcs11.c:257 (C_GetSlotInfo): Out
 OUT: pInfo = {
    slotDescription: "Yubico Yubikey NEO OTP+CCID 00 00"
    manufacturerID: "Yubico"
    flags: 5 = CKF_TOKEN_PRESENT | CKF_HW_SLOT
    hardwareVersion: 1.0
    firmwareVersion: 1.0
      }
C_GetSlotInfo = CKR_OK
C_OpenSession
  IN: slotID = SL0
  IN: flags = 4 = CKF_SERIAL_SESSION
  IN: pApplication = NULL
  IN: Notify = NULL
debug: ykcs11.c:506 (C_OpenSession): In
trying to connect to reader 'Yubico Yubikey NEO OTP+CCID 00 00'.
debug: yubico_token.c:273 (get_objects): Found AUTH cert (9a)
debug: yubico_token.c:282 (get_objects): Found CARD AUTH cert (9e)
debug: yubico_token.c:291 (get_objects): Found SIGNATURE cert (9c)
debug: yubico_token.c:300 (get_objects): Found KMK cert (9d)
debug: yubico_token.c:317 (get_objects): The total number of objects for this token is 21
debug: yubico_token.c:273 (get_objects): Found AUTH cert (9a)
debug: yubico_token.c:282 (get_objects): Found CARD AUTH cert (9e)
debug: yubico_token.c:291 (get_objects): Found SIGNATURE cert (9c)
debug: yubico_token.c:300 (get_objects): Found KMK cert (9d)
debug: yubico_token.c:317 (get_objects): The total number of objects for this token is 21
debug: ykcs11.c:634 (C_OpenSession): Out
 OUT: phSession = 0x7FFCF3DDAA00 = S5355104
C_OpenSession = CKR_OK
C_CloseSession
  IN: hSession = S5355104
debug: ykcs11.c:657 (C_CloseSession): In
debug: ykcs11.c:682 (C_CloseSession): Out
C_CloseSession = CKR_OK
pkcs11:model=YubiKey%20NEO;manufacturer=Yubico;serial=1234;token=YubiKey%20PIV
C_GetSlotList
  IN: tokenPresent = CK_TRUE
  IN: pulCount = 0x7FFCF3DDAA08 = 48
debug: ykcs11.c:194 (C_GetSlotList): In
debug: ykcs11.c:231 (C_GetSlotList): token present is 1
debug: ykcs11.c:232 (C_GetSlotList): number of slot(s) is 1
debug: ykcs11.c:234 (C_GetSlotList): Out
 OUT: pSlotList = (1) [ SL0 ]
C_GetSlotList = CKR_OK
C_GetTokenInfo
  IN: slotID = SL0
debug: ykcs11.c:266 (C_GetTokenInfo): In
debug: ykcs11.c:307 (C_GetTokenInfo): Out
 OUT: pInfo = {
    label: "YubiKey PIV"
    manufacturerID: "Yubico"
    model: "YubiKey NEO"
    serialNumber: "1234"
    flags: 1037 = CKF_RNG | CKF_LOGIN_REQUIRED | CKF_USER_PIN_INITIALIZED | CKF_TOKEN_INITIALIZED
    ulMaxSessionCount: CK_EFFECTIVELY_INFINITE
    ulSessionCount: 18446744073709551615
    ulMaxRwSessionCount: CK_EFFECTIVELY_INFINITE
    ulRwSessionCount: 18446744073709551615
    ulMaxPinLen: 8
    ulMinPinLen: 4
    ulTotalPublicMemory: CK_EFFECTIVELY_INFINITE
    ulFreePublicMemory: CK_EFFECTIVELY_INFINITE
    ulTotalPrivateMemory: CK_EFFECTIVELY_INFINITE
    ulFreePrivateMemory: CK_EFFECTIVELY_INFINITE
    ulFreePrivateMemory: CK_EFFECTIVELY_INFINITE
    hardwareVersion: 109.112
    firmwareVersion: 0.101
    utcTime: 
      }
C_GetTokenInfo = CKR_OK
C_GetSlotInfo
  IN: slotID = SL0
debug: ykcs11.c:243 (C_GetSlotInfo): In
debug: ykcs11.c:257 (C_GetSlotInfo): Out
 OUT: pInfo = {
    slotDescription: "Yubico Yubikey NEO OTP+CCID 00 00"
    manufacturerID: "Yubico"
    flags: 5 = CKF_TOKEN_PRESENT | CKF_HW_SLOT
    hardwareVersion: 1.0
    firmwareVersion: 1.0
      }
C_GetSlotInfo = CKR_OK
C_OpenSession
  IN: slotID = SL0
  IN: flags = 4 = CKF_SERIAL_SESSION
  IN: pApplication = NULL
  IN: Notify = NULL
debug: ykcs11.c:506 (C_OpenSession): In
trying to connect to reader 'Yubico Yubikey NEO OTP+CCID 00 00'.
debug: yubico_token.c:273 (get_objects): Found AUTH cert (9a)
debug: yubico_token.c:282 (get_objects): Found CARD AUTH cert (9e)
debug: yubico_token.c:291 (get_objects): Found SIGNATURE cert (9c)
debug: yubico_token.c:300 (get_objects): Found KMK cert (9d)
debug: yubico_token.c:317 (get_objects): The total number of objects for this token is 21
debug: yubico_token.c:273 (get_objects): Found AUTH cert (9a)
debug: yubico_token.c:282 (get_objects): Found CARD AUTH cert (9e)
debug: yubico_token.c:291 (get_objects): Found SIGNATURE cert (9c)
debug: yubico_token.c:300 (get_objects): Found KMK cert (9d)
debug: yubico_token.c:317 (get_objects): The total number of objects for this token is 21
debug: ykcs11.c:634 (C_OpenSession): Out
 OUT: phSession = 0x7FFCF3DDAA00 = S5355104
C_OpenSession = CKR_OK
C_CloseSession
  IN: hSession = S5355104
debug: ykcs11.c:657 (C_CloseSession): In
debug: ykcs11.c:682 (C_CloseSession): Out
C_CloseSession = CKR_OK

[mouse07410]

And back off-topic. :-)

I've added the appropriate (?) modules to the directories you mentioned (checked with pkg-config to make sure I'm not missing what p11tool` needs), but seem unable to get YKCS11 to work:

$ p11tool --list-all
warning: no token URL was provided for this operation; the available tokens are (with the NEO inserted):

pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=Default%20Trust
pkcs11:model=p11-kit-trust;manufacturer=PKCS%2311%20Kit;serial=1;token=System%20Trust
pkcs11:model=PKCS%2315%20emulated;manufacturer=piv_II;serial=a0fxxxxxx;token=PIV%20Card%20Holder%20pin%20%28PIV_II%29
pkcs11:model=YubiKey%20NEO;manufacturer=Yubico;serial=1234;token=YubiKey%20PIV
pkcs11:model=YubiKey%20NEO;manufacturer=Yubico;serial=1234;token=YubiKey%20PIV
pkcs11:model=YubiKey%20NEO;manufacturer=Yubico;serial=1234;token=YubiKey%20PIV
pkcs11:model=PKCS%2315%20emulated;manufacturer=piv_II;serial=a0fxxxxxx;token=PIV%20Card%20Holder%20pin%20%28PIV_II%29
$  $ openssl pkeyutl -engine pkcs11 -keyform engine -sign -in t256.dat -out t256.dat.sig -inkey 'pkcs11:model=YubiKey%20NEO;id=%02;object-type=private'
engine "pkcs11" set.
Specified object not found
PKCS11_get_private_key returned NULL
cannot load Private Key from engine
140735271227472:error:26096080:engine routines:ENGINE_load_private_key:failed loading private key:eng_pkey.c:124:
unable to load Private Key
Error initializing context
$ openssl pkeyutl -engine pkcs11 -keyform engine -sign -in t256.dat -out t256.dat.sig -inkey 'pkcs11:id=%02;object-type=private'
engine "pkcs11" set.
PKCS#11 token PIN: 
$ 

I added two modules:

$ cat /opt/local/share/p11-kit/modules/pkcs11.module 
module: /Library/OpenSC/lib/opensc-pkcs11.dylib
$ cat /opt/local/share/p11-kit/modules/ykcs11.module 
module: /usr/local/lib/libykcs11.dylib
$ 

What seems to be wrong, in your opinion?

[dwmw2]

What is engine_pkcs11 using as its default provider? It should be using p11-kit-proxy.so, which is a shim that parses the p11-kit configuration then loads the appropriate modules indicated therein, presenting them as its own slots. If it's built to use something else as its default, then you won't get the expected results....

[mouse07410]

What is engine_pkcs11 using as its default provider? It should be using p11-kit-proxy.so, which is a shim that parses the p11-kit configuration then loads the appropriate modules indicated therein, presenting them as its own slots

Hmm... On Mac p11-kit does not seem to build p11-kit-proxy.so... :-(
This is what it places in the library:

  /opt/local/lib/libp11-kit.0.dylib
  /opt/local/lib/libp11-kit.dylib
  /opt/local/lib/p11-kit/p11-kit-remote    (Mach-O 64-bit executable x86_64)
  /opt/local/lib/p11-kit/trust-extract-compat
  /opt/local/lib/pkcs11/p11-kit-trust.so
  /opt/local/lib/pkgconfig/p11-kit-1.pc

So in order to be able to use OpenSC and YKCS11 as you do, I have to have a working p11-kit? libp11 is not enough?

[dwmw2]

Perhaps it only builds if you have libffi?

[mouse07410]

Perhaps it only builds if you have libffi?

$ port installed libffi
The following ports are currently installed:
  libffi @3.2.1_0+universal (active)
$

???

[dwmw2]

@martinpaljak did you ever resolve https://lists.freedesktop.org/archives/p11-glue/2014-December/000550.html

[dwmw2]

@mouse07410 let's move the p11-kit discussion to OpenSC/libp11#117

In the meantime you can continue by using the PKCS11_MODULE_PATH to point engine_pkcs11 to the provider you want to use:

PKCS11_MODULE_PATH=/usr/local/lib/libykcs11.dylib openssl pkeyutil -engine pkcs11 …

It looks like we're reading all the objects on the card, the moment each session is opened. Perhaps we could read them only on demand?

[a-dma]

@dwmw2 thanks for all your work on this stuff!
I (we) really feel that this project needs to be worked on. The PKCS#11 module has been written to address a very specific problem and many things have been left hanging. The reason why this hasn't been done yet, boils down to lack of time/resources.

In principle I am in favor of incorporating most of the changes you have had to implement. I see some of these are self-contained small diffs. If you want to take the credit for them you are welcome to submit a bunch of pull requests and I'll be merging them, otherwise I'll add them myself.

For the "bigger" changes, as I said I agree with them in principle, but I think we should have a slightly different implementation. For example regarding #93 linking against zlib is fine, but I think that that part should be moved from the module into libykpiv.
We have been discussing internally about adding a series of utility functions ykpiv_util_* to libykpiv so that we could address high level operations like load a certificate, load a private key, generate a selfsigned certificate, etc. This would simplify the logic both in yubico-piv-tool and in ykcs11. However, as you can imagine it requires some time and work.
Other major fixes like the performance issue, or the support of multiple sessions (or lack thereof) are on a similar level: nice to have but no time to work on this now.

So, to make a long story short, what I'm trying to say is that this suggestions are valuable and will be addressed, alas not now. If in the meanwhile you are willing to contribute some time and code to the project pull requests are always welcome :)

[dwmw2]

May I ask without offending, what was the "very specific problem" that YKCS11 addresses?

I've been using the OpenSC support for a while, and using the Yubikey as the poster child for PKCS#11 support in places such as the OpenConnect PKCS#11 documentation and the Fedora packaging guidelines which say that any package accepting keys/certs on the command line or in a config file SHOULD also accept a PKCS#11 URI such as pkcs11:manufacturer=piv_II;id=%01 and that users should file bugs against any application which doesn't.

In that time, I haven't had cause even to file a bug against OpenSC, let alone to want a whole new PKCS#11 provider :)

Regarding the specific fixes... I think I only filed one patch that wasn't an incompete proof-of-concept, to fix #91. I'm not too bothered about attribution, and it's almost a one-liner so it's probably as easy for you to fix that yourself, as it is for me to set about submitting it in a PR and having you merge it.

[a-dma]

Basically generating and importing keys, while also exposing the possibility of adding vendor specific flags to enable things like touch to sign on the YubiKey. And signing data.

As I said before I think most of the suggestions make sense, being able to use ykcs11 as an OpenSSL engine is a nice addition.

All right then, I'll try to incorporate some of your fixes and when I have more time to dedicate to this project improve it general. I just didn't want you (or anybody) to have the impression that this is considered dead.

[lbschenkel]

FYI: p11-kit-proxy.so was not being built on OS X due to a Makefile bug that has been reported at https://bugs.freedesktop.org/show_bug.cgi?id=98022 and fixed today. But you don't really need the proxy because it's just a symlink to libp11-kit.dylib (the same file is both the library and a PKCS#11 module), so you can either create the symlink yourself or use the path to the dylib directly in any configuration files.

[dwmw2]

Basically generating and importing keys, while also exposing the possibility of adding vendor specific flags to enable things like touch to sign on the YubiKey. And signing data.

You may well find that OpenSC would be interested in accepting that the Yubikey is a "PIV-like" device, not actually PIV, and supporting all of those directly. Thus excusing you from reimplementing all the external support gubbins, just for a few extra features :)

[mouse07410]

I confirm the timing difference. YKCS11 takes roughly 3.5 times longer than OpenSC for the same (signing) operation (3.0 seconds vs. 0.85 seconds).

Update

OpenSC would be interested in accepting that the Yubikey is a "PIV-like" device, not actually PIV, and supporting all of those directly

Actually, I think we both successfully use PIV-II driver of OpenSC to use Yubikey (and this driver already has some accommodations for some peculiarities of Yubico implementation of NIST SP 800-73-3), so I'm not sure what you mean by "PIV-like" as opposed to "PIV". And I think that signing data works fine now using Yubikey with OpenSC, unless I misunderstand what "sign data" means.

[qpernil]

yubico-piv-tool 2.0.0 contains a major upgrade to ykcs11. Please try with that version.
I believe the issue has been adressed, closing the issue due to age. Please re-open if the problem persists.