freezing after 2x call to C_Initialize(NULL) with OpenVPN->pkcs11-helper->OpenSC.C_Initialize

[Wesseldr]

Using a smartcard to authenticate with OpenVPN freezes completely after entering the pin. The connection is authenticated and is getting setup but during the unwind of the card auth it freezes completly. It looks like a second time a call is made to the __pkcs11h_forkFixup breaks it down. This call is called inside the pkcs11-helper lib.
I traced down the death-trail as follow when the C_Initiliaze is called for a second time.

__pkcs11h_forkFixup(...)
-calling: current->f->C_Initialize (NULL);
-calling: C_Finalize(NULL_PTR);
-card_removed(sc_ctx_get_reader(context, i));
-sc_disconnect_card(card->card);
-card->reader->ops->disconnect(card->reader); // <- Sigfaults! freezes up all

Enviroment

OsX 10.8.3
pkcs11-helper-1.10
libusb 1.0.9
CCID 1.4.10
OpenVPN 2.3.1
Hardware Token: ePass2003 and SRC301
OpenSC 0.13.0

Steps to reproduce

Compile the components with the default settings.
Compile OpenVPN with:
./configure --enable-pkcs11

OpenVPN client.config

-- start openvpn.conf --

remote foo.bar 1194 udp
tls-client
tls-auth ta.key 1
pull
pkcs11-providers /Library/OpenSC/lib/opensc-pkcs11.so
pkcs11-id
dev tun
persist-tun
persist-key
comp-lzo adaptive
nobind
ca ca.crt
verb 900

---- end ---

openvpn --config client.config

Starting the openvpn client results in:
Wed May 15 17:55:01 2013 us=954513 TUN/TAP device /dev/tun0 opened
Wed May 15 17:55:01 2013 us=954530 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Wed May 15 17:55:01 2013 us=954560 /sbin/ifconfig tun0 delete
Wed May 15 17:55:01 2013 us=955103 PKCS#11: __pkcs11h_forkFixup entry pid=26671, activate_slotevent=1
Wed May 15 17:55:02 2013 us=927488 PKCS#11: __pkcs11h_forkFixup return
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
Wed May 15 17:55:02 2013 us=929588 NOTE: Tried to delete pre-existing tun/tap instance -- No Problem if failure
Wed May 15 17:55:02 2013 us=929670 /sbin/ifconfig tun0 10.10.0.6 10.10.0.5 mtu 1500 netmask 255.255.255.255 up
Wed May 15 17:55:02 2013 us=930107 PKCS#11: __pkcs11h_forkFixup entry pid=26673, activate_slotevent=1
Frozen

==== Dirty Fixes that Worked, but are probably not the solution ===
Dirty Fix (to dirty dont use):
Commenting out the line:
/pkcs11-helper-1.10//lib/pkcs11h-core.c:__pkcs11h_forkFixup line: 1309 calls the current->f->C_Initialize (NULL); Just removing it makes OpenVPN work properly and use smartcards for authentication.

Other Dirty Fix: (might be more viable but hopefully some one has the best solution)
In the function C_Finalize file:./src/pkcs11/pkcs11-global.c
there is a loop in the function C_Finalize:
for (i=0; i < (int)sc_ctx_get_reader_count(context); i++)
card_removed(sc_ctx_get_reader(context, i));
Just make a check before this and change it into:

if (!sc_pkcs11_conf.plug_and_play) {
for (i=0; i < (int)sc_ctx_get_reader_count(context); i++)
card_removed(sc_ctx_get_reader(context, i));
}
This solves the sigfault but I am lacking the knowledge if this is the best solution for this bug. The initialize reader was protected the same, so perhaps this is a solution.

Hope my ticket will help someone to improve it to a proper solution. and make software like OpenVPN work again.

Many thanks! Wessel

[alonbl]

This was already reported.

OpenSC expect some initialization sequence which is not required by the
PKCS#11 spec.

I suggest running OpenVPN as unprivileged and using the management
interface to enter password, this way the initialization of PKCS#11 is done
after the fork().

[1] http://openvpn.net/index.php/open-source/documentation/howto.html#pkcs11
[2] https://sites.google.com/site/alonbarlev/openvpn-pkcs11

On Wed, May 15, 2013 at 7:09 PM, Wessel dR notifications@github.com wrote:

Using a smartcard to authenticate with OpenVPN freezes completely after
entering the pin. The connection is authenticated and is getting setup but
during the unwind of the card auth it freezes completly. It looks like a
second time a call is made to the __pkcs11h_forkFixup breaks it down. This
call is called inside the pkcs11-helper lib.
I traced down the death-trail as follow when the C_Initiliaze is called
for a second time.

__pkcs11h_forkFixup(...)
-calling: current->f->C_Initialize (NULL);
-calling: C_Finalize(NULL_PTR);
-card_removed(sc_ctx_get_reader(context, i));
-sc_disconnect_card(card->card);
-card->reader->ops->disconnect(card->reader); // <- Sigfaults! freezes up
all
Enviroment

OsX 10.8.3
pkcs11-helper-1.10
libusb 1.0.9
CCID 1.4.10
OpenVPN 2.3.1
Hardware Token: ePass2003 and SRC301
OpenSC 0.13.0

Steps to reproduce
Compile the components with the default settings.
Compile OpenVPN with:
./configure --enable-pkcs11

OpenVPN config:
#-- start openvpn.conf --
remote foo.bar 1194 udp
tls-client
tls-auth ta.key 1
pull
pkcs11-providers /Library/OpenSC/lib/opensc-pkcs11.so
pkcs11-id
dev tun
persist-tun
persist-key
comp-lzo adaptive
nobind
ca ca.crt
verb 900
#---- end ---

Starting the openvpn client results in:
Wed May 15 17:55:01 2013 us=954513 TUN/TAP device /dev/tun0 opened
Wed May 15 17:55:01 2013 us=954530 do_ifconfig, tt->ipv6=0,
tt->did_ifconfig_ipv6_setup=0
Wed May 15 17:55:01 2013 us=954560 /sbin/ifconfig tun0 delete
Wed May 15 17:55:01 2013 us=955103 PKCS#11: __pkcs11h_forkFixup entry
pid=26671, activate_slotevent=1
Wed May 15 17:55:02 2013 us=927488 PKCS#11: __pkcs11h_forkFixup return
ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
Wed May 15 17:55:02 2013 us=929588 NOTE: Tried to delete pre-existing
tun/tap instance -- No Problem if failure
Wed May 15 17:55:02 2013 us=929670 /sbin/ifconfig tun0 10.10.0.6
10.10.0.5 mtu 1500 netmask 255.255.255.255 up
Wed May 15 17:55:02 2013 us=930107 PKCS#11: __pkcs11h_forkFixup entry
pid=26673, activate_slotevent=1
Frozen

==== Dirty Fixes that Worked, but are probably not the solution ===
Dirty Fix I:
Commenting out the line:
/pkcs11-helper-1.10//lib/pkcs11h-core.c:__pkcs11h_forkFixup line: 1309
calls the current->f->C_Initialize (NULL); Just removing it makes OpenVPN
work properly and use smartcards for authentication.

Dirty Fix II: ( might be more viable but hopefully some one has the best
solution)
In the function C_Finalize file:./src/pkcs11/pkcs11-global.c
there is a loop in the function C_Finalize:
for (i=0; i < (int)sc_ctx_get_reader_count(context); i++)
card_removed(sc_ctx_get_reader(context, i));
Just make a check before this and change it into:

if (!sc_pkcs11_conf.plug_and_play) {
for (i=0; i < (int)sc_ctx_get_reader_count(context); i++)
card_removed(sc_ctx_get_reader(context, i));
}
This solves the sigfault but I am lacking the knowledge if this is the
best solution for this bug. The initialize reader was protected the same,
so perhaps this is a solution.

Hope my ticket will help someone to improve it to a proper solution. and
make software like OpenVPN work again.

Many thanks! Wessel

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/159
.

[Wesseldr]

Thank you Alon for the comment and the links.

I'm slightly confused, The initialisation, request of the pincode, unlocking of the card and authorisation all works as far as I can see, its the cleaning up with the __pkcs11h_forkFixup wich blows up if it is called twice.
However if I understand correctly that happens because of the partly incomplete initialisation ?.

Viscosity and Tunnelblick are using the management interface to enter the pin they freeze up as well unfortunately. But I dont't think they run unprivileged that might do the trick after the fork.
I'll have a look at your links.

For now I'll just use the patched OpenSC version with my "Other Dirty Fix" which blocks card_removed in case its a plug&play. Both viscosity, tunnelblick and openvpn from the console worked well with it when I tested it.
(I'm a ware it's a dirty symptom killer and not a solution, but works for our requirement having a SC VPN running between our iMacs and the server)

Is it possible that you or someone could please give some hints or directions wich I could have a look at and see if I can fix it in a proper way ? It looks like some NULL pointer lands on a wrong place. That's why i tracked the whole function chain down in pursuit to find a pieve of code that did an unchecked pointer usages.

Thanks in advance,
Wessel

[alonbl]

On Wed, May 15, 2013 at 9:32 PM, Wessel dR notifications@github.com wrote:

Thank you Alon for the comment and the links.

I'm slightly confused, The initialisation, request of the pincode, unlocking of the card and authorisation all works as far as I can see, its the cleaning up with the __pkcs11h_forkFixup wich blows up if it is called twice.
However if I understand correctly that happens because of the partly incomplete initialisation ?.

I don't know, I did not debug this. I know it used to work few years
ago... and then broke.
I am almost sure the problem is at pkcs11-helper side, as based on
PKCS#11 spec, when doing fork() you need to call finalize and then
initialize for each fork.
The fact that OpenSC leaks something needs to be addressed.

Viscosity and Tunnelblick are using the management interface to enter the pin they freeze up as well unfortunately. But I dont't think they run unprivileged that might do the trick after the fork.
I'll have a look at your links.

OK, so the problem is when openvpn forks utilities.

For now I'll just use the patched OpenSC version with my "Other Dirty Fix" which blocks card_removed in case its a plug&play. Both viscosity, tunnelblick and openvpn from the console worked well with it when I tested it.
(I'm a ware it's a dirty symptom killer and not a solution, but works for our requirement having a SC VPN running between our iMacs and the server)

Is it possible that you or someone could please give some hints or directions wich I could have a look at and see if I can fix it in a proper way ? It looks like some NULL pointer lands on a wrong place. That's why i tracked the whole function chain down in pursuit to find a pieve of code that did an unchecked pointer usages.

I think that OpenSC needs to open handles as FD_CLOEXEC, or may be the
problem is in pcsc-lite, which should behave the same.

This way the following valid and required by PKCS#11 sequence will not
leak/hang:

if (fork() == 0) {
C_Finalize();
C_Initialize();
execvp();
}

Regards,
Alon

[alonbl]

On Wed, May 15, 2013 at 9:56 PM, Alon Bar-Lev alon.barlev@gmail.com wrote:

I don't know, I did not debug this. I know it used to work few years
ago... and then broke.

Maybe because I use openct and not pcsc-lite... so it is possible the
problem is in pcsc-lite.

[alonbl]

Replying via email looks so bad!!!!

[Wesseldr]

Your eMails i received looked good though, perhaps you can you edit your messages here and past your email into it :-D

Could be about the problem on an even lower level. The call:
card->reader->ops->disconnect(card->reader);
Wich is the call that actually craches I didnt find back.
My next investigation would be to have a look at the structure offered card->reader and see what the difference is between the 1st cal that comes by and the next call that comes by.
If i am able to see a distinction we could use it for an exception.

I found an old ticket that could have similarities:
https://www.opensc-project.org/opensc/ticket/383
It suggested that the NULL setting into the slot variable is giving the problems.
'''As you can see the slot variable is set to NULL at line 254, and the null pointer is dereferenced. Should I patch this by adding a check for that?''

I'll have a look and a try with your suggestion in the helper:
if (fork() == 0) {
C_Finalize();
C_Initialize();
execvp();
}
See if that avoids the blowup.

Wessel

[MeGaPk]

In Linux this bug have too.

[frankmorgner]

I hope the problem is solved with @alonbl's suggestion. If not, please reopen the ticket.