Inserting CAC causes Firefox to completely freeze #1278
Comments
|
Your description on Firefox seems very specific, did it work with some previous version? @Jakuje ? |
|
Can you try with OpenSC master? There were several fixes since 0.17.0 release so your issue might be already fixed. |
|
I also tried it with 0.17.0 and master and both hang. I have Firefox Developer Edition 60.0b2, on a Mac. |
|
So lets have a look into the logs to see where does it hang and why: https://github.com/OpenSC/OpenSC/wiki/Using-OpenSC#debug-level If it will not give anything useful, what does GDB says, when you attach it to the process that loaded the OpenSC PKCS#11 module. |
|
I noticed this issue with different card, that does not have random number generator. It seems like something new in Firefox/NSS started requesting Can you verify that the following code will hang for you (you will probably need to adjust some paths to pkcs11.h): If I see right, CAC and Coolkey get_challenge routines do not expect it can fail, which needs to be fixed and if we are not able to get any data from the card, we should return appropriate error to PKCS#11 interface. |
I confirm that Firefox 60+ hangs when the token is inserted, and any request is made to it. With the current master on MacOS 10.13.3, your code does not hang (nor does Here's the SPY: On YubiKey your code prints (I've added more verbose output): and SPY via |
|
If this happens also for you with PIV and Firefox, debug log from opensc, where does it hang for you would be helpful (just the end to see where it hangs/cycles should be enough). I still don't have Firefox 60 so I can not verify at the moment. |
|
Sounds like Firefox has problems with generating random numbers. Some
entropy problem in software in OS? Or problem in handling PKCS#11error.
What happens if the driver does not say it can generate random?
(Sorry dont have access to source for next two weeks.)
…On Mar 16, 2018 2:34 AM, "Jakub Jelen" ***@***.***> wrote:
If this happens also for you with PIV and Firefox, debug log from opensc,
where does it hang for you would be helpful (just the end to see where it
hangs/cycles should be enough).
I still don't have Firefox 60 so I can not verify at the moment.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1278 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AA00MUEgt516Jv0bCX8YVYLiKzImgF5jks5te4c3gaJpZM4SVfvv>
.
|
|
Firefox ESR 52.6.0 (64-bit) appears to work with CAC. Firefox 60.0 (6018.3.12) hangs even on the attempt to go to Security Devices: And this: |
|
Actually, with Firefox-60 it's bad. It hung as soon as I started typing the URL, even before the CAC was inserted. OPENSC_DEBUG when the system did even have a chance to talk to the card (card was not inserted yet): OPENSC_DEBUG (using PIV driver) when I got to the CAC-protected web site, and inserted CAC to authenticate: Same as above - but using CAC driver: If I start Firefox when the CAC is already inserted - everything works fine: |
|
Quick look from phone of PIV log shows get challenge for 32 bytes failing.
I recall Mouse saying some PIV card would only do one challenge.
Is Firefox now requeating random data when it did not before?
As I said before can you try modifying driver to not saying it can generte
random numbers.
We may have to revisit if a get Witness should be used for random nom
generation.
…On Mar 16, 2018 12:09 PM, "Mouse" ***@***.***> wrote:
Actually, with Firefox-60 it's bad. It hung as soon as I started typing
the URL, even before the CAC was inserted.
OPENSC_DEBUG when the system did even have a chance to talk to the card
(card was not inserted yet):
firefox-opensc.log.txt
<https://github.com/OpenSC/OpenSC/files/1820411/firefox-opensc.log.txt>
OPENSC_DEBUG (using PIV driver) when I got to the CAC-protected web site,
and inserted CAC to authenticate:
firefox-opensc.log2.txt
<https://github.com/OpenSC/OpenSC/files/1820462/firefox-opensc.log2.txt>
Same as above - but using CAC driver:
firefox-opensc.log3.txt
<https://github.com/OpenSC/OpenSC/files/1820483/firefox-opensc.log3.txt>
If I start Firefox when the CAC is already inserted - everything works
fine:
firefox-opensc.log4.txt
<https://github.com/OpenSC/OpenSC/files/1820504/firefox-opensc.log4.txt>
*The log is for the car driver - but it works fine with PIV driver too, in
this case.*
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1278 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AA00MXaBMg4ZCQi5pSy2Ig7vuPSKCcvYks5tfA3zgaJpZM4SVfvv>
.
|
I think I saw different PIV tokens exhibited different behavior here. :-(
I don't know. It does look like Firefox now does something different - because the older versions work fine with the same version of OpenSC and the same tokens.
I would really hate this, unless it is configurable via, e.g., |
|
If firefox is requesting random number and we can not provide it, maybe we
are returning the wrong CKR_ code.
We could look at for cards in question:
Dont advertise the card can return random numbers to start with.
Or return software generated random number
Or return some other CKR like not supported
What do others think?
Is this a MacOS problem only?
Can someone look at the Firefox source to see what it does with a bad CKR
in this case?
I cant do anything for the next week.
…On Mar 16, 2018 2:06 PM, "Mouse" ***@***.***> wrote:
I recall Mouse saying some PIV card would only do one challenge
I think I saw different PIV tokens exhibited different behavior here. :-(
Is Firefox now requeating random data when it did not before?
I don't know. It does look like Firefox now does something different -
because the older versions work fine with the same version of OpenSC and
the same tokens.
As I said before can you try modifying driver to not saying it can generate
random numbers.
I would *really* hate this, unless it is configurable via, e.g.,
opens.conf, and hopefully, on a per-token-type basis. So I *would* be
able to get random numbers fro those PIV tokens that behave correctly, but
I could make the driver not expose this functionality for those tokens
(like CAC) that are known to fail. *Actually, older CAC didn't seem to
have a problem generating random data, as I recall!*
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1278 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AA00McyULhlFb6B2UrmyeO2i3iTCR36Cks5tfClJgaJpZM4SVfvv>
.
|
This sounds right.
I'd say: if you recognized that this card is known to fail A question here: in the SPY log I did not see where the amount of random data required was passed to the card. Could you point me at the place where the card is informed how many random bytes it's supposed to produce?
If I understood the original reporter correctly - this problem exists on Linux (at least) as well.
Somebody should. Sorry, not me. |
|
@mouse07410 could you try to apply #1249? The PR allows fixed size responses on the card driver level (as returned by the PIV card) while still returning the correct amount of random data on the application level (i.e. to Firefox). |
|
That would make sense, that the first request would work, but not the second, since for random data, the GET_CHALLENGE instruction is "misused". But from the logs, I see that the get_challenge functions are called just once throughout all the logs. I still can not isolate the reproducer no reproduce it with any of my cards. I am starting with the card inserted in the reader, which might be the problem. But using C_WaitForSlotEvent and inserting the card later did not help me with reproducing the issue. It looks like some issue of the Firefox itself, because I see a |
|
Ok, on the second try, I don't see the defunct process, but the hang persists. Using the #1249 does not change anything and process still hangs. The backtraces of all the processes point someplace deep in the wait routines of Firefox. I used just standard PIV test cards to reproduce the issue on Linux. I can not reproduce it with default Firefox 59.0 in Fedora 27. But colleagues managed to reproduce it even with Firefox 52 [1] so this makes me think that the issue is somewhere deep in NSS (which I believe they got that one updated). I will have to dig into that deeper. |
Yes. I applied #1249 and tried using CAC with Firefox 60 (Dev Edition). Same thing: even when I start FF with CAC already inserted (which used to work more often than not), whenever I to go Preferences -> Security -> Certificates (or -> Devices) FF hangs up. |
|
That really sounds like an issue of Firefox. Did anybody already tried to fill a bug in their bugzilla? I would be glad to do that but I was not able to collect any reasonable information from Firefox yet. I can reproduce it with Nightly and coolkey, but not with softhsm (it loads the certificates properly). |
|
The reproducer:
Playing with mozregresion:
First I tried to work with an existing profile, but this makes the testing unreliable. The following with always-clean profile worked: mozregression --bad 2018-02-01 --good 2018-01-01 The led me to the date https://hg.mozilla.org/integration/mozilla-inbound/rev/348b36156a84 I filled the following bug: |
|
@Jakuje how do I get the latest NSS? Is there an analog to Github, and what are the commands to get it? |
|
You can get latest NSS with Firefox Nightly: http://nightly.mozilla.org/ |
I want the latest source NSS to build myself.
I'm afraid your reproducer still reproduces the problem. Firefox/NSS is not fixed yet, AFAICT. |
|
A simple test woukd be to build OpenSC to have card-piv.c and
card-openpgp.c or other OpenSc routjne to not advertise it can generate
random. NSS should then do its own.
This would prove if it has to do with random.
Then look at return codes for C_GenerateRandom.
…On Tue, Mar 27, 2018, 10:58 AM Mouse ***@***.***> wrote:
...It bundles the latest NSS
I want the latest *source* NSS to build myself.
If you will try my reproducer with the latest release...
I'm afraid your reproducer still reproduces the problem. Firefox/NSS is
not fixed yet, AFAICT.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1278 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AA00MW5-uDGc8Xlt3fdIEAc6WAh6-d5kks5tin24gaJpZM4SVfvv>
.
|
|
@mouse07410 Instructions how to build NSS itself are here: http://www.mozilla.org/projects/security/pki/nss/ But then you would have to modify also your firefox to use this version, instead of the bundled one. If I see right, the new NSS was not yet merged in the lastest Firefox, so it is not yet in nightly: https://bugzilla.mozilla.org/show_bug.cgi?id=1448404 @dengert No, the freeze was not related to OpenSC at all. I reproduced it also with different pkcs11 modules. It was only the first assumption, because it froze after the last (and only) call to |
|
OK, C_GenerateRandom was a read herring. There is a Github version of the Mozilla source: which leads to https://github.com/mozilla/gecko-dev I see updates from yesterday and in security/nss There are updates from 2 days ago and references to NSS 3_36-Beta3. So this might be an easy way to build it or at least see the source code. |
|
I built FireFox from git on Ubuntu-16.4 (VirtualBox client, Windows 10 host) yesterday. Starting firefox (./mach run) then using NIST PIV demo card:
Based on your comment about softhsm works, but opensc with coolkey does not, could this be a timing or tread/mutex problem? The softhsm is all in software so may initialize quickley. The OpenSC does OpenSC may be missing some locks in the code or we have a deadlock. As far as I can tell, OpenSC does not log uses of the mutex when the application supplies the routines. We could add some debugging. |
|
@dengert I did not try the card-insertion use case with softhsm, mostly out of my laziness. Feel free to reproduce the issue with softhsm. I don't think this is issue of OpenSC. |
|
Running Linux openSUSE Leap 42.3 . Update to NSS 3.36 affected Thunderbird and Chrome too. Symptoms are the same or similar as in Firefox - when inserting card during application run and card is accessed, app freezes. Waiting for NSS 3.36.1 as described in https://bugzilla.mozilla.org/show_bug.cgi?id=1447628 |
|
The latest NSS is merged in Firefox and this issue can be closed. |
|
great, thanks for the update! |
Problem Description
Using firefox nightly (60.0a1), or any version that is based off 60, seems to be impacted. If you have a running process of firefox and you insert a CAC and attempt to query it in any way (via a site that requests it, or via viewing your certificates) it will lock up the firefox process(s).
If you kill the processes and insert the CAC, then launch firefox, it seems to work nominally. Alternatively, if you put the CAC in prior to ever lunching firefox on a fresh install it works nominally.
If you have a timeout on your pin where firefox prompts you (in your main session or a private session), it entirely locks up the firefox process(s) until you enter your pin.
Proposed Resolution
reporting for awareness. I've talked to some other people who are seeing this and living with it without reporting. I realize that using nightly will come with edges that might cut, reporting this now so that it can get eyes before firefox ships stably with whatever is causing this issue.
Steps to reproduce
The other steps are enumerated above in the problem description.
Logs
What logs (assuming more than opensc) would be advisable to grab?
The text was updated successfully, but these errors were encountered: