X509Cert.set_serial breaks for Certificate SerialNumbers with MSB=1 #7

Open
rhetzler opened this Issue Jun 21, 2011 · 4 comments

Comments

Projects
None yet
3 participants
@rhetzler

in X509Cert.java line 303, set_serial makes a call to org.bouncycastle.x509.X509V3CertificateGenerator.setSerialNumber(BigInteger), which throws an exception if given a negative number.

Sample stacktrace:
Java::JavaLang::IllegalArgumentException (serial number must be a positive integer):
org.bouncycastle.x509.X509V3CertificateGenerator.setSerialNumber(Unknown Source)
org.jruby.ext.openssl.X509Cert.set_serial(X509Cert.java:303)
org.jruby.ext.openssl.X509Cert.initialize(X509Cert.java:166)
org.jruby.ext.openssl.X509Cert$i$0$1$initialize.call(X509Cert$i$0$1$initialize.gen:65535)
...

( refer to http://bouncycastle.sourcearchive.com/documentation/1.43/classorg_1_1bouncycastle_1_1x509_1_1X509V3CertificateGenerator_23c047ee3c96cb63a0f42cace5333b7b.html )

During initialization, such a certificate with a serial number having MSB=1 can be loaded via java java.security.cert.X509Certificate (line 158), this results in cert.getSerialNumber() (line 166) to yield a negative BigInteger due to it's internal 2's complement representation. Thus, a valid certificate can result in an error.

Typically, such a situation might arise from a certificate authority which generates serial numbers via UID (random bit generation) instead of an incremental approach. This results in a ~50% chance that the MSB=1, and causing this failure.
Eg, Serial number "d4 06 2d 6b c9 84 48 86 3d ea c2 cd c1 37 86 8a" (any serial where the first hex digit is 8-f)

The fix is simple, just make sure that the bouncycastle code does not receive a negative SerialNumber. This can be done by forcing BigInteger to interpret the serialNumber bits as positive number by setting the signum in the constructor, as follows:

X509Cert.java:303
-generator.setSerialNumber(bi);
+generator.setSerialNumber(new BigInteger(1, bi.toByteArray()));

note: This is difficult to reproduce using certificates generated via openssl CA, since it seems to pad the serial with extra 0s for MSB, neatly avoiding the problem.

Thank you

@sclasen

This comment has been minimized.

Show comment Hide comment
@sclasen

sclasen Jun 24, 2013

Hitting this behavior ATM, was there any decision on what to do here?

sclasen commented Jun 24, 2013

Hitting this behavior ATM, was there any decision on what to do here?

@sclasen

This comment has been minimized.

Show comment Hide comment
@sclasen

sclasen Jun 24, 2013

Well that was easy, works for me

sclasen commented Jun 24, 2013

Well that was easy, works for me

@sclasen sclasen referenced this issue in jruby/jruby Jun 24, 2013

Merged

fix #823 #824

@delwaterman

This comment has been minimized.

Show comment Hide comment
@delwaterman

delwaterman Apr 24, 2016

I am getting this error for jruby 9.0.5.0 (2.2.3) 2016-01-26 7bee00d Java HotSpot(TM) 64-Bit Server VM 25.74-b02 on 1.8.0_74-b02 +jit [darwin-x86_64]

I am getting this error for jruby 9.0.5.0 (2.2.3) 2016-01-26 7bee00d Java HotSpot(TM) 64-Bit Server VM 25.74-b02 on 1.8.0_74-b02 +jit [darwin-x86_64]

@delwaterman delwaterman referenced this issue in alloy/lowdown Apr 24, 2016

Open

Apple Certificate #16

@delwaterman

This comment has been minimized.

Show comment Hide comment
@delwaterman

delwaterman May 6, 2016

Actually ignore my previous comment I misunderstood something with regards to a test.

Actually ignore my previous comment I misunderstood something with regards to a test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment