ca behaviour with -spkac and stdout changed/broke text interface in 1.1.0i

[twegener-embertec]

Since around 1.1.0i, the following openssl ca command (with -spkac option and no -out option, i.e. use stdout) now generates binary certificate details rather than text (which breaks applications that parse that output):

OPENSSL_ENABLE_MD5_VERIFY=1 openssl ca -config test.openssl.cnf -passin stdin -batch -spkac input_file -startdate 190121130654Z

Check that the SPKAC request matches the signature
Signature ok
Certificate Details:
...
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Cert Type: 
                SSL Client
...
Certificate is to be certified until Jan 19 01:58:28 2029 GMT (3650 days)

Write out database with 1 new entries
...(elided binary goop)...

This was observed in openssl-1.1.0i from Fedora 28:
openssl-1.1.0i-1.fc28.x86_64

In earlier versions it would output the PEM format Base64 encoded certificate data in a block surrounded with:

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

This interface change/breakage is quite painful.

[twegener-embertec]

I suspect that this may have been introduced by the following commit (which was released in 1.1.0i):
29a7148

Associated pull request for that commit:
Fix openssl ca, to correctly make output file binary when using -spkac #6050

diff --git a/apps/ca.c b/apps/ca.c
index d474a2b69a..eb093d0e0b 100644
--- a/apps/ca.c
+++ b/apps/ca.c
@@ -725,8 +725,12 @@ end_of_options:
 
     /*****************************************************************/
     if (req || gencrl) {
-        /* FIXME: Is it really always text? */
-        Sout = bio_open_default(outfile, 'w', FORMAT_TEXT);
+        if (spkac_file != NULL) {
+            output_der = 1;
+            batch = 1;
+        }
+        Sout = bio_open_default(outfile, 'w',
+                                output_der ? FORMAT_ASN1 : FORMAT_TEXT);
         if (Sout == NULL)
             goto end;
     }
@@ -872,10 +876,6 @@ end_of_options:
                     BIO_printf(bio_err, "Memory allocation failure\n");
                     goto end;
                 }
-                if (outfile) {
-                    output_der = 1;
-                    batch = 1;
-                }
             }
         }
         if (ss_cert_file != NULL) {

[twegener-embertec]

Breaking the interface seems like kind of a big deal. Can someone please comment on this?

[mattcaswell]

Looks like a possible bug to me. This is what is in the documentation:

"When processing SPKAC format, the output is DER if the B<-out>
flag is used, but PEM format if sending to stdout or the B<-outdir>
flag is used."

Perhaps @levitte can comment.

[levitte]

Does the following diff fix the issue for you?

diff --git a/apps/ca.c b/apps/ca.c
index c69a2b5cdd..5b33fa6aa1 100644
--- a/apps/ca.c
+++ b/apps/ca.c
@@ -725,7 +725,7 @@ end_of_options:
 
     /*****************************************************************/
     if (req || gencrl) {
-        if (spkac_file != NULL) {
+        if (spkac_file != NULL && outfile != NULL) {
             output_der = 1;
             batch = 1;
         }

[levitte]

There is a PR for this now, #8368

[twegener-embertec]

@levitte Thanks, that diff fixes my issue.
Will this be backported to the 1.1.0 series? (That is used by Ubuntu 18.04 LTS, so it is currently broken there.)

[mattcaswell]

Will this be backported to the 1.1.0 series?

No. The 1.1.0 series is in security-fix only mode. Therefore this bug fix does not qualify.

[twegener-embertec]

Ah, okay, no worries.
It looks like openssl 1.1.1 will be coming to Ubuntu 18.04 LTS anyway:
https://bugs.launchpad.net/ubuntu/bionic/+source/openssl/+bug/1797386

[twegener-embertec]

Correction: Ubuntu 18.04 LTS has 1.1.0g-2ubuntu4.3, and so does not have this regression.
However, current 1.1.1 does have this regression, so when Ubuntu 18.04 brings in OpenSSL 1.1.1 via an SRU [1] it currently will introduce this regression. :-(

[1] https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1797386/

Any idea when the fix for this issue will make it into a 1.1.1 series update?

[mattcaswell]

Any idea when the fix for this issue will make it into a 1.1.1 series update?

It will be in 1.1.1c which we have no date for yet. However we typically do a release every 3-4 months unless a pressing security issue requires us to do one earlier. 1.1.1b came out at the end of February.

[xnox]

Will this be cherrypicked into https://github.com/openssl/openssl/commits/OpenSSL_1_1_1-stable/apps/ca.c ? i see other fixes on the 1_1_1 branch since 1.1.1b but not this one.

[mattcaswell]

Ah...#8368 was supposed to have been cherry-picked to 1.1.1 already, but it looks like it wasn't. I'll ping that PR.

[twegener-embertec]

Just noting FTR that this regression fix did not make it into 1.1.1c (presumably due to the oversight mentioned above). So presumably this will be in the (yet-to-be released) 1.1.1d?