Problems issuing Chromebook certificates through SCEP provider #908
Comments
JMyklebust
added
bug
needs triage
|
Hi @JMyklebust, Thanks for your report. That should help in resolving this issue. I'll have a look at it soon. I currently don't have a Chromebook, but I should be able to run it in a VM. Is it possible for you to capture the actual SCEP requests and responses off the wire (e.g. using Wireshark)? Or does Google Workspace require HTTPS, complicating that? A copy of your full CA config could also help, if you're still in the testing phase and no confidential data is in there. Otherwise I'll try to stand up one and try it with Workspace myself. |
|
Thank you for looking into this. The CA config I can share though: {
"root": "D:/Scripts/PKITest/.step/certs/root_ca.crt",
"federatedRoots": null,
"crt": "D:/Scripts/PKITest/.step/certs/intermediate_ca.crt",
"key": "D:/Scripts/PKITest/.step/secrets/intermediate_ca_key",
"address": ":9000",
"insecureAddress": ":9001",
"dnsNames": [
"<REMOVED>"
],
"logger": {
"format": "text"
},
"db": {
"type": "badgerv2",
"dataSource": "D:/Scripts/PKITest/.step/db",
"badgerFileLoadingMode": "FileIO"
},
"authority": {
"provisioners": [
{
"type": "SCEP",
"name": "scepca",
"forceCN": true,
"challenge": "secret1234",
"EncryptionAlgorithmIdentifier": 2,
"encryptionAlgorithm": 3,
"claims": {
"minTLSCertDuration": "24h",
"maxTLSCertDuration": "9000h",
"defaultTLSCertDuration": "4320h",
"disableRenewal": false
},
"options": {
"x509": {
"templateFile": "D:/Scripts/PKITest/.step/templates/ChromeSCEP.tpl"
}
}
}
]
},
"tls": {
"cipherSuites": [
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
],
"minVersion": 1.2,
"maxVersion": 1.3,
"renegotiation": false
}
}The Powershell script actually build the exact configs i use and I don't really modify them afterwards (rather i rebuild them using the script for consistency). |
|
No problem regarding not sharing all details. I think we can do without it too if I can set up an environment myself. Actually curious what it works like 🙂. One thing I noticed about your |
|
Corrected The part of |
|
Didn't expect it to matter much either, but better to be safe 🙂 You're right about the docs lacking a bit in this regard. Being able to change the algorithm is still kind of new and updating the docs is on my list. Haven't had time to look further into it yet. |
|
@JMyklebust: wanted to let you know that we're getting things in place to be able to test this. We need a test Workplace environment and Chromebook for this. Using a VM may still be an option, but need different hardware for that. |
|
@JMyklebust: could you try it with a 1024 bits RSA key for the CA (and maybe also the client)? That will probably not be what you want for your actual deployment, but I found this issue with the same error message: micromdm/scep#79 (comment). Since that's the library we use for PKCS7, I'm curious to know if the same solution works for you and then we might be able to narrow it down. |
|
I'll try to get around to test that next week. Though we had to push ahead a bit, we've set up a Windows NDES service so we could go ahead and test the rest of our configuration for the Chromebooks. I did note that while setting up for the NDES service that Google recommends changing one of the IIS settings like this. |
|
I tried to reason about the flows through which this error can be reached. There don't seem to be many so far. Assuming the correct hashing algorithm is (correctly) encoded in the message (and retrieved by Go), the only things that can differ are the encoded digest and the (parsed) content bytes. There's another PR open for the PKCS7 library that touches the content that is hashed, so it might be related: mozilla-services/pkcs7#64. I haven't been able to create a reproducer myself yet, though, so I can't easily test it with this change. Is it an option for you to share a full example message using a private communication channel? |
|
Haven't gotten around to test with a Chromebook just yet. But the CA keys are 1024 (and signs certs with 2048 keys just fine) so still need to do a propert test with a Chromebook (and can do that tomorrow). |
|
@JMyklebust: Do you mean you haven't tested with an actual Chromebook so far? Or just this week? If so, how did you do the testing before that? Just with Is it an option for you to share a SCEP request in a private channel instead? |
|
I haven't tested using 1024 bits CA keys with a Chromebook yet. I'm switch between working from home and from the office, and currently my test Chromebook is at the office. And I can probably share a SCEP request in a private channel yeah. |
Ah, gotcha. I was trying to make sure I understood you correctly 🙂
That would be great! |
|
I've just tested and it indeed works if the CA-keys are 1024 bit. I also found out that the SCEP profile accepts localhost as SCEP server address (since the Google Certificate Collector and step-ca are running on the same server and I'm using http). The initial pull of the request: I think this is processing the request: The signed certificate: So this is for the successfull request using 1024bit CA-keys. This is a failed request using 2048bit CA-keys.
The returned error from step-ca to the certificate connector. This is the log line from step-ca: |
|
@JMyklebust: thank you for these logs 😄 So, based on your observation with the 1024 bit key it seems we're on the right track with this issue: micromdm/scep#79. I saw from the logs that the user agent is I tried extracting some info from the logs, but those don't seem to contain the actual SCEP requests performed against |
|
Luckily this is a fresh server so the loopback interface is pretty quiet. Log: Wireshark trace: Edit: |
|
@JMyklebust: I'm getting closer to a solution. I've created a small POC to read your SCEP request. I've changed the part that reads the contents to this: if compound.IsCompound {
rest := compound.Bytes
for len(rest) > 0 {
if rest, err = asn1.Unmarshal(rest, &compound); err != nil {
return nil, err
}
// Don't allow further constructed types.
if compound.Class != asn1.ClassUniversal || compound.Tag != asn1.TagOctetString || compound.IsCompound {
return nil, errors.New("bad class or tag")
}
content = append(content, compound.Bytes...)
}
} else {
content = compound.Bytes
}That is part of what's in omorsi/pkcs7#2. The SCEP request then can be parsed correctly and results in a I'll need to see how we want to fix this going forward. Ideally the fix gets upstreamed (soon), but we could use a fork or create our own too. Either way, you should soon be able to use |
|
That's great news! 😄 |
|
@JMyklebust: I've opened a PR that uses a fork of the https://github.com/mozilla-services/pkcs7 library that is used to parse the contents of the SCEP request: #932. I've tested the fork with the patch applied on your example message and that was successful. You could try if the change works in your environment by building |
|
Setting up a build enviroment for Go is a bit more work than what i can put into this just at the moment. Might be able to find time to build it myself later, but might take a few days at the least. |
|
@JMyklebust: are you willing to run a custom build provided by me? Another alternative is to see if we can get an RC out soon. |
|
I've just merged #932 into master, so that the next release of |
|
Sorry a lot of stuff keeping me busy. |
|
@JMyklebust: I'll ask if we have an ETA for a release (candidate). We don't follow a strict release cadence at the moment, so it's hard to say for sure at this time. An RC should be possible within days, though. |
|
@JMyklebust: we're aiming for a release today/tomorrow |
|
Took me a bit of time to get around to test, but finally had time. |
|
@JMyklebust: that's great to hear! 🥳 Thank you for testing, too! I'm going to add the example message you provided as a positive test vector, so that we can go forward with our SCEP implementation with (more) confidence regarding backwards compatibility 🙂 |
Subject of the issue
We are trying to implement step-ca as the certificate provider for client certifcates to Chromebook devices in our organization.
Google Workspace uses SCEP protocol to request certificates.
Here is some of Google's documentation on this, they mostly refer to Windows ADCS as SCEP service:
https://support.google.com/chrome/a/answer/11053129?hl=en&ref_topic=6330253
https://support.google.com/chrome/a/answer/11338941?hl=en
But there is no technical limitation from Google's side on which CA service to use.
I have built a test enviroment that can issue SCEP certficates and testing the basic issuing using the application "sscep".
These parts are all working using the sscep client software, however when we test with a Chromebook we get the error:
scep post request failed: pkcs7: Message digest mismatchYour environment
Steps to reproduce
I've attached a Powershell script that i use to build the entire CA (and optionally install as a service using "shawl").
I've also attached a zip containing the exact executables i use for reference.
TestEXE.zip
WindowsCA.ps1.txt
But specifically these are used in addtion to step 0.18.2:
Shawl - https://github.com/mtkennerly/shawl - Pre-compiled version 1.1.0 ( not needed if not installing as a service)
SSCEP - https://github.com/certnanny/sscep - Self-compiled binary for Windows based on v0.10.0
OpenSSL - https://slproweb.com/products/Win32OpenSSL.html - using win64 v3.0.2
For testing with Google Workspace and Chromebook, we have set up a SCEP profile with these settings (omitted setting are blank/default):
<hostname:port>/scep/scepca<challenge password><Intermediate CA certificate>Obviously you need a Google Workspace and Chromebook to set this up.
The Google Certificate Connection got installed using a local user on the server (because the installer required it), and then the service was changed to use "LocalService" through Windows Services.
Expected behaviour
Expected that the Chromebook would complete the certificate request.
Request is passed to step-ca with Google Certificate Connector as a proxy relay (certificate request flow is decribed in Google's documentation "Configure SCEP with ADCS for Chromebooks" as Appendix A).
But short version is that Chromebook sends request to Google Workspace, then the Certificate Connector polls Workspace for any pending request. If request are pending they get pulled by the connector for signing and are pushed back to Workspace after signing is complete, afterwards Workspace push the certficate to the Chromebook.
Actual behaviour
Certificate request goes through, but final signing from step-ca seems to fail with error
scep post request failed: pkcs7: Message digest mismatch.Additional context
We managed to extract one of the failed requests. Since the request itelf only contains test-data there is no issue with sharing it here:
Of note a difference with the test performed via OpenSSL made request and the request from the Chromebook is this (after decoding the request via OpenSSL):
The text was updated successfully, but these errors were encountered: