Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/x509: make SystemCertPool work on Windows? #16736

Open
bradfitz opened this issue Aug 16, 2016 · 24 comments
Open

crypto/x509: make SystemCertPool work on Windows? #16736

bradfitz opened this issue Aug 16, 2016 · 24 comments

Comments

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Aug 16, 2016

https://golang.org/pkg/crypto/x509/#SystemCertPool doesn't work on Windows:

    func SystemCertPool() (*CertPool, error) {
        if runtime.GOOS == "windows" {
            return nil, errors.New("crypto/x509: system root pool is not available on Windows")
        }
        ....

I checked it in with the commit message "SystemCertPool returns an error on Windows. Maybe it's fixable later." (a62ae9f, golang.org/cl/21293, #13335)

This bug is about fixing it.

/cc @alexbrainman

@bradfitz bradfitz added this to the Go1.8Maybe milestone Aug 16, 2016
@alexbrainman
Copy link
Member

@alexbrainman alexbrainman commented Aug 18, 2016

I really don't know, I am not security expert. But I think you want to open LocalMachine\root (or maybe CurrentUser\root) certificate store, and read all certificates there with CertEnumCertificatesInStore or similar. What do you think?

Alex

@bradfitz
Copy link
Contributor Author

@bradfitz bradfitz commented Aug 22, 2016

Sounds plausible.

I don't think this requires a security expert as much as somebody who can read MSDN docs.

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 7, 2016

CL https://golang.org/cl/30578 mentions this issue.

@gopherbot gopherbot closed this in 05471e9 Oct 17, 2016
mariash added a commit to concourse/fly that referenced this issue Nov 21, 2016
SystemCertPool is not supported on windows in go 1.7.
see golang/go#16736
Once 1.8 is released we can remove special condition and always append
to system cert pool.

[#133304007]

Signed-off-by: Maria Shaldibina <mshaldibina@pivotal.io>
@jeffallen
Copy link
Contributor

@jeffallen jeffallen commented Feb 14, 2017

Note: This change was rolled back in #18609. SystemCertPool on Windows on Go 1.8 still returns nil. @bradfitz Maybe you could re-open this and remove the go1.8maybe tag on it? Thanks.

@alexbrainman alexbrainman modified the milestones: Go1.9, Go1.8Maybe Feb 14, 2017
@alexbrainman alexbrainman reopened this Feb 14, 2017
@alexbrainman
Copy link
Member

@alexbrainman alexbrainman commented Feb 14, 2017

@jeffallen Done.

Alex

@felixbecker
Copy link

@felixbecker felixbecker commented Mar 20, 2018

Hi, came from this issue #18609 and try to understand what can help. Maybe as an look over the fence this is how dotnetcore address this (https://github.com/dotnet/corefx/tree/master/src/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates). Just trying to get a better understanding what fails and what could help.

@rolandshoemaker
Copy link
Member

@rolandshoemaker rolandshoemaker commented Apr 1, 2020

I think adding winapi documentation references is the right path, but there should also be an expectation that developers understand the APIs they are working with. For low-level, highly nuanced APIs like this, there shouldn't be any protections for the developer, if they are making these calls they should be reading the winapi documentation on how to use it and this behaviour should only be a very thin wrapper for the winapi.

But this isn't really a very low-level nuanced API, whereas syscall.CertOpenSystemStore certainly is. On (basically) every other system SystemCertPool does a very simple, straight forward thing, it provides a list of certificates that the system trusts. If a thin wrapper around syscall.CertOpenSystemStore is added for Windows then that changes and it does become a nuanced API, because it'll do one thing one most systems and another very, very different thing on Windows.

@rasky
Copy link
Member

@rasky rasky commented Sep 8, 2020

If a thin wrapper around syscall.CertOpenSystemStore is added for Windows then that changes and it does become a nuanced API, because it'll do one thing one most systems and another very, very different thing on Windows.

Can you please clarify this? How the snippet in #16736 (comment) is doing something very very different compared to what happens on other operating systems?

@rolandshoemaker
Copy link
Member

@rolandshoemaker rolandshoemaker commented Sep 8, 2020

Can you please clarify this? How the snippet in #16736 (comment) is doing something very very different compared to what happens on other operating systems?

Windows root store is dynamically loaded. So unlike on Linux/macOS, CertEnumCertificatesInStore doesn't return every root you trust, it returns the roots you trust which you've already encountered. This means that if you haven't yet used Windows' CryptoAPI to verify a certificate chain including root X then CertEnumCertificatesInStore won't return root X, even if it is actually trusted. Using it as a drop in for SystemCertPool means the user would typically get only a subset of trusted roots.

@docsmooth
Copy link

@docsmooth docsmooth commented Nov 12, 2020

Edit as upon further reading I see that the below is exactly what @mxplusb is doing in the workaround @nikkictl merged 2 weeks before my post.

IIRC, instead of calling certutil to update the internet, and instead of calling CertEnumCertificatesInStore() (which as @rolandshoemaker pointed out, is dynamic), can't you just enumerate the 2 stores yourself? Here's PowerShell I use to do that all the time:`$certlist=@()
$certStore = "Root"
foreach ($certRootStore in @("LocalMachine", "CurrentUser")) {
$store = new-object system.security.cryptography.x509certificates.x509Store $certStore, $certRootStore
$store.Open('ReadOnly')
gci cert:$certRootStore$certStore | foreach-object {
$certlist+=$_
}
}

return $certlist`

If you're testing that in PoSH (Linux pwsh 6 doesn't have a Cert drive, fyi), instead of "return $certlist" do:
$certlist | foreach-object { write-host $_.Thumbprint }

You'll see a HUGE number of thumbprints printed, one for each cert in the store (check by running "certlm.msc" from the Run window to and count the output thumbprints from the local store.

gitlab-runner-bot pushed a commit to gitlabhq/gitlab-runner that referenced this issue Dec 1, 2020
On Windows runners, the error message is always displayed for a job:

```
crypto/x509: system root pool is not available on Windows"
```

Since `x509.SystemCertPool` doesn't work on Windows, don't log a
warning.

References:

1. golang/go#16736
2. golang/go#18609

Closes https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3035
awnumar added a commit to awnumar/rosen that referenced this issue Dec 13, 2020
awnumar added a commit to awnumar/rosen that referenced this issue Dec 13, 2020
jhernand added a commit to jhernand/ocm-sdk-go that referenced this issue Feb 5, 2021
This patch adds a new `loadSystemCAs` method to the connection builder, with
two implementations, one for Windows and another one for other operating
systems where Go knows how to load the Trusted CA store. The Windows
implementation explicitly trusts the CAs that we need in order to connect to
the SSO and API services.

This is a temporary hack, till Go learns to load the Windows trusted CA store,
and will break if the SSO or API servers change the CA that they are using.

Related: golang/go#16736
Related: golang/go#18609
Signed-off-by: Juan Hernandez <juan.hernandez@redhat.com>
jhernand added a commit to jhernand/ocm-sdk-go that referenced this issue Feb 5, 2021
This patch adds a new `loadSystemCAs` method to the connection builder, with
two implementations, one for Windows and another one for other operating
systems where Go knows how to load the Trusted CA store. The Windows
implementation explicitly trusts the CAs that we need in order to connect to
the SSO and API services.

This is a temporary hack, till Go learns to load the Windows trusted CA store,
and will break if the SSO or API servers change the CA that they are using.

Related: golang/go#16736
Related: golang/go#18609
Signed-off-by: Juan Hernandez <juan.hernandez@redhat.com>
@mahdiamer
Copy link

@mahdiamer mahdiamer commented Feb 27, 2021

https://golang.org/pkg/crypto/x509/#SystemCertPool doesn't work on Windows:

    func SystemCertPool() (*CertPool, error) {
        if runtime.GOOS == "windows" {
            return nil, errors.New("crypto/x509: system root pool is not available on Windows")
        }
        ....

I checked it in with the commit message "SystemCertPool returns an error on Windows. Maybe it's fixable later." (a62ae9f, golang.org/cl/21293, #13335)

This bug is about fixing it.

/cc @alexbrainman

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.