-
-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Option to use windows ca store with openssl. #4346
Conversation
The travis builds using boringssl fail, due to unrelated reasons. The rest seem green! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And perhaps consider to squash all commits into a single one and force-push, to make this patch easier to review!
src/tool_operate.c
Outdated
// so the only possible solution is using the windows ca store | ||
config->cacert = strdup(CURL_WINDOWS_CA_STORE); | ||
if(!config->cacert && !config->capath) { | ||
/* now, we are under MS-Windows. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that this is not using a 2-space indent level, which all curl code does. You should probably make sure all your code follow that style.
I tried to made squash , but I’m not a git guru!
Can you made now the work for merging as possible?
Thing will probably not be easy for me if there is some github/git special stuff…
De : Daniel Stenberg [mailto:notifications@github.com]
Envoyé : dimanche 15 septembre 2019 17:27
À : curl/curl
Cc : gvollant; Author
Objet : Re: [curl/curl] Option to use windows ca store with openssl. (#4346)
@bagder commented on this pull request.
|
I don't understand why somes test fails... |
With "squash", all your changes should be made into a single commit. Right now this change is very hard to review since it is spread out over a lot of commits where most of them change things done in previous commits... With the git command line tool, it can be made like this:
I can work around this, but if I provide feedback on this patch how are you going to update it? Just keep adding commits and make it harder and harder for others to read? PS at least one of your commits changed line endings on the code which makes things complicated at least for me. |
First failed is test 714. Running it here gives me no clues either, but the
|
38211dc
to
26866fb
Compare
I done the squash |
.travis.yml
Outdated
sudo: required | ||
go: | ||
"1.13" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change seems not to be related.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's probably a merge or rebase... I don't want change myself ...
it comes probably from ac34c70
include/curl/curl.h
Outdated
curl compiled for MS-Windows using OpenSSL for using Windows CA Store | ||
to verify peer | ||
as suggested in https://curl.haxx.se/mail/lib-2018-09/0039.html */ | ||
#define CURL_WINDOWS_CA_STORE "\\\\\\WINDOWS_CA_STORE" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The number of backslashes seems a bit weird, especially considering escaping. Maybe think of something more clear?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is 3*2=6. backslash
on windows a path beginning with two backslash can be a network path
three baskslash is impossible so there is no conflict
https://en.wikipedia.org/wiki/Path_(computing)#Uniform_Naming_Convention )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you'd use three forward-slashes instead, you'd avoid a lot of escaping problems and still be pretty safe, won't you? I think the main problem is here if we expect command line users to be able to set this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, I just replaced backslash by forward slash
Is review less hard now these two PR ?
<#4346> #4346
all modification are on
<26866fb> 26866fb
And <#4356> #4356
87f313d
De : Daniel Stenberg [mailto:notifications@github.com]
Envoyé : lundi 16 septembre 2019 10:24
À : curl/curl
Cc : gvollant; Author
Objet : Re: [curl/curl] Option to use windows ca store with openssl. (#4346)
With "squash", all your changes should be made into a single commit. Right now this change is very hard to review since it is spread out over a lot of commits where most of them change things done in previous commits...
With the git command line tool, it can be made like this:
1. update your master to be in sync with upstream
2. check out your feature branch git checkout branch
3. rebase your branch: git rebase -i origin/branch - the -i means interacetive. Make all commit lines except the first one say 'f' in the first column, which means fixup and that means it'll be merged into the previous commit.
4. Now all commits should have been turned into a single one in your branch.
5. You can't push this normally now, you need to use git push -f to force push
I can work around this, but if I provide feedback on this patch how are you going to update it? Just keep adding commits and make it harder and harder for others to read?
PS at least one of your commits changed line endings on the code which makes things complicated at least for me.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#4346?email_source=notifications&email_token=AAIKFN64LM52PD5SRVULNKLQJ47DFA5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6YNSLA#issuecomment-531683628> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIKFN5N4SICH7BGMIO2MCLQJ47DFANCNFSM4IWOA7RA> . <https://github.com/notifications/beacon/AAIKFN6MZSP2WVP2SB3DVLLQJ47DFA5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6YNSLA.gif>
|
You also merged two unrelated commits (from another branch), which is what broke the tests. When you've squashed and rebased, this should only be a single commit and definitely not commits that were authored by someone else than you! I think this starts to look fine. I mostly miss the documentation of this new feature now. To think about: Is there a way an application can know or should be able to figure out if libcurl supports this at run-time? |
26866fb
to
4d11fe5
Compare
I tried fix the git again….
I also made #4371 from memory for libcurl
4356 modification are included in 4371
De : Daniel Stenberg [mailto:notifications@github.com]
Envoyé : mardi 17 septembre 2019 16:25
À : curl/curl
Cc : gvollant; Author
Objet : Re: [curl/curl] Option to use windows ca store with openssl. (#4346)
I done the squash
You also merged two unrelated commits (from another branch), which is what broke the tests. When you've squashed and rebased, this should only be a single commit and definitely not commits that were authored by someone else than you!
I think this starts to look fine. I mostly miss the documentation of this new feature now.
To think about: Is there a way an application can know or should be able to figure out if libcurl supports this at run-time?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#4346?email_source=notifications&email_token=AAIKFN2JXBT6T2LZTU6OAKLQKDSEPA5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD64WRPQ#issuecomment-532244670> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIKFN6O5VT26AZ67KUENF3QKDSEPANCNFSM4IWOA7RA> . <https://github.com/notifications/beacon/AAIKFN3C262OMYZGWCZH2FLQKDSEPA5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD64WRPQ.gif>
|
138a6fa
to
1058cc0
Compare
« Is there a way an application can know or should be able to figure out if libcurl supports this at run-time”
I’m not sure understanding. You want known if we can check if current version of windows support it?
I can only say I tested witn Windows 7 SP1 and Windows 10
De : Daniel Stenberg [mailto:notifications@github.com]
Envoyé : mardi 17 septembre 2019 16:25
À : curl/curl
Cc : gvollant; Author
Objet : Re: [curl/curl] Option to use windows ca store with openssl. (#4346)
I done the squash
You also merged two unrelated commits (from another branch), which is what broke the tests. When you've squashed and rebased, this should only be a single commit and definitely not commits that were authored by someone else than you!
I think this starts to look fine. I mostly miss the documentation of this new feature now.
To think about: Is there a way an application can know or should be able to figure out if libcurl supports this at run-time?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#4346?email_source=notifications&email_token=AAIKFN2JXBT6T2LZTU6OAKLQKDSEPA5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD64WRPQ#issuecomment-532244670> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIKFN6O5VT26AZ67KUENF3QKDSEPANCNFSM4IWOA7RA> . <https://github.com/notifications/beacon/AAIKFN3C262OMYZGWCZH2FLQKDSEPA5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD64WRPQ.gif>
|
Sorry, I don't have idea of what need modification? Can you help me? |
03b4db2
to
0311f16
Compare
I don't understand the travis problem |
0311f16
to
182f3c5
Compare
lib/vtls/openssl.c
Outdated
"ROOT"); | ||
|
||
if(hStore) { | ||
while((pContext = CertEnumCertificatesInStore(hStore, pContext)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do any attributes of the certificate have to be checked like notbefore notafter etc or does openssl do this
182f3c5
to
df1ee46
Compare
df1ee46
to
a16cf9c
Compare
Maybe also add a little mention in the docs for the option where this can be set? |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
63b05f8
to
eddb82a
Compare
eddb82a
to
a1692ab
Compare
I ask myself whether it is a good idea to document |
Yes, good idea. This is probably easy https://stackoverflow.com/questions/42432473/programmatically-read-root-ca-certificates-in-ios |
a1692ab
to
e3fbb17
Compare
I do a small text modification |
Lots of CI failures still. Remember to run
|
0350ac8
to
1f150c7
Compare
Thank you a lot, I did not known checksrc
I fixed the source problem, but I don’t known how I can fix:
https://github.com/curl/curl/actions/runs/98001972
De : Daniel Stenberg [mailto:notifications@github.com]
Envoyé : jeudi 7 mai 2020 10:23
À : curl/curl
Cc : Gilles Vollant; Author
Objet : Re: [curl/curl] Option to use windows ca store with openssl. (#4346)
Lots of CI failures still. Remember to run make checksrc locally to pick most of these ones up yourself:
./vtls/openssl.c:2889:82: warning: Longer than 79 columns (LONGLINE)
/* Continue with a warning if no certificate verification is required. */
./vtls/openssl.c:2901:82: warning: Longer than 79 columns (LONGLINE)
/* Continue with a warning if no certificate verification is required. */
checksrc: 0 errors and 2 warnings
checksrc: 0 errors and 5 warnings suppressed
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#4346 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIKFN3TNWNQ7W5J6V22HSDRQJVX7ANCNFSM4IWOA7RA> . <https://github.com/notifications/beacon/AAIKFN3QUYTRKDPHHSD6PZDRQJVX7A5CNFSM4IWOA7RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEVBGCBY.gif>
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is an inconsistency throughout in the documentation and log messages. Sometimes "root certificates", "ca certificates", "CA certificates", etc. Can you apply/propose consistent naming?
Nitpicking: also note that the native CA store might not only contain root certificates, but also intermediate ones.
.IP CURLSSLOPT_NATIVE_CA | ||
With the build against OpenSSL library, uses the native operating system | ||
store of root certificates instead of any other CA certificates. | ||
Currenly supported with MS-Windows. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please either write "Windows" or "Microsoft Windows" consistently.
store of root certificates instead of any other CA certificates. | ||
Currenly supported with MS-Windows. | ||
Caution: use only CURLOPT_CAINFO with curl 7.70.1 or later for MS-Windows | ||
using OpenSSL library. On other configuration, curl will have no certificate. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence has a weird word oder.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you suggest me alternative text modified?
My english is not perfect, so I prefer you suggest a new version than ask me fix, then review and ask modify again
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand the sentence at all. It describes CURLSSLOPT_NATIVE_CA
and then it speaks of CURLOPT_CAINFO
without explaining how they relate here. Can you elaborate what this is trying to say?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My edited version says this now, but I'd like to understand the comment about CURLOPT_CAINFO
so that we can perhaps add that:
Tell libcurl to use the operating system's native CA store for certificate
verifiction. Works when built to use OpenSSL on Windows. (Added in 7.71.0)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wa must remove CURLOPT_CAINFO reference. I wrote this test before we add CURLSSLOPT_NATIVE_CA
So now, we must only say CURLOPT_CAINFO is only for new version of curl (for windows , and pehaps on other OS later).
Before new version of curl, we need provide a file of certificate
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can possibly add that CURLSSLOPT_NATIVE_CA
overrides CURLOPT_CAINFO
if both are set...
For those who'd like to play with EKUs, try our Siemens CAs: |
Thanks a lot @gvollant for your very hard work on this and for your patience. This has now landed, scheduled to ship in the next release; 7.71.0. |
no problem, and thank you for help me. You learned me somes git command, the nice checksrc tool and help me with my poor english :-) |
Sorry for chiming in late here, but I just saw @bagder's tweet and figured this should at least be mentioned: There's been an extensive discussion among Python folks regarding OpenSSL + system trust stores in psf/requests#2966. The consensus seems to be that enumerating the Windows CA store isn't really a good idea.
So yeah - I know this is merged, I really don't want to take away from @gvollant's excellent work here, but there are some security concerns so I figured I should bring it up. |
One alternative approach could be to instead disable the verification in OpenSSL, ask it to provide the server certificate to libcurl and then validate it similar to the code in curl/lib/vtls/schannel_verify.c Line 530 in 92501a1
I think this applies to intermediate CAs as root CAs are kept up to date via Windows Update. Update: I was wrong, root CAs are also fetched on demand: https://docs.microsoft.com/en-us/previous-versions//cc751157(v=technet.10)?redirectedfrom=MSDN
|
Security wise, this option has many angles and aspects and it is not at all necessary worse than without it. Many people and applications are already converting the windows CA store into a PEM to use with curl and other applications so for those users, this step just makes things easier and less error-prone. The argument about which CA store to use and trust could even be used the other way around as using the Windows store could in some cases be easier and be more likely to be updated and accurate rather than the separate PEM file. And more in sync with other applications and subsystems that might also be used. The PEM file itself is often extracted from Mozilla and used outside of their verification code which similar to the Windows verification APIs also does a lot of other things and checks apart from using just the certs, so "just" using the PEM is also already a shortcut and one with caveats. I think it is important to clearly document what this option means and does and leave to the user to make an enlightened decision.
Yes, I think that would be an improvement! |
IIRC recent versions of Windows routinely poll for root certificates. If you have a specific "incorrect trust success" example let's hear it. Regarding verification I just don't see why we should mix it up any more. We have Schannel backend, and OpenSSL backend, and now both of them can use the OS root store or the certificate bundle. That seems like enough to me. |
I wasn't talking about mixing it up, just change the way the OpenSSL backend uses the root CA store. Instead of "mirroring" it into the OpenSSL store, it could just ask Windows to verify the certificate. That would solve any potential CA refreshing issues and could improve the overall startup performance. Slightly off topic: I must say I only ever experienced the CA refresh with Outlook on Windows missing some intermediates while validating S/MIME e-mail signatures. This lead to the e-mail signature first showing up as invalid and a few seconds later showing as valid. |
somes tips for verify under mac osx: get x509 objet (like in get_cert_chain in openssl.c ) and call i2d_X509 get get der encoded data in memory then call https://developer.apple.com/documentation/security/1392592-secpolicycreatessl?language=objc |
This patch allow using Windows certificate store for Windows curl compiled with openssl
the curl command line utility with automatically uses it when there is no --cacert , no --capath option and curl-ca-bundle.crt is not specified
(before the patch, the https connexion failed)
user can also enter
https://curl.haxx.se//mail/lib-2018-09/0038.html
https://curl.haxx.se//mail/lib-2018-09/0028.html