schannel: add support for CURLOPT_CAINFO.

[mcnulty]

Modifications:

• make verify_certificate functionality in curl_schannel.c
  available on all versions of Windows instead of just
  Windows CE. verify_certificate will only be invoked on
  Windows CE or when the user specifies CURLOPT_CAINFO and
  CURLOPT_SSL_VERIFYPEER
• In verify_certificate, create a custom certificate chain
  engine that exclusively trusts the certificate store backed
  by the CURLOPT_CAINFO file
• HAVE_EXCLUSIVE_TRUST_MODE macro defined in config-win32.h for
  Windows 7+ to control enabling this feature that relies on
  Windows 7+ functionality
• doc updates of --cacert/CAINFO support for schannel
• Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
  when available. This implements a TODO in schannel.c to improve
  handling of multiple SANs in a certificate. In particular, all SANs
  will now be searched instead of just the first name.
• Add new test cases 314 and 315. These test cases check that
  the first and last SAN, respectively, matches the connection
  hostname. New test certificates have been added for these
  cases. For 314, the certificate prefix is Server-localhost-firstSAN
  and for 315, the certificate prefix is Server-localhost-secondSAN
• Remove TODO 15.2 (Add support for custom server certificate
  validation), this commit addresses it.

Testing

I didn't have any luck running the test suite for this particular platform/lib configuration so I validated the following test cases by hand:

• test305
• test310
• test311
• test312
• test314
• test315
• test2043

All the tests I validated (except for test305) required specifying --ssl-no-revoke because the test CA (EdelCurlRoot-ca) does not have any revocation checking data in its cert and SChannel requires revocation checking by default.

I also ran the testsuite (for good measure) on Ubuntu 16.04.2/x86_64, using ./configure without args (built against OpenSSL) and didn't see any test failures.

[mention-bot]

@mcnulty, thanks for your PR! By analyzing the history of the files in this pull request, we identified @masali-hp, @mback2k and @bagder to be potential reviewers.

[bagder]

The failed CI test is test 2033, which is yet another pipelining test we probably need to disable due to its flakiness.

[mback2k]

Thanks for your hard work on this feature and pull-request, I really appreciate it since I never had the time to implement this myself. I hope that you will find the time to address my review comments, thanks!

[mcnulty]

Thanks for your comments @mback2k! I hope to be able to address them within a week or so.

[mcnulty]

Just checking in on this PR. I think I addressed all of @mback2k's comments. Is there anything more I can do to help shepherd this change along?

[bagder]

Can you please squash the commits (and rebase them) so that they are the minimum set of logical changes required to provide this feature? It makes them easier for a final review and the rebase/ṕush will make them run through the (now) updated CI tests. (and I too would like @mback2k's ack on this)

[coveralls]

Coverage increased (+0.01%) to 76.01% when pulling 966583e on mcnulty:schannel-cainfo into ca3545d on curl:master.

[coveralls]

Coverage increased (+0.05%) to 76.022% when pulling 9cb1985 on mcnulty:schannel-cainfo into a548183 on curl:master.

[dscho]

@mcnulty the conflicts are most likely due to my recent changes to support switching the SSL backend at runtime. Would you like me to help you resolving them?

[mcnulty]

This PR has been rebased on a master.

@dscho: Thanks for the offer to help resolve the conflicts. I think I was successfully able to resolve the conflicts. It appears all that needed to be done was to define backend-specific data in the ssl_backend_data struct instead of in the ssl_connect_data struct and then utilize the BACKEND macro in schannel.c to access the backend-specific data. Does that align with your understanding?

[dscho]

It appears all that needed to be done was to define backend-specific data in the ssl_backend_data struct instead of in the ssl_connect_data struct and then utilize the BACKEND macro in schannel.c to access the backend-specific data. Does that align with your understanding?

Yes, it does! Thanks!

[mcnulty]

Looks like the Travis T=normal C=--enable-ares build failed on test 1208:

test 1208...[FTP PORT download, no data conn and no transient negative reply]
1208: protocol FAILED:

[jay]

In addition to the comments I made in the review one more thing concerns me, the behavior of the curl tool currently is set to auto-detect certificate bundles on windows and prior to this change when curl w/WinSSL was used nothing would happen, the default certificate bundle would still be used Edit: The default certificate store in Windows would still be used is more correct to say. With this change however the curl tool would prefer the bundle instead. I think we should consider making it so that the curl tool only passes the CAINFO option in the case of schannel when the user has explicitly requested it, and not use autodetect, that way we could still preserve backwards compatibility. Thoughts?

[jay]

If possible I would like for us to handle this at runtime, it seems that in CERT_CHAIN_ENGINE_CONFIG the flag is available in the struct even in earlier versions though it doesn't do anything. Can't we do it like Curl_verify_windows_version(6, 1, PLATFORM_WINNT, VERSION_GREATER_THAN_EQUAL);

[mcnulty]

Interesting. Perhaps the CERT_CHAIN_ENGINE_CONFIG definition has changed in more recent versions of the Windows SDK?

Here's what the definition looks like in the 8.0 SDK:

typedef struct _CERT_CHAIN_ENGINE_CONFIG {

    DWORD       cbSize;
    HCERTSTORE  hRestrictedRoot;
    HCERTSTORE  hRestrictedTrust;
    HCERTSTORE  hRestrictedOther;
    DWORD       cAdditionalStore;
    HCERTSTORE* rghAdditionalStore;
    DWORD       dwFlags;
    DWORD       dwUrlRetrievalTimeout;      // milliseconds
    DWORD       MaximumCachedCertificates;
    DWORD       CycleDetectionModulus;

#if (NTDDI_VERSION >= NTDDI_WIN7)
    HCERTSTORE  hExclusiveRoot;
    HCERTSTORE  hExclusiveTrustedPeople;
#endif

#if (NTDDI_VERSION >= NTDDI_WIN8)
    DWORD       dwExclusiveFlags;
#endif

} CERT_CHAIN_ENGINE_CONFIG, *PCERT_CHAIN_ENGINE_CONFIG;

When I explicitly set _WIN32_WINNT to 0x0600 (Windows Server 2008), the hExclusiveRoot field is not present in the structure. Given this, I don't think this check can be done at runtime.

But, I do notice that other ifdef checks specific to Schannel are in the schannel.c file, for example, HAS_ALPN. Given that, I can move this check into schannel.c (or schannel_verify.c) instead of having it live in config-win32.h.

[jay]

this seems like a large size to read into memory for a certificate bundle. I have a full mozilla certificate bundle that's like 256k

[mcnulty]

Good point. Would a smaller value like 512k or 1M work? I'll go with 1M unless there are objections to that.

[jay]

this I'm not fond of, windows users have the expectation that revocation checks are enabled by default. wouldn't we want to do the same for the certificate bundle they supply for consistency? (I'm assuming here that is what will happen since manual cred is now used for the bundle)

[mcnulty]

Great catch! Looks like revocation checks were being unconditionally disabled for CE, regardless of the setting of the no_revoke flag.

Interestingly enough, it looks like the call to CertGetCertificateChain overrides these flags by setting the CERT_CHAIN_REVOCATION_CHECK_CHAIN flag in the flags for that function if the no_revoke flag is set.

That said, I'll still update this code to set dwFlags based on the no_revoke flag.

[jay]

i rebased this and renamed the tests to avoid conflicts 314 => 316 and 315 => 317

[mcnulty]

Cool, thanks!

[mcnulty]

Thanks for your review @jay! I think I've addressed all of your comments with my most recent push.

I think we should consider making it so that the curl tool only passes the CAINFO option in the case of schannel when the user has explicitly requested it, and not use autodetect, that way we could still preserve backwards compatibility. Thoughts?

I added some code to tool_operate.c in the operate_do function to skip the autodetect search if the SSL backend is Schannel. I used the CURLINFO_TLS_SSL_PTR option for curl_easy_getinfo to determine the SSL backend in use. I'm not sure if this was the best way to determine the SSL backend, but it seemed to do the trick.

[mcnulty]

I went ahead and rebased this PR on master. Any chance these changes could be considered for inclusion in 7.60.0?

[jay]

Thanks Dan I will review again when I can.

edit 4-16 I will review this week before the feature window closes,

[jay]

Thanks. I changed the OS check from a build time check to a runtime check since it's common to build libcurl targeting an OS earlier than Windows 7, and yet we would still like this ability on Windows 7+ for those builds.