Invoke-WebRequest and Invoke-RestMethod do not support invalid HTTPS certificates #1945

Closed
ffeldhaus opened this Issue Aug 19, 2016 · 9 comments

Projects

None yet

7 participants

@ffeldhaus
Contributor

On Windows, I was able to connect to HTTPS endpoints with invalid certificates by setting a certificate policy via System.Net.ServicePointManager to ignore invalid certificates.

An integrated way to ignore invalid certificates with Invoke-RestMethod and Invoke-WebMethod would simplify implementing this cross platform.

When I run the same on Mac OS X, I get several errors regarding .NET dependencies. How can Invoke-RestMethod and Invoke-WebMethod be configured to ignore invalid certificates?

Steps to reproduce

add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem) {
            return true;
        }
 }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
Invoke-WebRequest https://expired.badssl.com/

Expected behavior

This is from Windows Server 2008R2 with PowerShell 5:

PS C:\>             add-type @"
>>                     using System.Net;
>>                     using System.Security.Cryptography.X509Certificates;
>>                     public class TrustAllCertsPolicy : ICertificatePolicy {
>>                        public bool CheckValidationResult(
>>                             ServicePoint srvPoint, X509Certificate certificate,
>>                             WebRequest request, int certificateProblem) {
>>                             return true;
>>                         }
>>                     }
>> "@
>>
PS C:\> [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
PS C:\> Invoke-WebRequest https://expired.badssl.com/


StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html>
                    <head>
                      <meta name="viewport" content="width=device-width, initial-scale=1">
                      <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
                      <link rel="apple-touch-icon" href="/i...
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Accept-Ranges: bytes
                    Content-Length: 469
                    Cache-Control: no-store
                    Content-Type: text/html
                    Date: Fri, 19 Aug 2016 08:36:02 GMT
                    ETag: "57b263a4-1d5"
                    Last-Mo...
Forms             : {}
Headers           : {[Connection, keep-alive], [Accept-Ranges, bytes], [Content-Length, 469], [Cache-Control,
                    no-store]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 469

Actual behavior

PS /Users/ffeldhaus>             add-type @"                                                                                                                                                                                                  >>                     using System.Net;                                                                                                                                                                                                      >>                     using System.Security.Cryptography.X509Certificates;                                                                                                                                                                   >>                     public class TrustAllCertsPolicy : ICertificatePolicy {                                                                                                                                                                >>                        public bool CheckValidationResult(                                                                                                                                                                                  >>                             ServicePoint srvPoint, X509Certificate certificate,                                                                                                                                                            >>                             WebRequest request, int certificateProblem) {                                                                                                                                                                  >>                             return true;                                                                                                                                                                                                   >>                         }                                                                                                                                                                                                                  >>                     }                                                                                                                                                                                                                      >> "@                                                                                                                                                                                                                                         
add-type : (2) : The type or namespace name 'X509Certificates' does not exist in the namespace 'System.Security.Cryptography' (are you missing an assembly reference?)
(1) :                     using System.Net;
(2) : >>>                     using System.Security.Cryptography.X509Certificates;
(3) :                     public class TrustAllCertsPolicy : ICertificatePolicy {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (3) : The type or namespace name 'ICertificatePolicy' could not be found (are you missing a using directive or an assembly reference?)
(2) :                     using System.Security.Cryptography.X509Certificates;
(3) : >>>                     public class TrustAllCertsPolicy : ICertificatePolicy {
(4) :                        public bool CheckValidationResult(
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (5) : The type or namespace name 'ServicePoint' could not be found (are you missing a using directive or an assembly reference?)
(4) :                        public bool CheckValidationResult(
(5) : >>>                             ServicePoint srvPoint, X509Certificate certificate,
(6) :                             WebRequest request, int certificateProblem) {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (5) : The type or namespace name 'X509Certificate' could not be found (are you missing a using directive or an assembly reference?)
(4) :                        public bool CheckValidationResult(
(5) : >>>                             ServicePoint srvPoint, X509Certificate certificate,
(6) :                             WebRequest request, int certificateProblem) {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (6) : The type or namespace name 'WebRequest' could not be found (are you missing a using directive or an assembly reference?)
(5) :                             ServicePoint srvPoint, X509Certificate certificate,
(6) : >>>                             WebRequest request, int certificateProblem) {
(7) :                             return true;
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : Cannot add type. Compilation errors occurred.
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Add-Type], InvalidOperationException
    + FullyQualifiedErrorId : COMPILER_ERRORS,Microsoft.PowerShell.Commands.AddTypeCommand
PS /Users/ffeldhaus> [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy                                                                                                                                     
Unable to find type [System.Net.ServicePointManager].
At line:1 char:1
+ [System.Net.ServicePointManager]::CertificatePolicy = New-Object Trus ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.ServicePointManager:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

PS /Users/ffeldhaus> Invoke-WebRequest https://expired.badssl.com/                                                                                                                                                                            
Invoke-WebRequest : An error occurred while sending the request.
At line:1 char:1
+ Invoke-WebRequest https://expired.badssl.com/
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Method: GET, Re...rShell/6.0.0
}:HttpRequestMessage) [Invoke-WebRequest], HttpRequestException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Environment data

Name                           Value
----                           -----                                                                                                                                                            
PSVersion                      6.0.0-alpha
PSEdition                      Core
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   3.0.0.0
GitCommitId                    v6.0.0-alpha.9
CLRVersion
WSManStackVersion              3.0  
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
@ffeldhaus
Contributor

Trusting self signed certifciates can be achieved in Mac OS X by adding them to the Mac OS X keychain (e.g. by opening the URI with Safari and always trusting the certificate) and restarting powershell. Then Invoke-RestMethod and Invoke-WebRequest will accept the certificate as trusted.

@Jaykul
Jaykul commented Aug 20, 2016

This isn't a question of trusting certificates. This is a question of getting/posting something even though you don't trust the certificate.

@lzybkr lzybkr added the Area-Cmdlets label Aug 20, 2016
@GavinEke

It would nice to have some a parameter like -IgnoreCertificateCheck which would be similar to wget's --no-check-certificate then also having a $IgnoreCertificateCheckPreference which can default to $false but still allows people to change it.

@ffeldhaus
Contributor
ffeldhaus commented Aug 21, 2016 edited

A -IgnoreCertificateCheck parameter could probably be implement using ServerCertificateCustomValidationCallback. This Gist has an example of using ServerCertificateCustomValidationCallback to suppress certificate checks. This probably has to be implemented in the GetHttpClient() method in WebRequestPSCmdlet.CoreClr.cs.

@halr9000
Contributor

Let's PLEASE have a skip SSL verification switch at last! Nobody likes dealing with certs in development.

@ffeldhaus
Contributor
ffeldhaus commented Aug 22, 2016 edited

I created pull request #2006 to implement a -SkipCertificateCheck switch parameter. It does not yet support a $SkipCertificateCheckPreference variable, but the functionality can be achieved using $PSDefaultParameterValues e.g. $PSDefaultParameterValues.Add("Invoke-RestMethod:SkipCertificateCheck",$true) or $PSDefaultParameterValues.Add("Invoke-WebRequest:SkipCertificateCheck",$true).

This also requires libcurl to be build against openssl. See dotnet/corefx#9728 for details and instructions how to build curl against openssl.

@nabarry
nabarry commented Sep 27, 2016

Please implement pull request #2006 This is critical functionality for internal API access to systems.

@SteveL-MSFT SteveL-MSFT added this to the 6.0.0 milestone Sep 29, 2016
@Jaykul
Jaykul commented Sep 30, 2016

This needs to be done right -- not with a -IgnoreAllSSLErrors switch...

P.S. for those who need something right now you can use my TunableSSLValidator module

@ffeldhaus
Contributor

As pull request #2006 is now merged, I'm closing the issue. The functionality (e.g. -SkipCertificateCheck switch) is now similar to that offered via curl and other standard tools.

@ffeldhaus ffeldhaus closed this Nov 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment