New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoke-WebRequest and Invoke-RestMethod do not support invalid HTTPS certificates #1945

Closed
ffeldhaus opened this Issue Aug 19, 2016 · 10 comments

Comments

Projects
None yet
8 participants
@ffeldhaus
Copy link
Contributor

ffeldhaus commented Aug 19, 2016

On Windows, I was able to connect to HTTPS endpoints with invalid certificates by setting a certificate policy via System.Net.ServicePointManager to ignore invalid certificates.

An integrated way to ignore invalid certificates with Invoke-RestMethod and Invoke-WebMethod would simplify implementing this cross platform.

When I run the same on Mac OS X, I get several errors regarding .NET dependencies. How can Invoke-RestMethod and Invoke-WebMethod be configured to ignore invalid certificates?

Steps to reproduce

add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem) {
            return true;
        }
 }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
Invoke-WebRequest https://expired.badssl.com/

Expected behavior

This is from Windows Server 2008R2 with PowerShell 5:

PS C:\>             add-type @"
>>                     using System.Net;
>>                     using System.Security.Cryptography.X509Certificates;
>>                     public class TrustAllCertsPolicy : ICertificatePolicy {
>>                        public bool CheckValidationResult(
>>                             ServicePoint srvPoint, X509Certificate certificate,
>>                             WebRequest request, int certificateProblem) {
>>                             return true;
>>                         }
>>                     }
>> "@
>>
PS C:\> [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
PS C:\> Invoke-WebRequest https://expired.badssl.com/


StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html>
                    <head>
                      <meta name="viewport" content="width=device-width, initial-scale=1">
                      <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
                      <link rel="apple-touch-icon" href="/i...
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Accept-Ranges: bytes
                    Content-Length: 469
                    Cache-Control: no-store
                    Content-Type: text/html
                    Date: Fri, 19 Aug 2016 08:36:02 GMT
                    ETag: "57b263a4-1d5"
                    Last-Mo...
Forms             : {}
Headers           : {[Connection, keep-alive], [Accept-Ranges, bytes], [Content-Length, 469], [Cache-Control,
                    no-store]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 469

Actual behavior

PS /Users/ffeldhaus>             add-type @"                                                                                                                                                                                                  >>                     using System.Net;                                                                                                                                                                                                      >>                     using System.Security.Cryptography.X509Certificates;                                                                                                                                                                   >>                     public class TrustAllCertsPolicy : ICertificatePolicy {                                                                                                                                                                >>                        public bool CheckValidationResult(                                                                                                                                                                                  >>                             ServicePoint srvPoint, X509Certificate certificate,                                                                                                                                                            >>                             WebRequest request, int certificateProblem) {                                                                                                                                                                  >>                             return true;                                                                                                                                                                                                   >>                         }                                                                                                                                                                                                                  >>                     }                                                                                                                                                                                                                      >> "@                                                                                                                                                                                                                                         
add-type : (2) : The type or namespace name 'X509Certificates' does not exist in the namespace 'System.Security.Cryptography' (are you missing an assembly reference?)
(1) :                     using System.Net;
(2) : >>>                     using System.Security.Cryptography.X509Certificates;
(3) :                     public class TrustAllCertsPolicy : ICertificatePolicy {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (3) : The type or namespace name 'ICertificatePolicy' could not be found (are you missing a using directive or an assembly reference?)
(2) :                     using System.Security.Cryptography.X509Certificates;
(3) : >>>                     public class TrustAllCertsPolicy : ICertificatePolicy {
(4) :                        public bool CheckValidationResult(
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (5) : The type or namespace name 'ServicePoint' could not be found (are you missing a using directive or an assembly reference?)
(4) :                        public bool CheckValidationResult(
(5) : >>>                             ServicePoint srvPoint, X509Certificate certificate,
(6) :                             WebRequest request, int certificateProblem) {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (5) : The type or namespace name 'X509Certificate' could not be found (are you missing a using directive or an assembly reference?)
(4) :                        public bool CheckValidationResult(
(5) : >>>                             ServicePoint srvPoint, X509Certificate certificate,
(6) :                             WebRequest request, int certificateProblem) {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (6) : The type or namespace name 'WebRequest' could not be found (are you missing a using directive or an assembly reference?)
(5) :                             ServicePoint srvPoint, X509Certificate certificate,
(6) : >>>                             WebRequest request, int certificateProblem) {
(7) :                             return true;
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : Cannot add type. Compilation errors occurred.
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Add-Type], InvalidOperationException
    + FullyQualifiedErrorId : COMPILER_ERRORS,Microsoft.PowerShell.Commands.AddTypeCommand
PS /Users/ffeldhaus> [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy                                                                                                                                     
Unable to find type [System.Net.ServicePointManager].
At line:1 char:1
+ [System.Net.ServicePointManager]::CertificatePolicy = New-Object Trus ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.ServicePointManager:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

PS /Users/ffeldhaus> Invoke-WebRequest https://expired.badssl.com/                                                                                                                                                                            
Invoke-WebRequest : An error occurred while sending the request.
At line:1 char:1
+ Invoke-WebRequest https://expired.badssl.com/
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Method: GET, Re...rShell/6.0.0
}:HttpRequestMessage) [Invoke-WebRequest], HttpRequestException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Environment data

Name                           Value
----                           -----                                                                                                                                                            
PSVersion                      6.0.0-alpha
PSEdition                      Core
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   3.0.0.0
GitCommitId                    v6.0.0-alpha.9
CLRVersion
WSManStackVersion              3.0  
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
@ffeldhaus

This comment has been minimized.

Copy link
Contributor

ffeldhaus commented Aug 19, 2016

Trusting self signed certifciates can be achieved in Mac OS X by adding them to the Mac OS X keychain (e.g. by opening the URI with Safari and always trusting the certificate) and restarting powershell. Then Invoke-RestMethod and Invoke-WebRequest will accept the certificate as trusted.

@Jaykul

This comment has been minimized.

Copy link

Jaykul commented Aug 20, 2016

This isn't a question of trusting certificates. This is a question of getting/posting something even though you don't trust the certificate.

@lzybkr lzybkr added the Area-Cmdlets label Aug 20, 2016

@GavinEke

This comment has been minimized.

Copy link

GavinEke commented Aug 20, 2016

It would nice to have some a parameter like -IgnoreCertificateCheck which would be similar to wget's --no-check-certificate then also having a $IgnoreCertificateCheckPreference which can default to $false but still allows people to change it.

@ffeldhaus

This comment has been minimized.

Copy link
Contributor

ffeldhaus commented Aug 21, 2016

A -IgnoreCertificateCheck parameter could probably be implement using ServerCertificateCustomValidationCallback. This Gist has an example of using ServerCertificateCustomValidationCallback to suppress certificate checks. This probably has to be implemented in the GetHttpClient() method in WebRequestPSCmdlet.CoreClr.cs.

@halr9000

This comment has been minimized.

Copy link
Contributor

halr9000 commented Aug 21, 2016

Let's PLEASE have a skip SSL verification switch at last! Nobody likes dealing with certs in development.

@ffeldhaus

This comment has been minimized.

Copy link
Contributor

ffeldhaus commented Aug 22, 2016

I created pull request #2006 to implement a -SkipCertificateCheck switch parameter. It does not yet support a $SkipCertificateCheckPreference variable, but the functionality can be achieved using $PSDefaultParameterValues e.g. $PSDefaultParameterValues.Add("Invoke-RestMethod:SkipCertificateCheck",$true) or $PSDefaultParameterValues.Add("Invoke-WebRequest:SkipCertificateCheck",$true).

This also requires libcurl to be build against openssl. See dotnet/corefx#9728 for details and instructions how to build curl against openssl.

@nabarry

This comment has been minimized.

Copy link

nabarry commented Sep 27, 2016

Please implement pull request #2006 This is critical functionality for internal API access to systems.

@SteveL-MSFT SteveL-MSFT added this to the 6.0.0 milestone Sep 29, 2016

@Jaykul

This comment has been minimized.

Copy link

Jaykul commented Sep 30, 2016

This needs to be done right -- not with a -IgnoreAllSSLErrors switch...

P.S. for those who need something right now you can use my TunableSSLValidator module

@ffeldhaus

This comment has been minimized.

Copy link
Contributor

ffeldhaus commented Nov 3, 2016

As pull request #2006 is now merged, I'm closing the issue. The functionality (e.g. -SkipCertificateCheck switch) is now similar to that offered via curl and other standard tools.

@Xeleema

This comment has been minimized.

Copy link

Xeleema commented Apr 13, 2017

This is needed for those wanting to use MSRC's "SecurityUpdates" Module through a Proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment