Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoke-WebRequest and Invoke-RestMethod do not support invalid HTTPS certificates #1945

Closed
ffeldhaus opened this issue Aug 19, 2016 · 13 comments
Labels
Issue-Enhancement the issue is more of a feature request than a bug Resolution-Fixed The issue is fixed. WG-Cmdlets general cmdlet issues

Comments

@ffeldhaus
Copy link
Contributor

On Windows, I was able to connect to HTTPS endpoints with invalid certificates by setting a certificate policy via System.Net.ServicePointManager to ignore invalid certificates.

An integrated way to ignore invalid certificates with Invoke-RestMethod and Invoke-WebMethod would simplify implementing this cross platform.

When I run the same on Mac OS X, I get several errors regarding .NET dependencies. How can Invoke-RestMethod and Invoke-WebMethod be configured to ignore invalid certificates?

Steps to reproduce

add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
        ServicePoint srvPoint, X509Certificate certificate,
        WebRequest request, int certificateProblem) {
            return true;
        }
 }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
Invoke-WebRequest https://expired.badssl.com/

Expected behavior

This is from Windows Server 2008R2 with PowerShell 5:

PS C:\>             add-type @"
>>                     using System.Net;
>>                     using System.Security.Cryptography.X509Certificates;
>>                     public class TrustAllCertsPolicy : ICertificatePolicy {
>>                        public bool CheckValidationResult(
>>                             ServicePoint srvPoint, X509Certificate certificate,
>>                             WebRequest request, int certificateProblem) {
>>                             return true;
>>                         }
>>                     }
>> "@
>>
PS C:\> [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
PS C:\> Invoke-WebRequest https://expired.badssl.com/


StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html>
                    <head>
                      <meta name="viewport" content="width=device-width, initial-scale=1">
                      <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
                      <link rel="apple-touch-icon" href="/i...
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Accept-Ranges: bytes
                    Content-Length: 469
                    Cache-Control: no-store
                    Content-Type: text/html
                    Date: Fri, 19 Aug 2016 08:36:02 GMT
                    ETag: "57b263a4-1d5"
                    Last-Mo...
Forms             : {}
Headers           : {[Connection, keep-alive], [Accept-Ranges, bytes], [Content-Length, 469], [Cache-Control,
                    no-store]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 469

Actual behavior

PS /Users/ffeldhaus>             add-type @"                                                                                                                                                                                                  >>                     using System.Net;                                                                                                                                                                                                      >>                     using System.Security.Cryptography.X509Certificates;                                                                                                                                                                   >>                     public class TrustAllCertsPolicy : ICertificatePolicy {                                                                                                                                                                >>                        public bool CheckValidationResult(                                                                                                                                                                                  >>                             ServicePoint srvPoint, X509Certificate certificate,                                                                                                                                                            >>                             WebRequest request, int certificateProblem) {                                                                                                                                                                  >>                             return true;                                                                                                                                                                                                   >>                         }                                                                                                                                                                                                                  >>                     }                                                                                                                                                                                                                      >> "@                                                                                                                                                                                                                                         
add-type : (2) : The type or namespace name 'X509Certificates' does not exist in the namespace 'System.Security.Cryptography' (are you missing an assembly reference?)
(1) :                     using System.Net;
(2) : >>>                     using System.Security.Cryptography.X509Certificates;
(3) :                     public class TrustAllCertsPolicy : ICertificatePolicy {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (3) : The type or namespace name 'ICertificatePolicy' could not be found (are you missing a using directive or an assembly reference?)
(2) :                     using System.Security.Cryptography.X509Certificates;
(3) : >>>                     public class TrustAllCertsPolicy : ICertificatePolicy {
(4) :                        public bool CheckValidationResult(
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (5) : The type or namespace name 'ServicePoint' could not be found (are you missing a using directive or an assembly reference?)
(4) :                        public bool CheckValidationResult(
(5) : >>>                             ServicePoint srvPoint, X509Certificate certificate,
(6) :                             WebRequest request, int certificateProblem) {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (5) : The type or namespace name 'X509Certificate' could not be found (are you missing a using directive or an assembly reference?)
(4) :                        public bool CheckValidationResult(
(5) : >>>                             ServicePoint srvPoint, X509Certificate certificate,
(6) :                             WebRequest request, int certificateProblem) {
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : (6) : The type or namespace name 'WebRequest' could not be found (are you missing a using directive or an assembly reference?)
(5) :                             ServicePoint srvPoint, X509Certificate certificate,
(6) : >>>                             WebRequest request, int certificateProblem) {
(7) :                             return true;
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Power...peCompilerError:AddTypeCompilerError) [Add-Type], Exception
    + FullyQualifiedErrorId : SOURCE_CODE_ERROR,Microsoft.PowerShell.Commands.AddTypeCommand

add-type : Cannot add type. Compilation errors occurred.
At line:1 char:13
+             add-type @"
+             ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Add-Type], InvalidOperationException
    + FullyQualifiedErrorId : COMPILER_ERRORS,Microsoft.PowerShell.Commands.AddTypeCommand
PS /Users/ffeldhaus> [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy                                                                                                                                     
Unable to find type [System.Net.ServicePointManager].
At line:1 char:1
+ [System.Net.ServicePointManager]::CertificatePolicy = New-Object Trus ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.ServicePointManager:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

PS /Users/ffeldhaus> Invoke-WebRequest https://expired.badssl.com/                                                                                                                                                                            
Invoke-WebRequest : An error occurred while sending the request.
At line:1 char:1
+ Invoke-WebRequest https://expired.badssl.com/
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Method: GET, Re...rShell/6.0.0
}:HttpRequestMessage) [Invoke-WebRequest], HttpRequestException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Environment data

Name                           Value
----                           -----                                                                                                                                                            
PSVersion                      6.0.0-alpha
PSEdition                      Core
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   3.0.0.0
GitCommitId                    v6.0.0-alpha.9
CLRVersion
WSManStackVersion              3.0  
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
@ffeldhaus
Copy link
Contributor Author

Trusting self signed certifciates can be achieved in Mac OS X by adding them to the Mac OS X keychain (e.g. by opening the URI with Safari and always trusting the certificate) and restarting powershell. Then Invoke-RestMethod and Invoke-WebRequest will accept the certificate as trusted.

@Jaykul
Copy link
Contributor

Jaykul commented Aug 20, 2016

This isn't a question of trusting certificates. This is a question of getting/posting something even though you don't trust the certificate.

@lzybkr lzybkr added the WG-Cmdlets general cmdlet issues label Aug 20, 2016
@GavinEke
Copy link

It would nice to have some a parameter like -IgnoreCertificateCheck which would be similar to wget's --no-check-certificate then also having a $IgnoreCertificateCheckPreference which can default to $false but still allows people to change it.

@ffeldhaus
Copy link
Contributor Author

ffeldhaus commented Aug 21, 2016

A -IgnoreCertificateCheck parameter could probably be implement using ServerCertificateCustomValidationCallback. This Gist has an example of using ServerCertificateCustomValidationCallback to suppress certificate checks. This probably has to be implemented in the GetHttpClient() method in WebRequestPSCmdlet.CoreClr.cs.

@halr9000
Copy link
Contributor

Let's PLEASE have a skip SSL verification switch at last! Nobody likes dealing with certs in development.

@ffeldhaus
Copy link
Contributor Author

ffeldhaus commented Aug 22, 2016

I created pull request #2006 to implement a -SkipCertificateCheck switch parameter. It does not yet support a $SkipCertificateCheckPreference variable, but the functionality can be achieved using $PSDefaultParameterValues e.g. $PSDefaultParameterValues.Add("Invoke-RestMethod:SkipCertificateCheck",$true) or $PSDefaultParameterValues.Add("Invoke-WebRequest:SkipCertificateCheck",$true).

This also requires libcurl to be build against openssl. See https://github.com/dotnet/corefx/issues/9728 for details and instructions how to build curl against openssl.

@nabarry
Copy link

nabarry commented Sep 27, 2016

Please implement pull request #2006 This is critical functionality for internal API access to systems.

@SteveL-MSFT SteveL-MSFT added this to the 6.0.0 milestone Sep 29, 2016
@Jaykul
Copy link
Contributor

Jaykul commented Sep 30, 2016

This needs to be done right -- not with a -IgnoreAllSSLErrors switch...

P.S. for those who need something right now you can use my TunableSSLValidator module

@ffeldhaus
Copy link
Contributor Author

As pull request #2006 is now merged, I'm closing the issue. The functionality (e.g. -SkipCertificateCheck switch) is now similar to that offered via curl and other standard tools.

@Xeleema
Copy link

Xeleema commented Apr 13, 2017

This is needed for those wanting to use MSRC's "SecurityUpdates" Module through a Proxy.

@cawoodm
Copy link

cawoodm commented Mar 24, 2020

Which cacerts (approved CA Root) list does Invoke-WebRequest use? I have a curl request which works fine in bash on CentOS 7 but failed in pwsh with Invoke-WebRequest. It would seem PowerShell is not respecting the system's setup in /etc/pki/ca-trust/source/anchors.

@cawoodm
Copy link

cawoodm commented Mar 24, 2020

My bad, it does respect them. I had copied the .cer files to the anchors directory but forgotten to run sudo update-ca-trust extract.

@tikondrus
Copy link

tikondrus commented Dec 28, 2020

I have the same issue on Centos

PS /root/> Add-Type -TypeDefinition @"

using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
  public bool CheckValidationResult(
  ServicePoint srvPoint, X509Certificate certificate,
  WebRequest request, int certificateProblem) {
    return true;
  }
}

"@
Add-Type: (3,40): error CS0246: The type or namespace name 'ICertificatePolicy' could not be found (are you missing a using directive or an assembly reference?)
public class TrustAllCertsPolicy : ICertificatePolicy {
^

Add-Type: Cannot add type. Compilation errors occurred.
PS /root/>
PS /root/> [System.Net.ServicePointManager]::CertificatePolicy = New-Object -TypeName TrustAllCertsPolicy
New-Object: Cannot find type [TrustAllCertsPolicy]: verify that the assembly containing this type is loaded.

$PSVersionTable

Name Value


PSVersion 7.1.0
PSEdition Core
GitCommitId 7.1.0
OS Linux 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020
Platform Unix
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Enhancement the issue is more of a feature request than a bug Resolution-Fixed The issue is fixed. WG-Cmdlets general cmdlet issues
Projects
None yet
Development

No branches or pull requests

10 participants