Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to setup the dev certificate when using Docker in development #6199

Closed
Rick-Anderson opened this issue May 3, 2018 · 75 comments
Closed

How to setup the dev certificate when using Docker in development #6199

Rick-Anderson opened this issue May 3, 2018 · 75 comments

Comments

@Rick-Anderson
Copy link
Contributor

@Rick-Anderson Rick-Anderson commented May 3, 2018

related #3310
Javier is contact:
This needs to go in Enforce HTTPS in an ASP.NET Core
The first time you run dotnet after installing the SDK you get this message
Successfully installed the ASP.NET Core HTTPS Development Certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). For establishing trust on other platforms please refer to the platform specific documentation.
For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.

Copied from #3310
We also need to cover how to setup the dev certificate when using Docker in development:

  • Create an application on Visual Studio using the MVC template.
  • Run the app to ensure its working.
  • Add docker support for the application through the tooling.
  • Modify the dockerfile to expose the port 443 with
    EXPOSE 443
  • Modify the docker-compose override file to map ports, volumes and environement variables as follows (this will all be unnecessary after docker tooling has support for HTTPS):
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ASPNETCORE_URLS=https://localhost;http://localhost
      - ASPNETCORE_HTTPS_PORT=44349
    ports:
    # Replace the values on the left by the values on your launchSettings.json
      - "51217:80"
      - "44349:443"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets/:/root/.microsoft/usersecrets
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https/
  • Export the HTTPS certificate into a PFX file using the dev-certs global tool to %APPDATA%/ASP.NET/Https/<>.pfx using a password of your choice (recommended password new-guid on powershell)
  • On your project, open user secrets and add the following configuration keys:
{
    "Kestrel":{
        "Certificates":{
            "Default":{
                "Path":     "/root/.aspnet/https/<AppName>>.pfx",
                "Password": "<<Your-Password>>"
            }
        }
    }
}
  • Run your application within the container.
  • Navigate to the HTTP endpoint on your application
    • You should not see any warning about the HTTPS certificate being invalid.
    • You should be redirected to the HTTPS endpoint automatically.
@Rick-Anderson Rick-Anderson added the 2.1 label May 3, 2018
@Rick-Anderson Rick-Anderson added this to the 2018 Q3 September 30 milestone May 3, 2018
@Rick-Anderson
Copy link
Contributor Author

@Rick-Anderson Rick-Anderson commented May 22, 2018

Subject: RE: Docker for ASP.NET Core 2.1 Preview 2

. Provided that your docker file and docker compose look like the ones that VS generates when you add VS support, there are a couple of steps you need to take to enable it manually.
In essence, you need to get the HTTPS development certificate from your machine into the container image as a pfx file in a special location on disk that we recognize and provide the password through user.

• Modify the dockerfile to expose the port 443 with
• EXPOSE 443
• Modify the docker-compose override file to map ports, volumes and environment variables as follows:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=https://+;http://+
- ASPNETCORE_HTTPS_PORT=44349
ports:
# Replace the values on the left by the values on your launchSettings.json
- "51217:80"
- "44349:443"
volumes:
- ${HOME}/.microsoft/usersecrets/:/root/.microsoft/usersecrets
- ${HOME}/.aspnet/https:/root/.aspnet/https/
• Export the HTTPS certificate into a PFX file using the dev-certs global tool to ${HOME}/.aspnet/https/<>.pfx using a password of your choice (dotnet dev-certs https -ep ${HOME}/.aspnet/https/ -p <>)
• Add the password to the user secrets in your project.
• Add the password to the user secrets in your project (dotnet user-secrets set "Kestrel:Certificates:Development:Password" "<>"

per Javier:
For setting up a non development certificate on kestrel the instructions are the same as for the development case (obviously you need to provide the certificate) and you need to set the path to the certificate in a configuration key.
"Kestrel:Certificates:Default:Path"

@Rick-Anderson Rick-Anderson changed the title how to setup the dev certificate when using Docker in development: How to setup the dev certificate when using Docker in development: May 22, 2018
@Rick-Anderson Rick-Anderson changed the title How to setup the dev certificate when using Docker in development: How to setup the dev certificate when using Docker in development May 22, 2018
@csharpfritz
Copy link
Contributor

@csharpfritz csharpfritz commented Jun 18, 2018

Is there an update for this now they we are in RTM for 2.1 and preparing for 2.1.1 release?

@robinmanuelthiel
Copy link

@robinmanuelthiel robinmanuelthiel commented Jun 26, 2018

Can we please avoid things like "Create an application on Visual Studio using the MVC template." in the documentation for a cross-platform framework like .NET Core, that also runs on Linux and Mac devices?

@RaccoonDev
Copy link

@RaccoonDev RaccoonDev commented Jun 26, 2018

Hi.

The method works fine if I run the application in development environment. Is there are way to specify what certificate and port should kestrel use in production mode?

@dradovic
Copy link

@dradovic dradovic commented Jul 17, 2018

When using a pfx file generated by dotnet dev-certs on my local Windows machine, I ran into a

error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure

So I switched to simply calling dotnet dev-certs https --export-path ./localhost.pfx -p $LOCALHOST_CERTIFICATE_PWD as part of building the Docker image and then referencing that in Startup.cs using KestrelServerOptions:

listenOptions.UseHttps("../../localhost.pfx", Environment.GetEnvironmentVariable("LOCALHOST_CERTIFICATE_PWD"));

@kdcllc
Copy link

@kdcllc kdcllc commented Aug 1, 2018

I had an issue with Docker-Compose.yml and Visual Studio.Net 15.7.5.
The resolution was adding the entry for secret,json:

{
    "Kestrel":{
        "Certificates":{
            "Default":{
                "Path":     "/root/.aspnet/https/<AppName>>.pfx",
                "Password": "<<Your-Password>>"
            }
        }
    }
}
@se-augustus
Copy link

@se-augustus se-augustus commented Aug 25, 2018

Do these same rules apply to MacOS users? How about those of us using Kitematic? I'm wondering if I can extrapolate the notes above to fit into the areas on/in the attached screenshots.
screen shot 2018-08-24 at 8 08 40 pm
screen shot 2018-08-24 at 8 08 56 pm

@chrisckc
Copy link

@chrisckc chrisckc commented Sep 3, 2018

In essence, you need to get the HTTPS development certificate from your machine into the container image as a pfx file in a special location on disk that we recognize and provide the password through user.

I have tried this in Docker and unless I specify the certificate location and password in the Startup configuration it crashes with:
System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

I followed the guide provided by @Rick-Anderson which seems to imply that UseStartup() now checks for the existence of the cert at root/.aspnet/https/<>.pfx or checks the UserSecrets for Kestrel:Certificates:Development:Path and Kestrel:Certificates:Development:Password

The UserSecrets is working and configured correctly as i am using it to get the path and password, it works using this code:

                .UseKestrel(options =>
                {
                    var configuration = (IConfiguration)options.ApplicationServices.GetService(typeof(IConfiguration));
                    var httpsPort = configuration.GetValue("ASPNETCORE_HTTPS_PORT", 443);
                    var certPassword = configuration.GetValue<string>("Kestrel:Certificates:Development:Password");
                    var certPath = configuration.GetValue<string>("Kestrel:Certificates:Development:Path");

                    options.Listen(IPAddress.Any, httpsPort, listenOptions =>
                    {
                        listenOptions.UseHttps(certPath, certPassword);
                    });
                });
@chrisckc
Copy link

@chrisckc chrisckc commented Sep 3, 2018

I have found that if the path to the cert is specified in user secrets or an env var, it breaks.

I was able to get it to work using the default startup by removing the path which i added when trying a bunch of other things:
dotnet user-secrets remove "Kestrel:Certificates:Development:Path"

I can break it again by re-adding the path:
dotnet user-secrets set "Kestrel:Certificates:Development:Path" "/root/.aspnet/https/<<AppName>>.pfx"

The path is correct and the cert file exists as demonstrated in the code above, but for some reason it doesn't like having the path specified even if it is correct. The guide above does only say to add the password, but there should be no reason that adding the correct path, or even any valid path where the cert exists could not be made to work.

This only applies to Development environment, in Production environment it works as per the docs using the 'Default' path:
dotnet user-secrets set "Kestrel:Certificates:Default:Path" "/root/.aspnet/https/<<AppName>>.pfx"

@chrisckc
Copy link

@chrisckc chrisckc commented Sep 6, 2018

I have published a repo which uses this technique which can be used to reproduce this issue:
https://github.com/chrisckc/DotNetCoreAureliaSPA

@Rick-Anderson
Copy link
Contributor Author

@Rick-Anderson Rick-Anderson commented Oct 2, 2018

@danroth27 what's the priority on this? We might need some help - at least for the first draft.
@chrisckc could you provide a draft?

@chrisckc
Copy link

@chrisckc chrisckc commented Oct 4, 2018

@Rick-Anderson I should be revisiting that repo soon to build something out so will have a dig around to see what is causing the issue i posted about.

@danroth27 danroth27 added the P1 label Oct 4, 2018
@chrisckc
Copy link

@chrisckc chrisckc commented Oct 8, 2018

Ok, i traced back the error message:

System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found.

which occurs when "Kestrel:Certificates:Development:Path" is specified in the configuration from UserSecrets or env vars despite the path being correct and the file existing.

I cloned the KestrelHttpServer repo and traced the message back to the Load() method inside the KestrelConfigurationLoader class (line 216).

The Load() method calls LoadDefaultCert(ConfigurationReader)
LoadDefaultCert checks if the "Kestrel:Certificates:Default" config entry exists, if so then the cert is loaded using the LoadCertificate(CertificateConfig certInfo, string endpointName) method which just builds the path using Path.Combine(env.ContentRootPath, certInfo.Path) , which if certInfo.Path is an absolute path just returns certInfo.Path due to the way Path.Combine works.

However if the "Kestrel:Certificates:Default" config entry does not exist is call FindDeveloperCertificateFile(configReader, logger) which is where things get a little odd.

The following particularly ugly piece of code is used to locate the development certificate:

               if (configReader.Certificates.TryGetValue("Development", out var certificateConfig) &&
                    certificateConfig.Path == null &&
                    certificateConfig.Password != null &&
                    TryGetCertificatePath(out certificatePath) &&
                    File.Exists(certificatePath))
                {
                    var certificate = new X509Certificate2(certificatePath, certificateConfig.Password);
                    return IsDevelopmentCertificate(certificate) ? certificate : null;
                }
                else if (!File.Exists(certificatePath))
                {
                    logger.FailedToLocateDevelopmentCertificateFile(certificatePath);
                }

The TryGetCertificatePath(out certificatePath) method looks for a cert named $"{appName}.pfx" in any of the following paths to cover both Mac OSX and Windows:
$APPDATA/ASP.NET/https, $APPDATA/.aspnet/https, $HOME/ASP.NET/https, $HOME/.aspnet/https

That piece of code is completely broken if it's intention is to either use the certificateConfig.Path or go and find the certificate file from a default location and naming convention, which would be the sensible thing to do.

The result of that 'if' statement is that if a certificate path is supplied in the config it is never used.

It also will report a misleading error because a missing password would result in a log error which says that the certificate could not be located and also an exception inside the Load() method stating that that certificate could not be found.

Experiencing misleading errors has always been an issue when working with Microsoft software in the past, was hoping that the situation has been improving.

I also noticed that the IsDevelopmentCertificate method checks is the certificate.Subject is equal to "CN=localhost", which may not always be the case for every developers development environment. For example if you are developing using docker running on another machine or a VM etc.

The code can be simplified, i don't really see the need to use the 2 separate configuration structures: "Kestrel:Certificates:Default" and "Kestrel:Certificates:Development" and check the subject of the cert. Just one config with a check to try and find the cert from the default location and name if it is not supplied in ":Path"?

@chrisckc
Copy link

@chrisckc chrisckc commented Oct 8, 2018

Looking at the referenced issue on line 332 of KestrelConfigurationLoader.cs

aspnet/Hosting#1294

It looks like the broken code was added as part of that, which seemed to start off with good intentions. Maybe the behaviour is intentional, if so i would like to know the reasoning.

@Rick-Anderson Any documentation written to cover this setup with its current behaviour would look rather odd.

@danroth27
Copy link
Member

@danroth27 danroth27 commented Oct 8, 2018

@Rick-Anderson Rick-Anderson added the P0 label Oct 8, 2018
@javiercn
Copy link
Contributor

@javiercn javiercn commented Oct 8, 2018

That piece of code is completely broken if it's intention is to either use the certificateConfig.Path or go and find the certificate file from a default location and naming convention,

It's not meant to be used with a path, the development certificate key is only there to support tooling scenarios. If you want to specify a path and a password, simply use the Default key.

I also noticed that the IsDevelopmentCertificate method checks is the certificate.Subject is equal to "CN=localhost", which may not always be the case for every developers development environment.

It's only meant to support localhost scenarios (including docker localhost). Other scenarios will have security implications we would have to worry about.

It also will report a misleading error because a missing password would result in a log error which says that the certificate could not be located and also an exception inside the Load() method stating that that certificate could not be found.

There's no way AFAIK to check the validity of the password on the PFX.

@PrefabPanda
Copy link

@PrefabPanda PrefabPanda commented Oct 10, 2018

I've been trying to follow this example and I'm tearing my hair out!

I get these exceptions:
AuthenticationException: The remote certificate is invalid according to the validation procedure.
HttpRequestException: The SSL connection could not be established, see inner exception.

I have two containers, one talks to the other by HTTPS and by use of an alias configured in the compose file. The container being talked to doesn't have a public IP on purpose (Gateway Microservice architecture).

I have generated a certificate and am applying it to the service being called.

Any ideas?

@PrefabPanda
Copy link

@PrefabPanda PrefabPanda commented Oct 10, 2018

Turns out I needed this: https://blog.roushtech.net/2016/12/20/ignoring-ssl-certificate-errors-net-core-httpclient/

and yes this is in my development environment.

@scottaddie scottaddie removed this from the 2018 Q 4 ends Dec 31 milestone Oct 12, 2018
@ma1f
Copy link

@ma1f ma1f commented Apr 11, 2019

describe how to setup a base docker image for dotnet core 2.2 with https & http/2 (no longer appears to be enabled by default) here - https://medium.com/@ma1f/docker-dotnet-3d979f56efe6
key environment settings as follows - no need to setup listener in startup with ports etc.

ENV Kestrel:Certificates:Default:Path=/etc/ssl/private/cert.pfx
ENV Kestrel:Certificates:Default:Password=changeit
ENV Kestrel:Certificates:Default:AllowInvalid=true
ENV Kestrel:EndPointDefaults:Protocols=Http1AndHttp2
@jcoutch
Copy link

@jcoutch jcoutch commented Apr 18, 2019

For anyone pulling their hair out trying to get developer certificates working locally with a Docker container in Visual Studio without having to jump through all these hoops, there's a bug with Visual Studio and/or Kestrel:
dotnet/aspnetcore#9528

Visual Studio auto-generates the certificate name based on your project's name, whereas Kestrel is expecting the name to match to your application/assembly name. To work around this, rename your project to match your assembly name, restart Visual Studio, and the certificate will get auto-generated with the correct name in %APPDATA%\ASP.NET\Https, and as long as you're using the default builder for your ASP.NET site (or manually add user secrets to your config builder), Kestrel will happily use the auto-generated developer certificate when running locally.

@killnine
Copy link

@killnine killnine commented Apr 23, 2019

Thanks, @jcoutch this worked for me. It's a little jankey seeing the full assembly name as the project name in my solution but w/e....

@Rick-Anderson
Copy link
Contributor Author

@Rick-Anderson Rick-Anderson commented May 2, 2019

@scottaddie are you able to do this in the next 6 weeks or should we assign to the PU?

@happyesthete
Copy link

@happyesthete happyesthete commented May 5, 2019

Easiest workaround, works for me (when creating new project with https and docker support with linux container):

  1. From visual studio, while creating new project select checkbox to add Docker Support. Visual studio will cry for Docker Desktop is not running. Ignore that.
  2. Next, add Orchestration support in your project and a docker compose will be generated automatically.
  3. Now run and access your app with docker ip.
    Hopefully, everything works if above steps done in sequence.

Note: I am on a Windows 10 Home VM running on Mac inside Parallels. I stopped and removed all the old docker containers before trying this, where I was facing the errors mentioned here.

@scottaddie
Copy link
Member

@scottaddie scottaddie commented May 6, 2019

@Rick-Anderson Please assign to PU

@Rick-Anderson Rick-Anderson removed the 2.1 label Jul 12, 2019
@mkArtakMSFT mkArtakMSFT added this to To Do in MVC.Docs via automation Sep 5, 2019
@javiercn
Copy link
Contributor

@javiercn javiercn commented Sep 6, 2019

@mkArtakMSFT I've already pointed out to the related docs, now sure what you expect from me here. I think this was already discussed on a thread and addressed, but I might be wrong. @danroth27 can confirm.

@mkArtakMSFT
Copy link
Contributor

@mkArtakMSFT mkArtakMSFT commented Sep 6, 2019

I see, hadn't read through all this. Should this be closed then? @Rick-Anderson ?

MVC.Docs automation moved this from To Do to Done Sep 6, 2019
@danroth27 danroth27 reopened this Sep 6, 2019
@danroth27
Copy link
Member

@danroth27 danroth27 commented Sep 6, 2019

The content in https://github.com/dotnet/dotnet-docker/blob/master/samples/aspnetapp/aspnetcore-docker-https.md should be part of our official docs. Someone needs to do the work of turning that content into official doc content.

@Rick-Anderson
Copy link
Contributor Author

@Rick-Anderson Rick-Anderson commented Sep 6, 2019

I've done that - see https://docs.microsoft.com/en-us/aspnet/core/security/docker-https?view=aspnetcore-2.2
Not sure why this didn't get closed.

@Rick-Anderson
Copy link
Contributor Author

@Rick-Anderson Rick-Anderson commented Sep 6, 2019

@danroth27
Copy link
Member

@danroth27 danroth27 commented Sep 6, 2019

Ah! Cool, sounds good then 😃.

@jorgeolive
Copy link

@jorgeolive jorgeolive commented Jan 2, 2020

I know this is closed already.. but, does the suggested configuration work with 3.0? I've followed above steps and I am getting a weird docker compose error "Duplicate mount point: /root/.aspnet/https" using the following images:

FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-buster-slim AS base

FROM mcr.microsoft.com/dotnet/core/sdk:3.0-buster AS build

Any help would be greatly appreciated.

Thanks,
Jorge.

@javiercn
Copy link
Contributor

@javiercn javiercn commented Jan 7, 2020

Hi.

It looks like you are posting on a closed issue!

We're very likely to lose track of your bug/feedback/question unless you:

  1. Open a new issue
  2. Explain very clearly what you need help with
  3. If you think you have found a bug, include detailed repro steps so that we can investigate the problem
@jnpwly
Copy link
Contributor

@jnpwly jnpwly commented Jan 14, 2020

@jorgeolive -- kia ora. Did you end up opening a new issue regarding this and whether it works on .NET Core 3.x? I am also having difficulty getting a multi-containerised topology up and running, so it is either "user error" on my part, or possibly something you have already fixed. So, I'm just checking :)

@KevinBurton
Copy link

@KevinBurton KevinBurton commented Jan 31, 2020

I started with the suggestions Rick-Anderson commented on May 22, 2018 but I am running into 'access denied' when exporting the certificate, for not only the .aspnet path as suggested but what seems to be any path. Ideas on how to get over this hurdle?

@javiercn
Copy link
Contributor

@javiercn javiercn commented Jan 31, 2020

Hi.

It looks like you are posting on a closed issue!

We're very likely to lose track of your bug/feedback/question unless you:

  1. Open a new issue
  2. Explain very clearly what you need help with
  3. If you think you have found a bug, include detailed repro steps so that we can investigate the problem
@bilalmalik777
Copy link

@bilalmalik777 bilalmalik777 commented Jan 1, 2021

i updated my application from 2.2 to 3.1 and facing the following error.It was working perfectly in 2.2 with docker
'error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure'

it was working fine with 2.2 but now facing an error in development mode. i run the following command to generate the dev certificate

dotnet dev-certs https -ep %APPDATA%\ASP.NET\Https\TT.Core.Portal.Web.AzureHybrid.pfx -p password
dotnet dev-certs https --trust
dotnet user-secrets -p TT.Core.Portal.Web.AzureHybrid.csproj set "Kestrel:Certificates:Development:Password" "password"

program.cs

            Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseKestrel(options =>
                             {
                                 bool.TryParse(Environment.GetEnvironmentVariable("IsDockerDeployment"), out bool isDockerDeployment);
                                 if (isDockerDeployment)
                                 {
                                     options.Listen(new IPEndPoint(IPAddress.Any, 443), listenOptions =>
                                     {
                                         var configuration = (IConfiguration)options.ApplicationServices.GetService(typeof(IConfiguration));
                                         var certPassword = Environment.GetEnvironmentVariable("ASPNETCORE_Kestrel__Certificates__Development__Password");
                                         var certPath = Environment.GetEnvironmentVariable("ASPNETCORE_Kestrel__Certificates__Development__Path");
                                         Console.WriteLine(certPassword);
                                         Console.WriteLine(certPath);

                                         var certificate = new X509Certificate2(certPath, certPassword);
                                         Console.WriteLine("Certificate provided");
                                         var httpsConnectionAdapterOptions = new HttpsConnectionAdapterOptions()
                                         {
                                             ClientCertificateMode = ClientCertificateMode.NoCertificate,
                                             SslProtocols = System.Security.Authentication.SslProtocols.Tls12,
                                             ServerCertificate = certificate,
                                         };
                                         listenOptions.UseHttps(httpsConnectionAdapterOptions);
                                     });
                                 }
                             });
                    ////webBuilder.UseIIS();
                    webBuilder.UseStartup<Startup>();
                });```

i also declared both environment varibale **ASPNETCORE_Kestrel__Certificates__Development__Password** &&  **ASPNETCORE_Kestrel__Certificates__Development__Path** in the docker compose file.
Please help me to solve this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
MVC.Docs
  
Done
Linked pull requests

Successfully merging a pull request may close this issue.

None yet