Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use letsencrypt certification generation #3041

Open
creekorful opened this issue Nov 2, 2019 · 32 comments
Open

Unable to use letsencrypt certification generation #3041

creekorful opened this issue Nov 2, 2019 · 32 comments

Comments

@creekorful
Copy link

Trying to use the registry with Letsencrypt builtin automatic certification generation fails with following error:

Error 403 - urn:acme:error:unauthorized - Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2

As Letsencrypt as announced here the account creation using API v1 is impossible since November 2019. It would be good to refactor registry to use the new ACMEv2 API to create account.

@codeclown
Copy link

In the meantime, anyone who needs to start a registry manually with Lets Encrypt... click Details:

Run certbot in standalone mode to generate certificates:

$ sudo certbot certonly --standalone

...

 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/example.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/example.com/privkey.pem

...

Files inside /etc/letsencrypt/live/example.com are symlinks! So you can't just mount the domain-specific folder when starting the container.

Instead you need to mount the whole letsencrypt-folder in order to keep the symlink paths intact. Relevant options for docker run:

  -e REGISTRY_HTTP_ADDR="0.0.0.0:443" \
  -v /etc/letsencrypt:/etc/letsencrypt \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/etc/letsencrypt/live/example.com/fullchain.pem \
  -e REGISTRY_HTTP_TLS_KEY=/etc/letsencrypt/live/example.com/privkey.pem \
  registry:2

Note that, as usual with certbot standalone, you should also remember to add a cronjob to run certbot renew automatically.

@cameron
Copy link

cameron commented Apr 18, 2020

Five months and not so much as peep re a documented feature that simply doesn't work as advertised? Boooo.

@nikandlv
Copy link

Got the same issue here

msg="register: acme: Error 403 - urn:acme:error:unauthorized - Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2 / RFC 8555. See https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 for details." 

@wilminator
Copy link

wilminator commented Jun 27, 2020

Edit- the shell command was bad. Changed from bash to sh. Also updated cp command to keep executable bit set.

I was able to get this to work on June 25, 2020. But I had to recompile the registry binary and update the image with it.

  • To recompile the registry executable, make a directory, change to it, and run this:
docker run --rm -v $(pwd):/mnt -it golang:alpine /bin/sh -c "apk update; apk upgrade; apk add --no-cache bash git openssh;go get github.com/docker/distribution/cmd/registry; mkdir /mnt/bin; cp -pv /go/bin/registry /mnt/bin"
  • To build a registry image that works, use this Dockerfile from the directory you created:
FROM registry:latest
COPY ./bin/registry /bin/registry

@wilminator
Copy link

For the devs, the probable quick fix here is to rebuild your 1.8 image with current golang libraries; I think that the library that enables Let's Encrypt has recently been updated to use the ACME 2.0 protocol and we just need to recompile against it.

@dorgan
Copy link

dorgan commented Jul 7, 2020

I was able to get this to work on June 25, 2020. But I had to recompile the registry binary and update the image with it.

  • To recompile the registry executable, make a directory, change to it, and run this:
mkdir fixme
cd fixme
docker run --rm -v $(pwd):/mnt -it golang:alpine /bin/bash -c "apk update; apk upgrade; apk add --no-cache bash git openssh;go get github.com/docker/distribution/cmd/registry; mkdir /mnt/bin; cp -v /go/bin/registry /mnt/bin"
  • To build a registry image that works, use this Dockerfile from the directory you created:
FROM registry:latest
COPY ./bin/registry /bin/registry

Trying to run that i get:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: "/bin/bash": stat /bin/bash: no such file or directory": unknown.

@robogeek
Copy link

Ditto

@wilminator
Copy link

@dorgan and @robogeek, I goofed the shell to use. I updated the docker command; it now uses /bin/sh, not /bin/bash. I verified that it works on Ubuntu 2004 on Windows, so it should behave itself on any *nix system now. You also don't need the additional mkdir and cd statements, as long as you don't mind a ./bin directory in your current dir when running the docker command.

@robogeek
Copy link

@wilminator I did not get the /bin/bash error. Instead I got the error at the top about ACME v1.

@wilminator
Copy link

@robogeek, I want to know if you tried using the stock registry image or if you tried my fix. Thanks.

@robogeek
Copy link

I used registry:latest

@locinus
Copy link

locinus commented Nov 27, 2020

To complete @wilminator answer: the docker container was unable to reach the DNS in my setting, so I added the --network host option, and it worked like a charm.

  • To recompile the registry executable, make a directory, change to it, and run this:
docker run --rm -v $(pwd):/mnt --network host -it golang:alpine /bin/sh -c "apk update; apk upgrade; apk add --no-cache bash git openssh;go get github.com/docker/distribution/cmd/registry; mkdir /mnt/bin; cp -pv /go/bin/registry /mnt/bin"
  • To build a registry image that works, use this Dockerfile from the directory you created:
FROM registry:latest
COPY ./bin/registry /bin/registry
  • I then ran
docker build --tag registry:fixed .

and used the registry:fixed image.

@luc122c
Copy link

luc122c commented Jan 26, 2021

Hello, I've just been affected by this. As discussed above, LetsEncrypt are deprecating ACMEv1. As part of the process they're running brown-outs. I just so happened to be setting up a private docker repository when they started one.

Come on Docker! Get upgraded to ACMEv2 already!

FATA[0000] create client: get directory at 'https://acme-v01.api.letsencrypt.org/directory': acme: Error 403 - urn:acme:error:serverInternal - ACMEv1 Brownout in Progress. ACMEv1 will fully turn off on June 1, 2021. Check https://letsencrypt.status.io/ for more details.

@oglematt
Copy link

Trying to implement the answer from @wilminator and I'm getting the following output. I know next to nothing about golang, so I'm not sure where to even begin. Any help would be greatly appreciated.

go: downloading github.com/docker/distribution v2.7.1+incompatible
go: downloading github.com/Azure/azure-sdk-for-go v0.2.0-beta
go: downloading github.com/Azure/azure-sdk-for-go v52.3.1+incompatible
go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d
go: downloading github.com/aws/aws-sdk-go v1.37.28
go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0
go: downloading github.com/bugsnag/bugsnag-go v1.9.0
go: downloading github.com/bugsnag/bugsnag-go v2.1.0+incompatible
go: downloading github.com/denverdino/aliyungo v0.0.0-20210222084345-ddfe3452f5e8
go: downloading github.com/docker/go-metrics v0.0.1
go: downloading github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7
go: downloading github.com/garyburd/redigo v1.6.2
go: downloading github.com/gorilla/handlers v1.5.1
go: downloading github.com/gorilla/mux v1.8.0
go: downloading github.com/mitchellh/mapstructure v1.4.1
go: downloading github.com/ncw/swift v1.0.53
go: downloading github.com/opencontainers/go-digest v1.0.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading github.com/spf13/cobra v1.1.3
go: downloading github.com/yvasiyarov/gorelic v0.0.7
go: downloading golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
go: downloading golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93
go: downloading google.golang.org/api v0.41.0
go: downloading google.golang.org/cloud v0.79.0
go: downloading google.golang.org/cloud/storage v1.14.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading rsc.io/letsencrypt v0.0.3
go get: google.golang.org/cloud@none updating to
        google.golang.org/cloud@v0.79.0: parsing go.mod:
        module declares its path as: cloud.google.com/go
                but was required as: google.golang.org/cloud
cp: can't stat '/go/bin/registry': No such file or directory

@wilminator
Copy link

wilminator commented Mar 11, 2021

Trying to implement the answer from @wilminator and I'm getting the following output. I know next to nothing about golang, so I'm not sure where to even begin. Any help would be greatly appreciated.

go: downloading github.com/docker/distribution v2.7.1+incompatible
go: downloading github.com/Azure/azure-sdk-for-go v0.2.0-beta
go: downloading github.com/Azure/azure-sdk-for-go v52.3.1+incompatible
go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d
go: downloading github.com/aws/aws-sdk-go v1.37.28
go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0
go: downloading github.com/bugsnag/bugsnag-go v1.9.0
go: downloading github.com/bugsnag/bugsnag-go v2.1.0+incompatible
go: downloading github.com/denverdino/aliyungo v0.0.0-20210222084345-ddfe3452f5e8
go: downloading github.com/docker/go-metrics v0.0.1
go: downloading github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7
go: downloading github.com/garyburd/redigo v1.6.2
go: downloading github.com/gorilla/handlers v1.5.1
go: downloading github.com/gorilla/mux v1.8.0
go: downloading github.com/mitchellh/mapstructure v1.4.1
go: downloading github.com/ncw/swift v1.0.53
go: downloading github.com/opencontainers/go-digest v1.0.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading github.com/spf13/cobra v1.1.3
go: downloading github.com/yvasiyarov/gorelic v0.0.7
go: downloading golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
go: downloading golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93
go: downloading google.golang.org/api v0.41.0
go: downloading google.golang.org/cloud v0.79.0
go: downloading google.golang.org/cloud/storage v1.14.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading rsc.io/letsencrypt v0.0.3
go get: google.golang.org/cloud@none updating to
        google.golang.org/cloud@v0.79.0: parsing go.mod:
        module declares its path as: cloud.google.com/go
                but was required as: google.golang.org/cloud
cp: can't stat '/go/bin/registry': No such file or directory

@oglematt Truth be told, I'm not a golang guy either; I just make the computers work ;)

It looks like an upstream Google cloud library was updated 4 hours before your post google-cloud-go and they renamed the library. Since the go.mod configuration file for registry still uses the old name, go panics because it downloaded a module that has a different name than what it was expecting.

The devs for this project probably need to fix their requirements file to use the new name of the Google Cloud module. I have submitted a ticket #3377, and if I have time will try to offer a patch to fix it.

@wilminator
Copy link

Hello,
With some help from my issue respondents (@milosgajdos and @wy65701436), I was able to get a working registry executable built that supports ACME v2.

docker run --rm -v "C:\Users\me\docreg:/mnt" -it golang:alpine /bin/sh -c "apk update&& apk upgrade&& apk add --no-cache bash git openssh make&& cd /opt&& git clone https://github.com/docker/distribution&&cd /opt/distribution&& make binaries&& mkdir /mnt/bin; cp -v bin/registry /mnt/bin"

Does this work for you?

@oglematt
Copy link

@wilminator That does! Thanks for tracking it down.

@Jean-Daniel
Copy link

18 months and still no fix for the stock image. Is this project dead ?

@gsaraf
Copy link

gsaraf commented May 20, 2021

This is affecting me as well. What can I do to help move this forward?

@wilminator
Copy link

Everyone, I have an answer. TL,DR: If we want Docker to fix their registry image, this is the wrong place to post this. Go to registry incompatible with Let's Encrypt #96 and post there.

I did some more digging because this project has quite a few recent commits. Also, the one-liner above to launch a container to compile the registry binary shows that the project itself works fine- we get a working binary.

The problem most of us are facing is that the Docker registry image has a two year old version of the registry binary baked in, and this repo does not control that binary.

@gsaraf To fix this we need to use an issue with the main Docker GitHub repo and propose a merge into docker/distribution-library-image and hope they compile that down into a new image for us. The location this updated binary needs to get stashed is at https://github.com/docker/distribution-library-image/tree/master/amd64. However, this is the "wrong" way to fix this. The reality is that they should recompile their own source tree to fix this issue; it looks like we need to prod them to do it.

See registry incompatible with Let's Encrypt #96 and upvote like is suggested in the comments. We can also clone the repo, run their update, confirm their updated binary is ACME v2 compliant, and submit a merge request. However, I'm not holding my breath to see if their source is v2 compatible without any hacking.

@Jean-Daniel
Copy link

A better solution would be to stop relying on this link and docker official account and just published an updated image in a new official distribution account in docker hub.

While it is convenient to just have to use image registry:2, I don't mind switching to distribution/registry:2 if I can get an up-to-date image out of the box.

@Bec-k
Copy link

Bec-k commented Oct 11, 2022

Still not fixed? 3 years have passed...

@milosgajdos
Copy link
Member

Would you like to open a PR @denissabramovs ?

@Bec-k
Copy link

Bec-k commented Oct 11, 2022

Nope, why would i? I just installed certbot myself, issued certs through it and configured crontab. Problem solved on my end.
It took me an hour, but it seems you can't fix it for 3 years...

@Jean-Daniel
Copy link

Jean-Daniel commented Oct 13, 2022

Would you like to open a PR @denissabramovs ?

I don't think it is possible to open a PR that performs a clean compilation of the project using a recent go toolchain and publish it…

This is all that is required to fix this issue.

@milosgajdos
Copy link
Member

What are you missing in the toolchain that prevents you from doing this work @Jean-Daniel?

@Jean-Daniel
Copy link

Jean-Daniel commented Oct 13, 2022

What I am saying is that only the project maintainers can decide to release a new version and publish it (which is all that is needed to fix this issue, as go 1.18 is compatible with recent acme protocol). No PR can do that for you.

The last release was built with Go 1.16, which is why it fails to generate certificate.

On my side, that make a long time that I did it myself and published it in a public repository dedicated to that single task.

@tianon

This comment was marked as duplicate.

@milosgajdos
Copy link
Member

@Jean-Daniel these are all fair comments. I've been trying to attend to various issues and PRs, but I agree we need more activity from maintainers. I'll be raising my voice more strongly at the next community call -- can't do more as is at the moment :-(

@olakmal
Copy link

olakmal commented Jan 23, 2023

Damn still no solution for this? 😥

@bhhaskin
Copy link

bhhaskin commented Oct 9, 2023

Still an issue

@milosgajdos
Copy link
Member

@creekorful with the new release cut last night https://github.com/distribution/distribution/releases/tag/v3.0.0-alpha.1 would you mind taking it for a spin and test it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests