Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use client cert to access secure private registry #1320

Closed
hobti01 opened this issue Feb 22, 2017 · 25 comments

Comments

@hobti01
Copy link

commented Feb 22, 2017

Expected behavior

docker login my.secure.registry
Username: myusername
Password:
Login Succeeded

Actual behavior

docker login my.secure.registry
Username: myusername
Password:
Error response from daemon: Get https://my.secure.registry/v1/users/: remote error: tls: handshake failure

Information

Diagnostic ID: 6A4673D8-43E5-4A49-8E31-CB17E771C8C3
Docker for Mac: version: 1.13.1 (94675c5a7)
macOS: version 10.12.3 (build: 16D32)
logs: /tmp/6A4673D8-43E5-4A49-8E31-CB17E771C8C3/20170222-153031.tar.gz
[OK] vmnetd
[OK] dns
[OK] driver.amd64-linux
[OK] virtualization VT-X
[OK] app
[OK] moby
[OK] system
[OK] moby-syslog
[OK] db
[OK] env
[OK] virtualization kern.hv_support
[OK] slirp
[OK] osxfs
[OK] moby-console
[OK] logs
[OK] docker-cli
[OK] menubar
[OK] disk

Steps to reproduce the behavior

  1. Implement docker registry protected by TLS on port 443 and requiring client certificate
  2. Add CA cert for registry to Mac OS keychain
  3. Attempt login

According to the Linux documentation the client cert and key can be stored in /etc/docker/certs.d and this is confirmed to work on Linux.

/etc/docker/certs.d/my.secure.registry/client.cert
/etc/docker/certs.d/my.secure.registry/client.key

With Docker for Mac it might be presumed (there is no documentation regarding client certificates for Docker for Mac) that ~/.docker could be used for the same purpose as /etc/docker but the following does not work:

~/.docker/certs.d/my.secure.registry/client.cert
~/.docker/certs.d/my.secure.registry/client.key

How can a client cert be used on Docker for Mac?

@ijc

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

Hi @hobti01 thank you for your report. Adding the key to the MacOS keychain is supposed to be sufficient according to https://docs.docker.com/docker-for-mac/faqs/#how-do-i-add-custom-ca-certificates, although I've not tried this myself.

According to the docs you should need a step 2b: restart docker 4 mac.

For step 2 what command did you use to add the cert to the keychain?

@hobti01

This comment has been minimized.

Copy link
Author

commented Feb 22, 2017

When you say "key", which key do you mean? I am explicitly asking about the client cert and key, not the server cert or ca cert. You have linked to the section on ca certificates, which is not relevant to client certificates. :(

Even if the client cert would be used from the keychain, there must be a method to specify which client cert to use to connect to this registry.

@ijc

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

Thanks for correcting me, I was indeed confusing server certs with client certs.

I've now raised a feature request for this internally.

@hobti01

This comment has been minimized.

Copy link
Author

commented Feb 23, 2017

Thank you! I'm happy to provide additional details. I have also created a related issue moby/moby#31295

@matthewbarr

This comment has been minimized.

Copy link

commented Feb 23, 2017

We actually need this as well, and would love to pull the client cert & key out of the keychain. Curl on the Mac actually implements this.

It might be possible to use the login subcommand to specify which client cert to use for a specific host.

For now, we actually copy the cert / key to a directory in the xhyve VM, via the git repo in ~/Library/Containers/com.docker.docker/Data/database/

and git reset --hard

then adding to : com.docker.driver.amd64-linux/etc/docker/certs.d/$HOSTNAME/

Just like you would on the linux system.

@ijc

This comment has been minimized.

Copy link
Contributor

commented Feb 23, 2017

@matthewbarr you would also need to git add + git commit those new files to make things visible I think?

@matthewbarr

This comment has been minimized.

Copy link

commented Feb 23, 2017

I didn't pull the full info in, sorry. Yes, you're correct. And then restart the xhyve VM, too.

We don't like storing private key's unencrypted on disk, but there's no other way for now.

We will probably start deleting the files after we do the commit, so they aren't left on disk in quite so obvious a manner.

@ijc

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2017

And then restart the xhyve VM, too.

It's supposed to autorestart when you do the git commit, is this not happening for you?

@hobti01

This comment has been minimized.

Copy link
Author

commented Feb 24, 2017

@matthewbarr Could you help me understand the steps for this workaround? I see that you've been part of the discussion in the forums where there are other workarounds, but not as you describe. I think you are doing something like this?

Create registry directories within your docker dir:
~/.docker/certs.d/my-registry.hostname.com:port

then:

VM_DIR=com.docker.driver.amd64-linux/etc/docker
cd ~/Library/Containers/com.docker.docker/Data/database
git reset --hard
mkdir -p ${VM_DIR}
rm -rf ${VM_DIR}/certs.d
cp -R -L ~/.docker/certs.d ${VM_DIR}
git add .
git commit -m "Add client certs"
rm -rf com.docker.driver.amd64-linux
git add .

echo "You must restart Docker!"

It seems to me that everything is left in more-or-less the state it was in before the manual addition, but it's unclear to me what the git status should be. Could this be improved?

@ijc25 git commit within a shell would cause a VM restart, but within a script did not restart the VM

@ijc

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2017

@hobti01 it turns out it depends on which path you touch, only certain paths are explicitly watched and cause a reboot. So for example:

$ cat ~/enable-docker-debug.sh 
#!/bin/bash
set -e


VM_DIR=com.docker.driver.amd64-linux/etc/docker
cd ~/Library/Containers/com.docker.docker/Data/database
git reset --hard
echo '{"debug": true}' > $VM_DIR/daemon.json
git add $VM_DIR/daemon.json
git commit -m "Enable engine debug"

Running that script does do a reboot. However you are correct, certs.d is not one of the watched paths and therefore doesn't cause a reboot so it needs to be done by hand, sorry for the earlier misinformation.

@matthewbarr

This comment has been minimized.

Copy link

commented Feb 24, 2017

Is there a safe file that can cause the reboot? or another safe way to ensure that the reboot happens? I'd like to be able to script that.

(And I'll post our new script soon.)

@ijc

This comment has been minimized.

Copy link
Contributor

commented Feb 24, 2017

I don't see anything which looks like it could be idempotently prodded to force a change. You could add/remove a blank line from the end of several files though, com.docker.driver.amd64-linux/etc/docker/daemon.json looks like it would be safe enough to do that to.

@matthewbarr

This comment has been minimized.

Copy link

commented Feb 24, 2017

We store our client certs in the keychain, and you can extract the public half easily. You can't get the key from the keychain, though. We happen to store it in ~/.certs, encrypted, with the password stored in the keychain, though.

If you have the cert / key pair somewhere else, you can just replace the middle code section.

#!/bin/bash
DOCKER_HOST=devprod.docker-dev.example.com
DOCKER_CERT_LOCATION=~/Library/Containers/com.docker.docker/Data/database/com.docker.driver.amd64-linux/etc/docker/certs.d/$DOCKER_HOST

cd ~/Library/Containers/com.docker.docker/Data/database/
git reset --hard

cd com.docker.driver.amd64-linux/etc/docker/
mkdir -p certs.d/$DOCKER_HOST


#COPY CERTS into ~/Library/Containers/com.docker.docker/Data/database/com.docker.driver.amd64-linux/etc/docker/....>
# they must be named foo.cert & foo.key,  no passwords.
# Retrieve the public half of the client cert from your keychain
security get-identity-preference -s '*.example.com' -p  > $DOCKER_CERT_LOCATION/user.cert

# Find latest private key in ~/.certs 
key=`ls -t  ~/.certs/*.key| head -n1`
echo $key

# find password for private key in keychain, save non-encrypted version to $DOCKER_CERT_LOCATION
security find-generic-password -w -s `basename $key .key` | openssl rsa -in $key -passin stdin -out $DOCKER_CERT_LOCATION/user.key


#add line to daemon.json to trigger an auto restart of docker VM, when committed to git.
echo >> daemon.json
git add daemon.json

git add certs.d/
git commit -m "add certs to docker"

There's no real reason to delete the directories from disk, unless you'd like to ensure they aren't stored on disk in an unencrypted form which can be found with a find ~ |grep key..
-- Which you might want to do, and I'll probably add soon, after I finish developing the user scripts.

@matthewbarr

This comment has been minimized.

Copy link

commented Feb 24, 2017

This is all work we're doing to roll out Docker for the Mac to Akamai developers. I'm going to probably try something similar for Windows, too, but that's another kettle of fish.

@hobti01

This comment has been minimized.

Copy link
Author

commented Feb 24, 2017

@matthewbarr Thanks for your script. A Docker for Windows solution would also be useful unless a cross-platform solution is implemented. I'm happy to use the workaround for now, but would like to see Docker for Mac enhanced to support this in a more natural way.

@noderunner

This comment has been minimized.

Copy link

commented Jun 2, 2017

The latest version of Edge: Docker Community Edition 17.06.0-rc1-ce-mac13, 2017-06-01 (edge), supposedly fixes this issue. Unfortunately, I haven't been able to find any documentation on how a user is supposed to configure client certs with this update. Does anyone know how this is supposed to work? It looks like the update is an inclusion of this other project: https://github.com/docker/docker-credential-helpers, but I can't find any examples on how I'm supposed to configure my client key/cert.

Update: It looks like the work-around method described earlier of adding your certs into the xhyve VM no longer works with this update to Edge. I'm gonna have to revert to Stable to have a working system for now.

@matthewbarr

This comment has been minimized.

Copy link

commented Jun 19, 2017

@ijc - any where we can look for documentation on how to test this feature, since it's in the RC's for 17.06? It's broken the existing functionality, and the RC's are moving along. I'm afraid y'all will release it without having a resolution, and we'll be stuck w/ developers downloading stable and not being able to work at all.

We actually don't use passwords on our docker registry, just the client cert which provides the username, which is going to be interesting, and possibly require some changes on our registry side. Docker login does require you to send a username/password, which means we'll have to strip the password out, somehow. I'd really like to make sure to get some testing and fix time before the 17.06 release, since it'll go to stable, vs edge.

@matthewbarr

This comment has been minimized.

Copy link

commented Jun 19, 2017

In particular: We need to know:

  1. How does one select which client cert should be sent a request?

  2. How can one debug what's being sent?

@matthewbarr

This comment has been minimized.

Copy link

commented Jun 19, 2017

In particular: We need to know:

  1. How does one select which client cert should be sent a request?

  2. How can one debug what's being sent?

@ebriney

This comment has been minimized.

Copy link
Member

commented Jun 19, 2017

@matthewbarr : with 17.06, you don't have to push your certs with git commands anymore, we copy
certificates from your mac folder ~/.docker/certs.d in the database when starting the app.

@matthewbarr

This comment has been minimized.

Copy link

commented Jun 19, 2017

Thanks! I'd actually intended to have it pull from the keychain, but that's not what this issue requested, so.. I'll go open a proper request for that :) This issue was the result of other conversation, which I think specified that, but.. fair is fair!

Thanks!

@londoncalling

This comment has been minimized.

Copy link

commented Jun 20, 2017

@matthewbarr

This comment has been minimized.

Copy link

commented Jun 20, 2017

Nope - this is all about the client certs. I'd hoped to be able to use the client certs stored in the keychain, but that's not what this issue said :) And it'd be much harder, since we don't have a good mechanism for specifying the host to use a specific keychain client cert.

I just wasn't expecting to actually being able to use the Mac .docker/certs.d directory. It's obvious, now, but after figuring out the various workarounds, I wasn't expecting it.

@londoncalling

This comment has been minimized.

Copy link

commented Jun 20, 2017

@matthewbarr Thanks, I'll add to the docs that this is now possible.

@guillaumerose

This comment has been minimized.

Copy link
Member

commented Mar 23, 2018

Closed in favor of #1757

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.