New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker authorization plug-in infrastructure #15365

Merged
merged 6 commits into from Dec 12, 2015

Conversation

Projects
None yet
@liron-l
Contributor

liron-l commented Aug 6, 2015

REMARK: Code is intended to be complementary to the docker (authZ discussion) [https://github.com/moby/moby/issues/14674]. There are several aspects of the proposed code which require additional work and testing. Please use this commit as a low level design proposal.

Docker authorization plug-in infrastructure enables extending the functionality of the Docker daemon with respect to user authorization.
The infrastructure enables registering a set of external authorization
plug-in. Each plug-in receives information about the user and the
request and decides whether to allow or deny the request. Only in case
all plug-ins allow accessing the resource the access is granted.

Each plug-in operates as a separate service, and registers with Docker
through general (plug-ins API)
[https://blog.docker.com/2015/06/extending-docker-with-plugins/]. No
Docker daemon recompilation is required in order to add / remove an
authentication plug-in. Each plug-in is notified twice for each
operation: 1) before the operation is performed and, 2) before the
response is returned to the client. The plug-ins can modify the response
that is returned to the client.

The authorization depends on the authorization effort that takes place
in parallel [https://github.com/moby/moby/issues/13697].

This is the official issue of the authorization effort:
#14674

(Here)[https://github.com/rhatdan/docker-rbac] you can find an open
document that discusses a default RBAC plug-in for Docker.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Aug 6, 2015

Member

cc @icecrime @crosbymichael @NathanMcCauley @diogomonica perhaps you can have a look at this? General API description can be found in this document

Member

thaJeztah commented Aug 6, 2015

cc @icecrime @crosbymichael @NathanMcCauley @diogomonica perhaps you can have a look at this? General API description can be found in this document

@NathanMcCauley

This comment has been minimized.

Show comment
Hide comment
@NathanMcCauley

NathanMcCauley Aug 10, 2015

Contributor

Should there any limitations to what the Response rewriters can and cannot do? It looks like the proposal allows arbitrary modifications? I'm concerned that'll mean that we end up with accidental dependencies on the structure of the Docker Remote API request/response structure as 3rd party authz plugins now depend on being able to directly parse (and modify) the response bodies.

Contributor

NathanMcCauley commented Aug 10, 2015

Should there any limitations to what the Response rewriters can and cannot do? It looks like the proposal allows arbitrary modifications? I'm concerned that'll mean that we end up with accidental dependencies on the structure of the Docker Remote API request/response structure as 3rd party authz plugins now depend on being able to directly parse (and modify) the response bodies.

@NathanMcCauley

This comment has been minimized.

Show comment
Hide comment
@NathanMcCauley

NathanMcCauley Aug 10, 2015

Contributor

How will this handle cases where data is streamed live back to the client, as in the case of the /logs endpoint. How will it deal with attach?

Contributor

NathanMcCauley commented Aug 10, 2015

How will this handle cases where data is streamed live back to the client, as in the case of the /logs endpoint. How will it deal with attach?

@liron-l

This comment has been minimized.

Show comment
Hide comment
@liron-l

liron-l Aug 11, 2015

Contributor

Added unit tests and fixed implementation according to comments.

Contributor

liron-l commented Aug 11, 2015

Added unit tests and fixed implementation according to comments.

@liron-l

This comment has been minimized.

Show comment
Hide comment
@liron-l

liron-l Aug 19, 2015

Contributor

@icecrime @crosbymichael @NathanMcCauley @diogomonica
Added detailed design document for the authorization feature here.

Contributor

liron-l commented Aug 19, 2015

@icecrime @crosbymichael @NathanMcCauley @diogomonica
Added detailed design document for the authorization feature here.

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Aug 21, 2015

Contributor

How does this work w PAM, it would be nice if it was easy for people to understand and familiar the design that way

Contributor

jessfraz commented Aug 21, 2015

How does this work w PAM, it would be nice if it was easy for people to understand and familiar the design that way

@liron-l

This comment has been minimized.

Show comment
Hide comment
@liron-l

liron-l Aug 23, 2015

Contributor

Thanks for the review @diogomonica, @NathanMcCauley, @endophage,

We've updated the design doc and answered all comments, looking forward to discuss further and agree on the right approach for plugin authorization.
I expect that in the majority of scenarios, plugins will not be required to perform any additional queries to the daemon. However, we should find a proper mechanism for such scenarios (e.g., for plugins that need to scan the content of an image to decide whether permission should be granted or not).
Aggregating the feedback, I'd like to propose three possible solutions:

  1. Capability based solution -
    In the daemon level, each plugin will be configured with a dedicated policy for the command it can invoke (e.g., pull, inspect)
  2. Authorization policy based solution -
    Admin sets up a dedicated user for the plugin and defines a policy for this user. The plugin will be authenticated and authorized as a regular user.
  3. Sending required metadata -
    For example, send all image info for container run command.
    I think we can rule out this solution since this can lead to sending a large chunk of redundant data. For example, assume that some flow requires examining the image history, which leads to sending the entire history for each command related to images, a scenario which we would probably like to avoid.

I personally think both solution 1 and 2 are plausible, but the former (1) will require much more logic in the daemon core, and will have to be maintained anytime new commands are added (which can have some maintenance pain).

What do you think?

Contributor

liron-l commented Aug 23, 2015

Thanks for the review @diogomonica, @NathanMcCauley, @endophage,

We've updated the design doc and answered all comments, looking forward to discuss further and agree on the right approach for plugin authorization.
I expect that in the majority of scenarios, plugins will not be required to perform any additional queries to the daemon. However, we should find a proper mechanism for such scenarios (e.g., for plugins that need to scan the content of an image to decide whether permission should be granted or not).
Aggregating the feedback, I'd like to propose three possible solutions:

  1. Capability based solution -
    In the daemon level, each plugin will be configured with a dedicated policy for the command it can invoke (e.g., pull, inspect)
  2. Authorization policy based solution -
    Admin sets up a dedicated user for the plugin and defines a policy for this user. The plugin will be authenticated and authorized as a regular user.
  3. Sending required metadata -
    For example, send all image info for container run command.
    I think we can rule out this solution since this can lead to sending a large chunk of redundant data. For example, assume that some flow requires examining the image history, which leads to sending the entire history for each command related to images, a scenario which we would probably like to avoid.

I personally think both solution 1 and 2 are plausible, but the former (1) will require much more logic in the daemon core, and will have to be maintained anytime new commands are added (which can have some maintenance pain).

What do you think?

@lioryanko

This comment has been minimized.

Show comment
Hide comment
@lioryanko

lioryanko Aug 23, 2015

@jfrazelle PAM is one of the mechanisms which can be used for authentication by plugins which will be built on top of the authentication framework (#13697) that is currently under development. Authorization plugins will depend upon the authentication data to achieve access control with greater granularity than the current dichotomy of full access or no access.
We'd love you to ask further questions and give suggestions regarding PAM and authentication in general at #13697 or straight at the design and UX doc

lioryanko commented Aug 23, 2015

@jfrazelle PAM is one of the mechanisms which can be used for authentication by plugins which will be built on top of the authentication framework (#13697) that is currently under development. Authorization plugins will depend upon the authentication data to achieve access control with greater granularity than the current dichotomy of full access or no access.
We'd love you to ask further questions and give suggestions regarding PAM and authentication in general at #13697 or straight at the design and UX doc

@mheon

This comment has been minimized.

Show comment
Hide comment
@mheon

mheon Aug 26, 2015

Contributor

@liron-l Looking at the options you've laid out, I would tend to prefer the third from an architectural perspective, as it does not require additional requests to the public API. However, I can certainly understand how this might massively increase overhead, particularly in cases dealing with images. Option one would be a good alternative, assuming we have a reliable way of authenticating plugins (perhaps they could declare a public key in the plugin specification?). I really don't like the second option - it seems the sort of thing where it would be very easy for a user to shoot themselves in the foot if they had multiple authorization plugins.

As a possible alternative, perhaps plugins could tell the daemon they required more information to make a decision, and the daemon could reissue the authorization request with additional information on the image or container in question. This would significantly increase the complexity of authorization responded and the complexity of the daemon code, but would remove the need for additional requests from the authorization plugin to the public API.

Contributor

mheon commented Aug 26, 2015

@liron-l Looking at the options you've laid out, I would tend to prefer the third from an architectural perspective, as it does not require additional requests to the public API. However, I can certainly understand how this might massively increase overhead, particularly in cases dealing with images. Option one would be a good alternative, assuming we have a reliable way of authenticating plugins (perhaps they could declare a public key in the plugin specification?). I really don't like the second option - it seems the sort of thing where it would be very easy for a user to shoot themselves in the foot if they had multiple authorization plugins.

As a possible alternative, perhaps plugins could tell the daemon they required more information to make a decision, and the daemon could reissue the authorization request with additional information on the image or container in question. This would significantly increase the complexity of authorization responded and the complexity of the daemon code, but would remove the need for additional requests from the authorization plugin to the public API.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah
Member

thaJeztah commented Oct 5, 2015

@calavera

This comment has been minimized.

Show comment
Hide comment
@calavera

calavera Oct 16, 2015

Contributor

I'd rather avoid premature optimizations. That will allow us to get this in sooner rather than later. If we don't need cross references with the daemon, let's not add them from now.

From a design point of view, this change looks good to me, but I'll wait for other maintainers to confirm.
From a code point of view, there have been a lot of changes in the server package that conflict with this. We can probably reduce the conflicts transforming this code in a middleware like we've done with other pre processing code in the server:

https://github.com/docker/docker/blob/master/api/server/middleware.go

Contributor

calavera commented Oct 16, 2015

I'd rather avoid premature optimizations. That will allow us to get this in sooner rather than later. If we don't need cross references with the daemon, let's not add them from now.

From a design point of view, this change looks good to me, but I'll wait for other maintainers to confirm.
From a code point of view, there have been a lot of changes in the server package that conflict with this. We can probably reduce the conflicts transforming this code in a middleware like we've done with other pre processing code in the server:

https://github.com/docker/docker/blob/master/api/server/middleware.go

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Oct 16, 2015

Contributor

I agree with @calavera and I think using middleware for this would be great.

Contributor

jessfraz commented Oct 16, 2015

I agree with @calavera and I think using middleware for this would be great.

@liron-l

This comment has been minimized.

Show comment
Hide comment
@liron-l

liron-l Oct 24, 2015

Contributor

@calavera, @jfrazelle thanks for the feedback and sounds good and much more appropriate moving the authZ plugin code to the middleware layer.
Any additional feedback about design/implemenation we should consider?

Contributor

liron-l commented Oct 24, 2015

@calavera, @jfrazelle thanks for the feedback and sounds good and much more appropriate moving the authZ plugin code to the middleware layer.
Any additional feedback about design/implemenation we should consider?

@calavera

This comment has been minimized.

Show comment
Hide comment
@calavera

calavera Oct 26, 2015

Contributor

@NathanMcCauley any concerns about the design?

Contributor

calavera commented Oct 26, 2015

@NathanMcCauley any concerns about the design?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 11, 2015

Member

@liron-l can you change the flag to --authz-plugin to be more consistent with other flags? I discussed this internally, also with @moxiegirl, and if that's just a find & replace, we can merge this after! \o/

Member

thaJeztah commented Dec 11, 2015

@liron-l can you change the flag to --authz-plugin to be more consistent with other flags? I discussed this internally, also with @moxiegirl, and if that's just a find & replace, we can merge this after! \o/

Change authz plugin argument name
Signed-off-by: Liron Levin <liron@twistlock.com>
@liron-l

This comment has been minimized.

Show comment
Hide comment
@liron-l

liron-l Dec 11, 2015

Contributor

Thanks @thaJeztah, we've replaced all occurrences of the authz argument.

Contributor

liron-l commented Dec 11, 2015

Thanks @thaJeztah, we've replaced all occurrences of the authz argument.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 11, 2015

Member

Thanks @liron-l, awesome!!

Member

thaJeztah commented Dec 11, 2015

Thanks @liron-l, awesome!!

runcom added a commit that referenced this pull request Dec 12, 2015

Merge pull request #15365 from twistlock/14674-docker-authz
Docker authorization plug-in infrastructure

@runcom runcom merged commit 1fffc02 into moby:master Dec 12, 2015

5 of 6 checks passed

windows Jenkins build Windows-PRs 18608 is running
Details
docker/dco-signed All commits signed
Details
documentation success 2 tests run, 0 skipped, 0 failed.
Details
experimental Jenkins build Docker-PRs-experimental 12114 has succeeded
Details
janky Jenkins build Docker-PRs 20888 has succeeded
Details
userns Jenkins build Docker-PRs-userns 3361 has succeeded
Details
@cyphar

This comment has been minimized.

Show comment
Hide comment
@cyphar

cyphar Dec 12, 2015

Contributor

Awesome. Great to finally see some form of ACL support in Docker. It's been a long time coming. 😸

Contributor

cyphar commented Dec 12, 2015

Awesome. Great to finally see some form of ACL support in Docker. It's been a long time coming. 😸

@moxiegirl

This comment has been minimized.

Show comment
Hide comment
@moxiegirl

moxiegirl Dec 13, 2015

Contributor

Thanks @thaJeztah ... sorry to be MIA, I'll take a look at the pr and carry any changes that might be necessary. I'll ping @liron-l if that is actually necessary.

Contributor

moxiegirl commented Dec 13, 2015

Thanks @thaJeztah ... sorry to be MIA, I'll take a look at the pr and carry any changes that might be necessary. I'll ping @liron-l if that is actually necessary.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 14, 2015

Member

just realized this also needs changes to the completion scripts. ping @albers @sdurrheimer if you have time to look at those? Otherwise I can see if there's a maintainer able to look at them ❤️

Member

thaJeztah commented Dec 14, 2015

just realized this also needs changes to the completion scripts. ping @albers @sdurrheimer if you have time to look at those? Otherwise I can see if there's a maintainer able to look at them ❤️

@albers

This comment has been minimized.

Show comment
Hide comment
@albers

albers Dec 14, 2015

Member

Sure, I'll take care of bash completion. Thanks for pinging me.

Member

albers commented Dec 14, 2015

Sure, I'll take care of bash completion. Thanks for pinging me.

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Dec 14, 2015

Contributor

@albers you da bomb!

On Mon, Dec 14, 2015 at 5:19 AM, Harald Albers notifications@github.com
wrote:

Sure, I'll take care of bash completion. Thanks for pinging me.


Reply to this email directly or view it on GitHub
#15365 (comment).

Contributor

jessfraz commented Dec 14, 2015

@albers you da bomb!

On Mon, Dec 14, 2015 at 5:19 AM, Harald Albers notifications@github.com
wrote:

Sure, I'll take care of bash completion. Thanks for pinging me.


Reply to this email directly or view it on GitHub
#15365 (comment).

@sdurrheimer

This comment has been minimized.

Show comment
Hide comment
@sdurrheimer

sdurrheimer Dec 14, 2015

Contributor

So do I with zsh completion.

If I understand well, there is only a single new option --authz-plugin for docker daemon, right ?

Contributor

sdurrheimer commented Dec 14, 2015

So do I with zsh completion.

If I understand well, there is only a single new option --authz-plugin for docker daemon, right ?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 14, 2015

Member

@sdurrheimer correct! And to use @jfrazelle's words; you're also da bomb! thanks so much

Member

thaJeztah commented Dec 14, 2015

@sdurrheimer correct! And to use @jfrazelle's words; you're also da bomb! thanks so much

@doronp

This comment has been minimized.

Show comment
Hide comment
@doronp

doronp Dec 15, 2015

Contributor

Been following this one. Awesome job.
Just a clarification - Did you ever mean that two users will not be aware about each others stuff if need be - i.e. - two users can have 2 container with the SAME name on the same engine (otherwise the later one gets the error about name exists and hence information he should not be getting).

Contributor

doronp commented Dec 15, 2015

Been following this one. Awesome job.
Just a clarification - Did you ever mean that two users will not be aware about each others stuff if need be - i.e. - two users can have 2 container with the SAME name on the same engine (otherwise the later one gets the error about name exists and hence information he should not be getting).

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 15, 2015

Member

@doronp no, that still won't be possible with these changes. Container names need to be unique per daemon, however, an Authz plugin would probably be able to prevent a user from connecting to a network that is not owner by that user, or use an image that's not allowed to be used by that user. There's lots of things involved in a multi-tenant setup, and this can help, but won't solve all.

Member

thaJeztah commented Dec 15, 2015

@doronp no, that still won't be possible with these changes. Container names need to be unique per daemon, however, an Authz plugin would probably be able to prevent a user from connecting to a network that is not owner by that user, or use an image that's not allowed to be used by that user. There's lots of things involved in a multi-tenant setup, and this can help, but won't solve all.

@boucher boucher referenced this pull request Dec 16, 2015

Closed

Cr combined #4

@doronp

This comment has been minimized.

Show comment
Hide comment
@doronp

doronp Dec 16, 2015

Contributor

It could also concatenate a unique ID such as user ID to the container name (modify request) and filter it out (modify response) in every response for example.
Not sure it the best path to go though.
I'd be great to see even a preliminary mapping of what is missing sometime.
WDYT?

Contributor

doronp commented Dec 16, 2015

It could also concatenate a unique ID such as user ID to the container name (modify request) and filter it out (modify response) in every response for example.
Not sure it the best path to go though.
I'd be great to see even a preliminary mapping of what is missing sometime.
WDYT?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 16, 2015

Member

@doronp yes, that would partly solve the problem, but may be quite tedious, and won't be a full solution; containers and images can be accessed by ID as well. Also think of (e.g.) volumes, and bind-mounted directories, people running docker-in-docker.

This PR offers the option to develop plugins to limit access, but for multi-tenant situations there are still lots of things to tackle. Writing up a list of things that may be needed to cover that can be a start; if you have ideas, feel free to open an issue (there may already be issues in that area)

Member

thaJeztah commented Dec 16, 2015

@doronp yes, that would partly solve the problem, but may be quite tedious, and won't be a full solution; containers and images can be accessed by ID as well. Also think of (e.g.) volumes, and bind-mounted directories, people running docker-in-docker.

This PR offers the option to develop plugins to limit access, but for multi-tenant situations there are still lots of things to tackle. Writing up a list of things that may be needed to cover that can be a start; if you have ideas, feel free to open an issue (there may already be issues in that area)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment