Adding Kerberos support to Docker #13697
Comments
|
@dimastopel is the ticket sent to the daemon via an http header? |
|
@duglin Yes, HTTP Authorization header |
|
@dimastopel you may want to look at how the config file allows you to specify additional http headers for all CLI->Daemon flows: https://github.com/docker/docker/blob/master/cliconfig/config.go#L43 Ideally, we'd like to have a CLI plugin do that and have |
|
@dimastopel this is a good direction to consider, I'd be interested in more elaboration on the technical design behind this. Especially interested in the daemon side handling of the ticket and access control that would be based on the ticket there. |
|
@NathanMcCauley sure thing. We are working on this and I'll share a more elaborated story in the next couple of days. @duglin thanks. Will check this out. |
|
Below is a more technical elaboration on how I see each component will operate. It doesn't contain the integration points yet. Client In case
In case of any error during the process, the client exits with the appropriate error code and message. Example request header with the token: Daemon Upon receiving the request the server extracts the TGS from the HTTP Authorization header and validates it. No communication with KDC is required to validate the TGS. Once the TGS is validated, and the User Principal Name (UPN) is extracted the authentication process completes. In case the authentication process fails an HTTP 401 code is returned to the client. In case the SPN was incorrect HTTP 403 will be returned. Kerberos authentication requires one hop between the client and the daemon. Access Control We use authentication both for auditing but also to provide the ability to create policies that control user access. For example, limit each developer to 5 containers max on the specific server. See below an example screen for configuring such policy. I propose a thin framework that will enable adding external modules via Docker HTTP API. The modules are external services that expose HTTP REST API. The framework will enable registering to system events, so that each module will be able to register for the relevant event groups. However the framework will also enable the modules to control the flow of the system via return codes. This implies that the calls to the modules will be blocking unless (configurable) timeout occurs. In our case we will register for user authenticated event and check against the policies we have whether this specific user is authorized to perform the specific task. If we see that the user is not allowed to do the task we need to be able to tell the daemon to stop processing the request and provide the appropriate reason. I didn't investigate yet to what extent the current events infra can be used to achieve this. I guess the current events infra is very relevant. Conceptually, this is also close to what @duglin suggests for CLI. I think the line between the platform (Docker) and an external partner (e.g., Twistlock) should be this very framework. The framework is part of the platform but the modules are supplied by the external partner. As part of the partner's installation process the module adds itself via Docker's daemon (future) API. Last note on this. In my opinion, we should differentiate between Kerberos authentication support and access control, and concentrate first on Kerberos authentication only. Once we have this we would be glad to go further with Access Control support. We would be glad to be the once to drive and implement both. Your comments on this are very welcome. |
|
I realise this is less important for the global concept of this proposal, but I would prefer to use a more generic flag to pass the options, similar to the way options for logging and graph-drivers are passed. That way other authentication mechanisms can be added in future without introducing new flags. Something like; (perhaps even
Fully agree on that; access control should not be tied to authentication here. I ❤️ the looks of that screenshot though! |
|
@thaJeztah - fully agree with your proposal re flags. If there are no major objections to what I have proposed I’m going to draft a pull request with the initial implementation and continue the discussion from there. If there are objections please let me know ASAP. |
Purely based on the discussion above, I don't think there are objections (but please don't hold that against me :)). Creating a draft pull-request usually works well, as that gives the maintainers a bit more insight in the proposed changes. |
|
@dimastopel Have you considered what it would look like for this notion of pluggable authentication to be implemented as integration with PAM? Then, a bunch of the existing work on different PAM backends could be utilized by the Docker daemon to allow varied methods of authentication (LDAP, kerberos, OTP, etc). |
|
@NathanMcCauley Let me look into it. |
|
My guess is that, if we were to call into PAM at all, it would be for implementing the password checking part of the Basic auth scheme at the server. |
|
Sorry for the late reply on this After researching this for some time I agree that PAM looks as a great way to get "pluggable" behavior with a relatively low effort. In addition, numerous existing PAM modules can be used OOB. It also became clear to me that adding generic PAM support to Docker is a superior task than adding a vanilla Kerberos support. Specifically with Kerberos, PAM might not provide a good solution to the Single Sign On (SSO) functionality, but this issue can be addressed after PAM is in place. This module, for example, claims it solves this issue OOB. I didn't verify it yet, though. Unless there are objections we will work on the general PAM support and then move to Kerberos. As far as I understand we can get support for more than just basic AuthN in case PAM is in place. E.g., OTP, LDAP, Challenge-Response with Yubico, etc. What do I miss? |
|
ping @icecrime to check if this lines up with the roadmap Thanks in advance, @dimastopel ! |
|
@dimastopel PAM's generally only going to be of use when you want to ask for information that can be directly input by a user, such as a password which can be used for obtaining Kerberos credentials, and Basic auth more or less works for that case. The PAM model allows modules to prompt and ask other questions, too, which is how it can support things like OTP. When modules start asking for things other than passwords, or asking for more than one piece of information, the limitations of what you can communicate between the client and the server using the Basic scheme start creating problems -- a server implementation that just supplies the user's Basic password whenever modules ask for information during authentication is likely going to be providing the wrong answer to some of those modules. Going by OpenSSH as an example, full PAM support, which avoids that sort of problem, is only available when using "keyboard-interactive" authentication, while "password" authentication's use of PAM is there as a best-effort for the sake of clients that don't implement "keyboard-interactive". When authentication is being done using a binary wire protocol like the one used in GSSAPI and/or Negotiate, or using public keys, PAM isn't used for the authentication step, and the daemon only calls into PAM's account management function to do an authorization check. In #13994, which is linked here because it overlaps somewhat, I've been focusing on making client- and server-side implementations of auth schemes pluggable, in a way that would allow the server's Basic implementation to be swapped out, or to have more than one available, calling into all of the built plugins which don't disable themselves until one of them succeeds. On a second branch that builds on that, there's a Basic server implementation that uses libsasl2's sasl_checkpass() function. The libsasl2 library can be configured to send the user names and passwords off to saslauthd, and saslauthd can be configured to use PAM to check them. An implementation that calls directly into libpam could be slotted in next to it. |
|
Nothing conflicting with the roadmap from my point of view. I'll leave it to @NathanMcCauley and @diogomonica to see how they want to move forward with this. |
|
@nalind thanks for the reply! Can you please elaborate on "a server implementation that just supplies the user's Basic password whenever modules ask for information during authentication is likely going to be providing the wrong answer to some of those modules."? Not sure I understand why server supplies authn information and why wrong answers will be provided. Regarding text based authentication, as far as I know, PAM supports binary messages and client side modules that enables quite rich customization of the authentication process. As I understand, your solution provides a plug-in based approach for the authentication of the Docker client against the Docker daemon. Is this correct? PAM provides something that is quite similar. Do you see your solution as an alternative to PAM? If not, how do you see the two solutions working together? |
|
@dimastopel The traditional strategy people have used when trying to support PAM in implementations of protocols like FTP is to call into PAM with a prompter callback that walks the list of passed-in prompts and hands back the user name when the msg_style is PAM_PROMPT_ECHO_ON, and to hands back the password when the msg_style is PAM_PROMPT_ECHO_OFF. You run into weird behavior and bug reports if the dialog needs to be more complicated than that. If the prompt text is required in order for the user to supply the right information (I'm thinking of challenge-response OTP schemes here), and it isn't displayed, or if the user is asked for information before the PAM conversation even starts, it can't work. That's a risk when trying to fit something like PAM into a wire protocol that can't be as flexible as PAM expects it to be. OpenSSH ended up needing to implement keyboard-interactive authentication to be able to properly handle the cases where its password authentication scheme wasn't flexible enough. The Linux-PAM binary prompt interface isn't widely adopted. Outside of the Linux-PAM tree itself, I see one module (the Openwall pam_userpass module) making use of the constants defined as part of the interface, and no references to it in the source trees of the packages in Fedora that link to libpam. Granted, that's not exhaustive. The patch I'm proposing makes things pluggable at the level of client and server implementations of authentication schemes - client and server implementations for Basic and Negotiate (or others, though I haven't tried to implement others) can be enabled (or not) at build-time, and the internal interface allows them to disable themselves at startup (for example, if the Negotate server discovers that it has no creds, so that client's won't be told that they can try Negotiate when the server is unable to accept it). The server portion of Basic would be where I'd expect to be calling into libpam. Negotiate, for example, would call into a GSSAPI library. My personal leaning toward using libsasl2 for the server portion of Basic is based partly on wanting to avoid having to implement that sort of logic in a conversation callback, partly on liking that the process separation we get by having libsasl2 call out to the external saslauthd process reduces possible side-effects, and partly on saslauthd offering a caching option. |
|
@nalind Makes sense. Several questions:
|
|
@dimastopel Thanks for suffering through my wall of text! Using libsasl2's GSSAPI support would require some additional pipe-fitting -- to start, we'd have to map from HTTP authentication schemes to SASL mechanisms. While that part looks pretty straightforward for Basic and Negotiate, I don't have a good plan for what we'd do for handling Bearer or OAuth, as there doesn't appear to be a simple mapping between the two sets (going off of http://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml and https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xml). The SASL GSSAPI mechanism specifies Kerberos as the only supported GSSAPI mechanism, while Negotiate specifies SPNEGO, and that could create difficulties if the server side is, or ever becomes, clever enough about validating incoming data. You read correctly, saslauthd can call into PAM to check the password, though you have the same limitations there that you'd have when calling into PAM directly - the protocol between libsasl2 and saslauthd is a single round trip (the library sends the user, password, SASL realm, and service name, and receives either an "OK" response or an error message), but I hope it's clearer to . If/when we disable mechanisms at build-time is more of a question of how we're handling library dependencies - if we're trying to gracefully handle the absence of some libraries at build-time, then we can have the build scripts check for the presence of libraries using pkg-config, and set build tags to build things conditionally. If we're not, and we can straight-up require libraries to be present to build, we can build everything in. Either way, the various schemes can check the run-time configuration and either not register themselves at startup, or not participate on a per-request basis. We don't know whether or not libsasl2 and saslauthd are even set up at run-time, and offering a scheme that can't possibly succeed is going to annoy clients (I'm thinking particularly about prompting for passwords, only to have the password check fail every time), so we'd a runtime option regardless. |
|
@nalind Thanks PAM limitations are clear. What was important for me to verify is that customer with an existing PAM infra (or who just prefers using PAM) will be able to use it with Docker+libsasl2. And as far as I understand the answer is yes. I think that we can figure out along the way how to deal with issues like SASL + Bearer, OAuth. Trying to take a bit more pragmatic approach here: @lioryanko from our team tried to build nalind/docker/daemonauth so we could play with it, identify the gaps, and try to build a mutual plan on closing those gaps. What we see is that some of the Basically I think that we need to start with two chunks: 1) add pluggable authN framework with (indirect) PAM support. 2) Add support for Negotiate. Then we can continue with OAuth etc. WDYT? |
|
Right, the |
|
@nalind Your code looks good. I'm trying to build it now. I'm having a bit of trouble building your daemonauth branch (without excluding the server's authenticator modules). When trying to build the way you said (from inside docker's build environment container, built using the Dockerfile which includes your modifications) I got the following errors: I tried using 'go build -tags "daemon libsasl2 gssapi btrfs_noversion libdm_no_deferred_remove"' (hope that makes sense). The error I'm having now is: From what I can tell it looks like libsasl2-dev package supplied by ubuntu is outdated and doesn't include a package configuration file. Any thoughts? |
|
@lioryanko It looks like you need a newer version of the As for libsasl2, replacing |
|
@nalind Well, the basic authentication looks good: There are bugs though which I haven't really inspected thoroughly, here's an example: Also, when trying to build an image (note that there's no authentication prompt): All in all it looks pretty sweet :) |
thaJeztah
added
the
status/needs-attention
|
@lioryanko it seems you had success with basic auth. i can get htpasswd based auth to work. But so far no success on gssapi or libsasl. I ran the AuthSuite tests and they seem to pass. When i start with a --realm and connect over tcp, i get "An error occurred trying to connect: Failed to authenticate to docker daemon; server offered no authentication methods" |
|
@ppadmavilasom Try starting the server with debugging enabled. The server doesn't offer Negotiate unless it can acquire acceptor creds at startup (i.e., unless it can read the keytab and there are keys in it), but it should log a debug message if that's the case. If you're using mandatory access control, permissions may be a factor. If the server is being started with --libsasl2 and --realm but isn't offering Basic, either, then there must have been a problem initializing the SASL library. My best guess would be to check the /etc/sasl2/docker.conf or /usr/lib/sasl2/docker.conf files, both contents and permissions. |
|
I'm wondering whether this can just be implemented in the AuthZ plugin (#15365) and the lessons learnt making this could be channeled to improve the current AuthZ proposal. I personally am adverse to adding very specific authorisation systems which aren't compatible with others. While I'm not exceptionally pleased with the AuthZ plugin as it stands, it's probably how we'll end up doing plugin authorisation. |
…ctionality of the Docker daemon with respect to user authorization. The infrastructure enables registering a set of external authorization plug-in. Each plug-in receives information about the user and the request and decides whether to allow or deny the request. Only in case all plug-ins allow accessing the resource the access is granted. Each plug-in operates as a separate service, and registers with Docker through general (plug-ins API) [https://blog.docker.com/2015/06/extending-docker-with-plugins/]. No Docker daemon recompilation is required in order to add / remove an authentication plug-in. Each plug-in is notified twice for each operation: 1) before the operation is performed and, 2) before the response is returned to the client. The plug-ins can modify the response that is returned to the client. The authorization depends on the authorization effort that takes place in parallel [moby#13697]. This is the official issue of the authorization effort: moby#14674 (Here)[https://github.com/rhatdan/docker-rbac] you can find an open document that discusses a default RBAC plug-in for Docker.
…ctionality of the Docker daemon with respect to user authorization. The infrastructure enables registering a set of external authorization plug-in. Each plug-in receives information about the user and the request and decides whether to allow or deny the request. Only in case all plug-ins allow accessing the resource the access is granted. Each plug-in operates as a separate service, and registers with Docker through general (plug-ins API) [https://blog.docker.com/2015/06/extending-docker-with-plugins/]. No Docker daemon recompilation is required in order to add / remove an authentication plug-in. Each plug-in is notified twice for each operation: 1) before the operation is performed and, 2) before the response is returned to the client. The plug-ins can modify the response that is returned to the client. The authorization depends on the authorization effort that takes place in parallel [moby#13697]. This is the official issue of the authorization effort: moby#14674 (Here)[https://github.com/rhatdan/docker-rbac] you can find an open document that discusses a default RBAC plug-in for Docker. Signed-off-by: Liron Levin <liron@twistlock.com> Added container create flow test and extended the verification for ps
…ctionality of the Docker daemon with respect to user authorization. The infrastructure enables registering a set of external authorization plug-in. Each plug-in receives information about the user and the request and decides whether to allow or deny the request. Only in case all plug-ins allow accessing the resource the access is granted. Each plug-in operates as a separate service, and registers with Docker through general (plug-ins API) [https://blog.docker.com/2015/06/extending-docker-with-plugins/]. No Docker daemon recompilation is required in order to add / remove an authentication plug-in. Each plug-in is notified twice for each operation: 1) before the operation is performed and, 2) before the response is returned to the client. The plug-ins can modify the response that is returned to the client. The authorization depends on the authorization effort that takes place in parallel [moby#13697]. This is the official issue of the authorization effort: moby#14674 (Here)[https://github.com/rhatdan/docker-rbac] you can find an open document that discusses a default RBAC plug-in for Docker. Signed-off-by: Liron Levin <liron@twistlock.com> Added container create flow test and extended the verification for ps
|
@nalind @NathanMcCauley do you know what's status here? I see some references from another repos, but not sure how to interpret them. |
|
Closing this one because it went stale, and probably can be implemented using the AuthZ plugins |
thaJeztah
removed
the
status/needs-attention
Hi Guys
We develop a container security suite oriented to the enterprises. As part of it we support Kerberos as an authentication mechanism between Docker CLI and the Docker daemon. We currently have it up and running but, honestly, think it should be part of the platform. Hence, we would like to contribute the code to Docker.
Today we use Nalin's GSS Go wrapper (https://github.com/nalind/gss) over vanilla MIT Kerberos (https://github.com/krb5/krb5).
I'd like to propose the following, simple, addition to the command line interface that enables generating Kerberos ticket (TGS) on the client (CLI), and require validating Kerberos ticket on the daemon.
CLI example:
Docker daemon example:
The SPN argument is mandatory. However, keytab argument is not. In case keytab is not specified, a default path is used. The default path is:
/etc/krb5.keytab(as in MIT krb5)All variables can be configured using environmental variables:
We will support MIT krb5 and Microsoft Active Directory as KDC.
I would be glad to hear you general thoughts and if make sense I'll send a more elaborated technical descriptions of what we'd like to do.
The text was updated successfully, but these errors were encountered: