Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add helm revoke functionality #50

Merged
2 changes: 1 addition & 1 deletion .circleci/config.yml
Expand Up @@ -11,7 +11,7 @@ defaults: &defaults
PACKER_VERSION: NONE
GOLANG_VERSION: 1.11.2
KUBECONFIG: /home/circleci/.kube/config
HELM_VERSION: v2.11.0 # Same as helm provider
HELM_VERSION: v2.14.0 # Same as helm provider
bwhaley marked this conversation as resolved.
Show resolved Hide resolved


install_helm_client: &install_helm_client
Expand Down
2 changes: 2 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion HELM_GUIDE.md
Expand Up @@ -291,7 +291,7 @@ The command does not expose a way to turn this off.

### Enable TLS verification in the Server

`kubergrunt helm deploy` will deploy Tiller with TLS authentication turned on (see section [Client
`kubergrunt helm deploy` will deploy Tiller with mutual TLS authentication turned on (see section [Client
Authentication](#client-authentication) for more details on TLS authentication in Tiller). The command does not expose a
way to turn this off.

Expand Down Expand Up @@ -322,11 +322,18 @@ authenticate against the deployed Tiller instance. This is done by:
- Downloading the CA certificate key pair.
- Generating new signed certificate key pair for each RBAC entity passed in.
- Storing each new certificate key pair as a `Secret` in the Tiller namespace.
- Create a role and rolebinding in the Tiller namespace for each RBAC entity.

The `Secret` containing the new certificate key pair is shared with the RBAC entity so that they can download it to
configure their client. `kubergrunt` provides the `helm configure` subcommand for your users to use to setup their local
`helm` client with the new certificate key pairs.

You can use `kubergrunt helm revoke` to deauthorize access to Tiller from a given set of RBAC entities. This command removes
the role, rolebinding, and certificate key pair secrets for the provided entities. Be aware that revoking does not invalidate
the signed TLS certificate because Helm/Tiller will not check a Certificate Revocation List or otherwise respect certificate
revocation. However, by removing the roles, the entity is deauthorized from access to Tiller. If you wish to invalidate
the TLS key pair, Tiller's CA must be replaced and all key pairs reissued.
bwhaley marked this conversation as resolved.
Show resolved Hide resolved

Note that `helm grant` should only be run by an administrator of your cluster. This is because only administrators should
have access to the CA certificate key pair, as that enables you to grant anyone access to the deployed Tiller instance.

Expand Down
22 changes: 11 additions & 11 deletions README.md
Expand Up @@ -53,7 +53,7 @@ The following commands are available as part of `kubergrunt`:
* [undeploy](#undeploy)
* [configure](#helm-configure)
* [grant](#grant)
<!-- not implemented * [revoke](#revoke) -->
* [revoke](#revoke)
1. [k8s](#k8s)
* [wait-for-ingress](#wait-for-ingress)
1. [tls](#tls)
Expand Down Expand Up @@ -385,7 +385,7 @@ This subcommand will grant access to an installed helm server to a given RBAC en
to. This access is readonly.
- Remove the local copies of the downloaded and generated certificates.

This command assumes that the authenticated entitiy running the command has enough permissions to access the generated
This command assumes that the authenticated entity running the command has enough permissions to access the generated
bwhaley marked this conversation as resolved.
Show resolved Hide resolved
CA `Secret`.

For example, to grant access to a Tiller server deployed in the namespace `tiller-world` to the RBAC group `developers`:
Expand All @@ -404,27 +404,27 @@ This command should be run by a **cluster administrator** to grant a user, group
instance. The user or group should be an RBAC entity (RBAC user or RBAC group). The pod should gain access via the
mounted `ServiceAccount`.

<!-- Not implemented

#### revoke

This subcommand will revoke access to an installed helm server for a given RBAC role. This will:
This subcommand will revoke access to an installed helm server for a given RBAC entity. This will:

- Download the corresponding CA keypair for the Tiller deployment from Kubernetes.
- Download the TLS certificate keypair for the RBAC role.
- Revoke the certificate in the CA.
- Update the CA certificate keypair in both the `Secret` and the installed Tiller server.
- Restart Tiller.
- Remove the role and rolebinding associated with the RBAC entity (user, group, or service account)
- Remove the TLS keypair Secret associated with the RBAC entity

For example, to revoke access to a Tiller server deployed in the namespace `tiller-world` from the RBAC role `dev`:
For example, to revoke access to a Tiller server deployed in the namespace `tiller-world` from the RBAC user `dev`:

```bash
kubergrunt helm revoke --tiller-namespace tiller-world --rbac-user dev
```

See the command help for all the available options: `kubergrunt helm revoke --help`.

-->
**Note**: The Go TLS library [does not support certificate revocation](https://www.imperialviolet.org/2014/04/19/revchecking.html).
As a consequence, Helm/Tiller cannot check for revocation. The upshot is that a client that retains a previously signed TLS keypair can
still authenticate to tiller, even after running `kubergrunt helm revoke`. However, since `kubergrunt` removes the authorizations associated
with that entity, the entity is effectively disabled. If you wish to render the signed keypair invalid, you must generate a new
Certificate Authority for tiller and reissue all keypairs.
bwhaley marked this conversation as resolved.
Show resolved Hide resolved

### k8s

Expand Down
56 changes: 48 additions & 8 deletions cmd/helm.go
Expand Up @@ -21,7 +21,7 @@ import (

const (
DefaultTillerImage = "gcr.io/kubernetes-helm/tiller"
DefaultTillerVersion = "v2.11.0"
DefaultTillerVersion = "v2.14.0"
DefaultTillerDeploymentName = "tiller-deploy"
CreateTmpFolderForHelmHome = "__TMP__"
)
Expand Down Expand Up @@ -147,15 +147,15 @@ var (
// Configurations for granting and revoking access to clients
grantedRbacGroupsFlag = cli.StringSliceFlag{
Name: "rbac-group",
Usage: "The name of the RBAC group that should be granted access to tiller. Pass in multiple times for multiple groups.",
Usage: "The name of the RBAC group that should be granted access to (or revoked from) tiller. Pass in multiple times for multiple groups.",
}
grantedRbacUsersFlag = cli.StringSliceFlag{
Name: "rbac-user",
Usage: "The name of the RBAC user that should be granted access to Tiller. Pass in multiple times for multiple users.",
Usage: "The name of the RBAC user that should be granted access to (or revoked from) Tiller. Pass in multiple times for multiple users.",
}
grantedServiceAccountsFlag = cli.StringSliceFlag{
Name: "rbac-service-account",
Usage: "The name and namespace of the ServiceAccount (encoded as NAMESPACE/NAME) that should be granted access to tiller. Pass in multiple times for multiple accounts.",
Usage: "The name and namespace of the ServiceAccount (encoded as NAMESPACE/NAME) that should be granted access to (or revoked from) tiller. Pass in multiple times for multiple accounts.",
}
bwhaley marked this conversation as resolved.
Show resolved Hide resolved

// Configurations for undeploying helm
Expand Down Expand Up @@ -309,10 +309,12 @@ This allows you to use the configure command as a data source that is passed int
},
},
cli.Command{
Name: "grant",
Usage: "Grant access to a deployed Helm server.",
Description: "Grant access to a deployed Helm server to a client by issuing new TLS certificate keypairs that is accessible by the provided RBAC group.",
Action: grantHelmAccess,
Name: "grant",
Usage: "Grant access to a deployed Helm server.",
Description: `Grant access to a deployed Helm server to a client by issuing new TLS certificate keypairs that is accessible by the provided RBAC group.

At least one of --rbac-user, --rbac-group, or --rbac-service-account are required.`,
Action: grantHelmAccess,
Flags: []cli.Flag{
tillerNamespaceFlag,
grantedRbacGroupsFlag,
Expand All @@ -336,6 +338,25 @@ This allows you to use the configure command as a data source that is passed int
helmKubectlTokenFlag,
},
},
cli.Command{
Name: "revoke",
Usage: "Revoke access to a deployed Helm server.",
Description: `Revoke access to a deployed Helm server from a client by removing the role and role bindings for a provided RBAC entity. Also removes the signed TLS certificate and key from the Secrets associated with this entity.

At least one of --rbac-user, --rbac-group, or --rbac-service-account are required.`,
Action: revokeHelmAccess,
Flags: []cli.Flag{
tillerNamespaceFlag,
grantedRbacGroupsFlag,
grantedRbacUsersFlag,
grantedServiceAccountsFlag,
helmKubectlContextNameFlag,
helmKubeconfigFlag,
helmKubectlServerFlag,
helmKubectlCAFlag,
helmKubectlTokenFlag,
},
},
cli.Command{
Name: "wait-for-tiller",
Usage: "Wait for Tiller to be provisioned.",
Expand Down Expand Up @@ -587,6 +608,25 @@ func grantHelmAccess(cliContext *cli.Context) error {
return helm.GrantAccess(kubectlOptions, tlsOptions, tillerNamespace, rbacGroups, rbacUsers, serviceAccounts)
}

// revokeHelmAccess is the action function for the helm revoke command.
func revokeHelmAccess(cliContext *cli.Context) error {
tillerNamespace, err := entrypoint.StringFlagRequiredE(cliContext, tillerNamespaceFlag.Name)
if err != nil {
return err
}
kubectlOptions, err := parseKubectlOptions(cliContext)
if err != nil {
return err
}
rbacGroups := cliContext.StringSlice(grantedRbacGroupsFlag.Name)
rbacUsers := cliContext.StringSlice(grantedRbacUsersFlag.Name)
serviceAccounts := cliContext.StringSlice(grantedServiceAccountsFlag.Name)
if len(rbacGroups) == 0 && len(rbacUsers) == 0 && len(serviceAccounts) == 0 {
return entrypoint.NewRequiredArgsError("At least one --rbac-group, --rbac-user, or --rbac-service-account is required")
bwhaley marked this conversation as resolved.
Show resolved Hide resolved
}
return helm.RevokeAccess(kubectlOptions, tillerNamespace, rbacGroups, rbacUsers, serviceAccounts)
}

// parseTLSArgs will take CLI args pertaining to TLS and extract out a TLSOptions struct.
func parseTLSArgs(cliContext *cli.Context, isClient bool) (tls.TLSOptions, error) {
var distinguishedName pkix.Name
Expand Down
52 changes: 47 additions & 5 deletions helm/deploy_test.go
Expand Up @@ -13,6 +13,8 @@ import (
"github.com/gruntwork-io/terratest/modules/shell"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/helm/pkg/helm/portforwarder"

"github.com/gruntwork-io/kubergrunt/kubectl"
Expand Down Expand Up @@ -53,25 +55,28 @@ func TestValidateRequiredResourcesForDeploy(t *testing.T) {
assert.NoError(t, err)
}

// This is an end to end integration for the commands to setup helm access. This integrationtest is designed this way
// This is an end to end integration for the commands to setup helm access. This integration test is designed this way
// due to the way each step is setup to build on the previous step. For example, it is impossible to test grant without
// having a helm server deployed, and configure without running grant.
//
// Test that we can:
// Test that we can include:
bwhaley marked this conversation as resolved.
Show resolved Hide resolved
// 1. Generate certificate key pairs for use with Tiller
// 2. Upload certificate key pairs to Kubernetes secrets
// 3. Deploy Helm with TLS enabled in the specified namespace
// 4. Grant access to helm
// 5. Configure helm client
// 6. Deploy a helm chart
// 7. Undeploy helm
// 7. Revoke a service account
// 8. Undeploy helm
func TestHelmDeployConfigureUndeploy(t *testing.T) {
t.Parallel()

imageSpec := "gcr.io/kubernetes-helm/tiller:v2.11.0"
imageSpec := "gcr.io/kubernetes-helm/tiller:v2.14.0"

kubectlOptions := kubectl.GetTestKubectlOptions(t)
terratestKubectlOptions := k8s.NewKubectlOptions("", "")
kubeClient, err := k8s.GetKubernetesClientFromOptionsE(t, terratestKubectlOptions)
assert.NoError(t, err)
tlsOptions := tls.SampleTlsOptions(tls.ECDSAAlgorithm)
clientTLSOptions := tls.SampleTlsOptions(tls.ECDSAAlgorithm)
clientTLSOptions.DistinguishedName.CommonName = "client"
Expand All @@ -96,7 +101,7 @@ func TestHelmDeployConfigureUndeploy(t *testing.T) {
// server so that it crashes should the release removal fail.
assert.NoError(t, Undeploy(kubectlOptions, namespaceName, "", false, true))
}()
// Deploy, Grant, and Configure
// Deploy, Grant, Configure, and Revoke
assert.NoError(t, Deploy(
kubectlOptions,
namespaceName,
Expand Down Expand Up @@ -132,6 +137,21 @@ func TestHelmDeployConfigureUndeploy(t *testing.T) {

// Check that the rendered helm env file works
validateHelmEnvFile(t, testServiceAccountKubectlOptions)

// Revoke the tiller service account
rbacGroups := []string{}
rbacUsers := []string{}
serviceAccounts := []string{fmt.Sprintf("%s/%s", namespaceName, testServiceAccountName)}
assert.NoError(t, RevokeAccess(kubectlOptions, namespaceName, rbacGroups, rbacUsers, serviceAccounts))
bwhaley marked this conversation as resolved.
Show resolved Hide resolved

// No role for service account
assert.Error(t, validateNoRole(t, kubeClient, namespaceName, testServiceAccountName))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one issue I see with this is that you're assuming a certain type of error will be returned: role does not exist. But if a different error occurs here (e.g. "permissions denied") the test won't distinguish between the two.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming that I cannot get the role (or role binding, or secret; more or less the same approach for each of these tests). If I can get the role, then something went wrong on the revocation. If there was a permissions problem or otherwise, it would've been thrown in RevokeAccess().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you take the updated labels approach I proposed here (#50 (comment)), you can use the List interface instead for a more targeted existence check.


// No rolebinding for service account
assert.Error(t, validateNoRoleBinding(t, kubeClient, namespaceName, testServiceAccountName))

// No TLS keypair secret for service account
assert.Error(t, validateNoTLSSecret(t, kubectlOptions, namespaceName, serviceAccountName))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on Josh's comment above: we should check which error comes back in all these checks, as these could be returning errors for a variety of reasons, and not all of them are necessarily what we're checking for.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, is the real way to test this to try to make requests as the user, check they succeed, revoke the user's access, and check the requests again to make sure they fail?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that would certainly be the more complete test. However, kubergrunt currently doesn't run any tests from the perspective of multiple users. It relies on the kubectl credentials currently available in the executing user's path. Adding the ability to switch from admin/user context is a bit more time than I can afford at the moment unfortunately and would have some downstream implications.

}

// validateTillerPodDeployedInNamespace validates that the tiller pod was deployed into the provided namespace and
Expand Down Expand Up @@ -295,3 +315,25 @@ func validateHelmEnvFile(t *testing.T, options *kubectl.KubectlOptions) {
}
shell.RunCommand(t, cmd)
}

// validateNoServiceAccountRole validates that the mock service account does not have an associated role
func validateNoRole(t *testing.T, client *kubernetes.Clientset, namespace, serviceAccountName string) error {
roleName := getTillerAccessRoleName(serviceAccountName, namespace)
_, err := client.RbacV1().Roles(namespace).Get(roleName, metav1.GetOptions{})
return err
}

// validateNoServiceAccountRoleBinding validates that the mock service account does not have an associated rolebinding
func validateNoRoleBinding(t *testing.T, client *kubernetes.Clientset, namespace, serviceAccountName string) error {
roleName := getTillerAccessRoleName(serviceAccountName, namespace)
roleBindingName := getTillerAccessRoleBindingName(serviceAccountName, roleName)
_, err := client.RbacV1().RoleBindings(namespace).Get(roleBindingName, metav1.GetOptions{})
return err
}

// validateNoTLSSecret validates that the mock service account does not have an associated secret TLS keypair
func validateNoTLSSecret(t *testing.T, options *kubectl.KubectlOptions, namespace, serviceAccountName string) error {
secretName := getTillerClientCertSecretName(serviceAccountName)
_, err := kubectl.GetSecret(options, namespace, secretName)
return err
}
8 changes: 8 additions & 0 deletions helm/names.go
Expand Up @@ -19,3 +19,11 @@ func getTillerCACertSecretName(tillerNamespace string) string {
func md5HashString(input string) string {
return fmt.Sprintf("%x", md5.Sum([]byte(input)))
}

func getTillerAccessRoleName(entityID, namespace string) string {
return fmt.Sprintf("%s-%s-tiller-access", entityID, namespace)
}

func getTillerAccessRoleBindingName(entityID, roleName string) string {
return fmt.Sprintf("%s-%s-binding", entityID, roleName)
}