Skip to content

Commit

Permalink
feat: add support for serviceaccount object (#249)
Browse files Browse the repository at this point in the history
  • Loading branch information
rasta-rocket committed Jun 19, 2023
1 parent 1d0516c commit eab8eec
Show file tree
Hide file tree
Showing 3 changed files with 248 additions and 22 deletions.
42 changes: 21 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# ConfigMap, Secret and Role, and RoleBinding replication for Kubernetes
# ConfigMap, Secret and Role, RoleBinding and ServiceAccount replication for Kubernetes

![Build Status](https://github.com/mittwald/kubernetes-replicator/workflows/Compile%20&%20Test/badge.svg)

Expand Down Expand Up @@ -51,10 +51,10 @@ $ kubectl apply -f https://raw.githubusercontent.com/mittwald/kubernetes-replica

To create a new role, your own account needs to have at least the same set of privileges as the role you're trying to create. The chart currently offers two options to grant these permissions to the service account used by the replicator:

- Set the value `grantClusterAdmin`to `true`, which grants the service account admin privileges. This is set to `false` by default, as having a service account with that level of access might be undesirable due to the potential security risks attached.
- Set the value `grantClusterAdmin`to `true`, which grants the service account admin privileges. This is set to `false` by default, as having a service account with that level of access might be undesirable due to the potential security risks attached.

- Set the lists of needed api groups and resources explicitely. These can be specified using the value `privileges`. `privileges` is a list that contains pairs of api group and resource lists.

- Set the lists of needed api groups and resources explicitely. These can be specified using the value `privileges`. `privileges` is a list that contains pairs of api group and resource lists.

Example:

```yaml
Expand All @@ -63,14 +63,14 @@ To create a new role, your own account needs to have at least the same set of pr
annotations: {}
name:
privileges:
- apiGroups: [ "", "apps", "extensions" ]
- apiGroups: [ "", "apps", "extensions" ]
resources: ["secrets", "configmaps", "roles", "rolebindings",
"cronjobs", "deployments", "events", "ingresses", "jobs", "pods", "pods/attach", "pods/exec", "pods/log", "pods/portforward", "services"]
- apiGroups: [ "batch" ]
resources: ["configmaps", "cronjobs", "deployments", "events", "ingresses", "jobs", "pods", "pods/attach", "pods/exec", "pods/log", "pods/portforward", "services"]
```

These settings permit the replication of Roles and RoleBindings with privileges for the api groups `""`. `apps`, `batch` and `extensions` on the resources specified.
These settings permit the replication of Roles and RoleBindings with privileges for the api groups `""`. `apps`, `batch` and `extensions` on the resources specified.

### "Push-based" replication

Expand Down Expand Up @@ -113,19 +113,19 @@ It is possible to use both methods of push-based replication together in a singl

### "Pull-based" replication

Pull-based replication makes it possible to create a secret/configmap/role/rolebindings and select a "source" resource
Pull-based replication makes it possible to create a secret/configmap/role/rolebindings and select a "source" resource
from which the data is replicated from.

#### Step 1: Create the source secret

If a secret or configMap needs to be replicated to other namespaces, annotations should be added in that object
If a secret or configMap needs to be replicated to other namespaces, annotations should be added in that object
permitting replication.
- Add `replicator.v1.mittwald.de/replication-allowed` annotation with value `true` indicating that the object can be

- Add `replicator.v1.mittwald.de/replication-allowed` annotation with value `true` indicating that the object can be
replicated.
- Add `replicator.v1.mittwald.de/replication-allowed-namespaces` annotation. Value of this annotation should contain
a comma separated list of permitted namespaces or regular expressions. For example `namespace-1,my-ns-2,app-ns-[0-9]*`:
in this case replication will be performed only into the namespaces `namespace-1` and `my-ns-2` as well as any
- Add `replicator.v1.mittwald.de/replication-allowed-namespaces` annotation. Value of this annotation should contain
a comma separated list of permitted namespaces or regular expressions. For example `namespace-1,my-ns-2,app-ns-[0-9]*`:
in this case replication will be performed only into the namespaces `namespace-1` and `my-ns-2` as well as any
namespace that matches the regular expression `app-ns-[0-9]*`.

```yaml
Expand All @@ -141,7 +141,7 @@ permitting replication.

#### Step 2: Create an empty destination secret

Add the annotation `replicator.v1.mittwald.de/replicate-from` to any Kubernetes secret or config map object. The value
Add the annotation `replicator.v1.mittwald.de/replicate-from` to any Kubernetes secret or config map object. The value
of that annotation should contain the the name of another secret or config map (using `<namespace>/<name>` notation).

```yaml
Expand All @@ -154,13 +154,13 @@ metadata:
data: {}
```

The replicator will then copy the `data` attribute of the referenced object into the annotated object and keep them in
sync.
The replicator will then copy the `data` attribute of the referenced object into the annotated object and keep them in
sync.

#### Special case: TLS secrets

Secrets of type `kubernetes.io/tls` are treated in a special way and need to have a `data["tls.crt"]` and a
`data["tls.key"]` property to begin with. In the replicated secrets, these properties need to be present to begin with,
Secrets of type `kubernetes.io/tls` are treated in a special way and need to have a `data["tls.crt"]` and a
`data["tls.key"]` property to begin with. In the replicated secrets, these properties need to be present to begin with,
but they may be empty:

```yaml
Expand All @@ -178,8 +178,8 @@ data:

#### Special case: Docker registry credentials

Secrets of type `kubernetes.io/dockerconfigjson` also require special treatment. These secrets require to have a
`.dockerconfigjson` key that needs to require valid JSON. For this reason, a replicated secret of this type should be
Secrets of type `kubernetes.io/dockerconfigjson` also require special treatment. These secrets require to have a
`.dockerconfigjson` key that needs to require valid JSON. For this reason, a replicated secret of this type should be
created as follows:

```yaml
Expand Down Expand Up @@ -215,7 +215,7 @@ data:

#### Special case: Resource with .metadata.ownerReferences

Sometimes, secrets are generated by external components. Such secrets are configured with an ownerReference. By default, the kubernetes-replicator will delete the
Sometimes, secrets are generated by external components. Such secrets are configured with an ownerReference. By default, the kubernetes-replicator will delete the
ownerReference in the target namespace.

ownerReference won't work [across different namespaces](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/#owners-and-dependents) and the secret at the destination will be removed by the kubernetes garbage collection.
Expand Down
6 changes: 5 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"github.com/mittwald/kubernetes-replicator/replicate/role"
"github.com/mittwald/kubernetes-replicator/replicate/rolebinding"
"github.com/mittwald/kubernetes-replicator/replicate/secret"
"github.com/mittwald/kubernetes-replicator/replicate/serviceaccount"

log "github.com/sirupsen/logrus"

Expand Down Expand Up @@ -84,6 +85,7 @@ func main() {
configMapRepl := configmap.NewReplicator(client, f.ResyncPeriod, f.AllowAll)
roleRepl := role.NewReplicator(client, f.ResyncPeriod, f.AllowAll)
roleBindingRepl := rolebinding.NewReplicator(client, f.ResyncPeriod, f.AllowAll)
serviceAccountRepl := serviceaccount.NewReplicator(client, f.ResyncPeriod, f.AllowAll)

go secretRepl.Run()

Expand All @@ -93,8 +95,10 @@ func main() {

go roleBindingRepl.Run()

go serviceAccountRepl.Run()

h := liveness.Handler{
Replicators: []common.Replicator{secretRepl, configMapRepl, roleRepl, roleBindingRepl},
Replicators: []common.Replicator{secretRepl, configMapRepl, roleRepl, roleBindingRepl, serviceAccountRepl},
}

log.Infof("starting liveness monitor at %s", f.StatusAddr)
Expand Down
222 changes: 222 additions & 0 deletions replicate/serviceaccount/serviceaccounts.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
package serviceaccount

import (
"context"
"encoding/json"
"fmt"
"time"

"github.com/mittwald/kubernetes-replicator/replicate/common"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
v1 "k8s.io/api/core/v1"

corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"
)

type Replicator struct {
*common.GenericReplicator
}

// NewReplicator creates a new serviceaccount replicator
func NewReplicator(client kubernetes.Interface, resyncPeriod time.Duration, allowAll bool) common.Replicator {
repl := Replicator{
GenericReplicator: common.NewGenericReplicator(common.ReplicatorConfig{
Kind: "ServiceAccount",
ObjType: &corev1.ServiceAccount{},
AllowAll: allowAll,
ResyncPeriod: resyncPeriod,
Client: client,
ListFunc: func(lo metav1.ListOptions) (runtime.Object, error) {
return client.CoreV1().ServiceAccounts("").List(context.TODO(), lo)
},
WatchFunc: func(lo metav1.ListOptions) (watch.Interface, error) {
return client.CoreV1().ServiceAccounts("").Watch(context.TODO(), lo)
},
}),
}
repl.UpdateFuncs = common.UpdateFuncs{
ReplicateDataFrom: repl.ReplicateDataFrom,
ReplicateObjectTo: repl.ReplicateObjectTo,
PatchDeleteDependent: repl.PatchDeleteDependent,
DeleteReplicatedResource: repl.DeleteReplicatedResource,
}

return &repl
}

func (r *Replicator) ReplicateDataFrom(sourceObj interface{}, targetObj interface{}) error {
source := sourceObj.(*corev1.ServiceAccount)
target := targetObj.(*corev1.ServiceAccount)

logger := log.
WithField("kind", r.Kind).
WithField("source", common.MustGetKey(source)).
WithField("target", common.MustGetKey(target))

// make sure replication is allowed
if ok, err := r.IsReplicationPermitted(&target.ObjectMeta, &source.ObjectMeta); !ok {
return errors.Wrapf(err, "replication of target %s is not permitted", common.MustGetKey(source))
}

targetVersion, ok := target.Annotations[common.ReplicatedFromVersionAnnotation]
sourceVersion := source.ResourceVersion

if ok && targetVersion == sourceVersion {
logger.Debugf("target %s/%s is already up-to-date", target.Namespace, target.Name)
return nil
}

targetCopy := target.DeepCopy()
targetCopy.ImagePullSecrets = source.ImagePullSecrets

log.Infof("updating target %s/%s", target.Namespace, target.Name)

targetCopy.Annotations[common.ReplicatedAtAnnotation] = time.Now().Format(time.RFC3339)
targetCopy.Annotations[common.ReplicatedFromVersionAnnotation] = source.ResourceVersion

s, err := r.Client.CoreV1().ServiceAccounts(target.Namespace).Update(context.TODO(), targetCopy, metav1.UpdateOptions{})
if err != nil {
err = errors.Wrapf(err, "Failed updating target %s/%s", target.Namespace, targetCopy.Name)
} else if err = r.Store.Update(s); err != nil {
err = errors.Wrapf(err, "Failed to update cache for %s/%s: %v", target.Namespace, targetCopy, err)
}

return err
}

// ReplicateObjectTo copies the whole object to target namespace
func (r *Replicator) ReplicateObjectTo(sourceObj interface{}, target *v1.Namespace) error {
source := sourceObj.(*corev1.ServiceAccount)
targetLocation := fmt.Sprintf("%s/%s", target.Name, source.Name)

logger := log.
WithField("kind", r.Kind).
WithField("source", common.MustGetKey(source)).
WithField("target", targetLocation)

targetResource, exists, err := r.Store.GetByKey(targetLocation)
if err != nil {
return errors.Wrapf(err, "Could not get %s from cache!", targetLocation)
}
logger.Infof("Checking if %s exists? %v", targetLocation, exists)

var targetCopy *corev1.ServiceAccount
if exists {
targetObject := targetResource.(*corev1.ServiceAccount)
targetVersion, ok := targetObject.Annotations[common.ReplicatedFromVersionAnnotation]
sourceVersion := source.ResourceVersion

if ok && targetVersion == sourceVersion {
logger.Debugf("ServiceAccount %s is already up-to-date", common.MustGetKey(targetObject))
return nil
}

targetCopy = targetObject.DeepCopy()
} else {
targetCopy = new(corev1.ServiceAccount)
}

keepOwnerReferences, ok := source.Annotations[common.KeepOwnerReferences]
if ok && keepOwnerReferences == "true" {
targetCopy.OwnerReferences = source.OwnerReferences
}

if targetCopy.Annotations == nil {
targetCopy.Annotations = make(map[string]string)
}

labelsCopy := make(map[string]string)

stripLabels, ok := source.Annotations[common.StripLabels]
if !ok && stripLabels != "true" {
if source.Labels != nil {
for key, value := range source.Labels {
labelsCopy[key] = value
}
}

}

targetCopy.Name = source.Name
targetCopy.Labels = labelsCopy
targetCopy.ImagePullSecrets = source.ImagePullSecrets
targetCopy.Annotations[common.ReplicatedAtAnnotation] = time.Now().Format(time.RFC3339)
targetCopy.Annotations[common.ReplicatedFromVersionAnnotation] = source.ResourceVersion

var obj interface{}

if exists {
if err == nil {
logger.Debugf("Updating existing serviceAccount %s/%s", target.Name, targetCopy.Name)
obj, err = r.Client.CoreV1().ServiceAccounts(target.Name).Update(context.TODO(), targetCopy, metav1.UpdateOptions{})
}
} else {
if err == nil {
logger.Debugf("Creating a new serviceAccount %s/%s", target.Name, targetCopy.Name)
obj, err = r.Client.CoreV1().ServiceAccounts(target.Name).Create(context.TODO(), targetCopy, metav1.CreateOptions{})
}
}
if err != nil {
return errors.Wrapf(err, "Failed to update serviceAccount %s/%s", target.Name, targetCopy.Name)
}

if err := r.Store.Update(obj); err != nil {
return errors.Wrapf(err, "Failed to update cache for %s/%s", target.Name, targetCopy)
}

return nil
}

func (r *Replicator) PatchDeleteDependent(sourceKey string, target interface{}) (interface{}, error) {
dependentKey := common.MustGetKey(target)
logger := log.WithFields(log.Fields{
"kind": r.Kind,
"source": sourceKey,
"target": dependentKey,
})

targetObject, ok := target.(*corev1.ServiceAccount)
if !ok {
err := errors.Errorf("bad type returned from Store: %T", target)
return nil, err
}

patch := []common.JSONPatchOperation{{Operation: "remove", Path: "/imagePullSecrets"}}
patchBody, err := json.Marshal(&patch)

if err != nil {
return nil, errors.Wrapf(err, "error while building patch body for serviceAccount %s: %v", dependentKey, err)

}

logger.Debugf("clearing dependent serviceAccount %s", dependentKey)
logger.Tracef("patch body: %s", string(patchBody))

s, err := r.Client.CoreV1().ServiceAccounts(targetObject.Namespace).Patch(context.TODO(), targetObject.Name, types.JSONPatchType, patchBody, metav1.PatchOptions{})
if err != nil {
return nil, errors.Wrapf(err, "error while patching serviceAccount %s: %v", dependentKey, err)
}
return s, nil
}

// DeleteReplicatedResource deletes a resource replicated by ReplicateTo annotation
func (r *Replicator) DeleteReplicatedResource(targetResource interface{}) error {
targetLocation := common.MustGetKey(targetResource)
logger := log.WithFields(log.Fields{
"kind": r.Kind,
"target": targetLocation,
})

object := targetResource.(*corev1.ServiceAccount)
logger.Debugf("Deleting %s", targetLocation)
if err := r.Client.CoreV1().ServiceAccounts(object.Namespace).Delete(context.TODO(), object.Name, metav1.DeleteOptions{}); err != nil {
return errors.Wrapf(err, "Failed deleting %s: %v", targetLocation, err)
}
return nil
}

0 comments on commit eab8eec

Please sign in to comment.