Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dat feat awsmp sleek rebase #193

Draft
wants to merge 85 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
e7990ea
feat: new chart for aws marketplace product. The product supports lic…
Sep 6, 2022
6397cdd
fix: qoute productSKU value; add logs for checkout license
Sep 6, 2022
b658876
refactor: upgrade values.yaml with updated image pushed to aws mp pro…
Sep 6, 2022
0640270
fix: support region, product, fingerprint and debug env variables; re…
Sep 7, 2022
fdc41c2
fix: run license cron job on @daily. fix values naming bug
Sep 7, 2022
8ebf020
fix: upgrade image in aws mp helm chart values
Sep 7, 2022
17e6251
fix: remove awsmp-... chart. datree-admission-webhook chart should ho…
Sep 12, 2022
e0a348b
fix: add * to ignore luanch.json file
Sep 12, 2022
5289408
fix: remove ** to ignore luanch.json file
Sep 12, 2022
275e001
fix: accidently deleted gh-pages values file with awsmp values file. …
Sep 21, 2022
eeaa96b
fix: added aws.values
Sep 21, 2022
5e73163
feat: added validation for value.yaml file
Sep 12, 2022
2abeacc
fix: combine charts and added values.yaml for each option
Sep 19, 2022
cc12656
feat: script for packaing
Sep 19, 2022
2ccc1f3
fix: added script package for free-datree
Sep 19, 2022
595a75e
feat: added chart
Sep 20, 2022
42fbd5e
feat: charts file structure
Sep 20, 2022
f896fba
fix: combine charts and added values.yaml for each option
Sep 19, 2022
6489875
feat: script for packaing
Sep 19, 2022
150fba7
fix: added script package for free-datree
Sep 19, 2022
002513b
feat: added chart
Sep 20, 2022
f650e64
fix: move tempaltes to lib template
shmu3l Sep 28, 2022
448011f
fix: remove templates use include from lib
shmu3l Sep 28, 2022
b43ef0f
fix: remove templates use include from lib
shmu3l Sep 28, 2022
771c57a
fix: add dynamic webhook server alt name to cert
shmu3l Oct 2, 2022
2002a8d
fix: clear datree webhook values and update schema
shmu3l Oct 2, 2022
9b446e7
feat: awsmp datree use datree lib
shmu3l Oct 2, 2022
af85f57
fix: awsmp file structure
shmu3l Oct 2, 2022
d1ef809
feat: add release admission webhook script
shmu3l Oct 3, 2022
121763b
fix: release admission webhook
shmu3l Oct 3, 2022
37a828d
release chart 0.1.3
shmu3l Oct 3, 2022
8ddb66f
fix: release admission webhook helm script update
shmu3l Oct 6, 2022
23a42d7
fix: update .gitignore
Oct 6, 2022
30c96e4
fix: change messages
Oct 6, 2022
572c8c5
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
a13fabc
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
57a809a
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
b79e351
fix: rename script
Oct 6, 2022
edfa70f
fix: change the order of helm file creations
Oct 6, 2022
88d5416
fix: remove files
Oct 6, 2022
7e60bf9
fix: release file index yaml
Oct 6, 2022
2d54188
fix: dont stash pop
Oct 6, 2022
56e619f
feat: certificate alternative names use datree.namespace template.
Oct 18, 2022
c02ba66
fix: bump lib chart version
Oct 18, 2022
7132337
fix: added dynamic namespace installation. Rename chart according to …
Oct 24, 2022
883f25c
fix: aligned charts with aws-marketplace ECR repositories.
Nov 3, 2022
d4c0867
feat: pull bitnami-kubectl from private ecr registry
Nov 3, 2022
a09986d
fix: updated bitnami in free offer chart
Nov 3, 2022
378ef6a
feat: added new binary in cmd folder for init container
Nov 7, 2022
a5e0e21
wip
Nov 8, 2022
7d9411c
wip: poc of webhook race condition
Nov 9, 2022
076b597
fix: working product on minikube. NOTE: very slow
Nov 10, 2022
d4cd131
fix: working version
Nov 13, 2022
a1d8ba3
fix: wokring production version on minikube
Nov 13, 2022
4e504da
fix: working production version. Minikube. 3minutes sleep time
Nov 13, 2022
956b584
fix: working version ECR repositories on Faragate.
Nov 13, 2022
3cfe527
feat: support uninstall Datree product
Nov 13, 2022
d2eab5f
fix: working version in AWS Marketplace. Datree product
Nov 13, 2022
58e430c
feat: working version. Datree Free product. version 1.0.1-rc.1
Nov 13, 2022
7ba108f
feat: check for webhook existence, wait for running pods. Refactor
Nov 15, 2022
1f1d134
fix: wait for all deployment replicas to be ready. updated ecr images
Nov 15, 2022
07f0a82
fix: update aws mp Chart
Nov 16, 2022
d51dfde
fix: remove .vscode luanch.json
Nov 16, 2022
cb90d7b
fix: update Dockerfiles
Nov 16, 2022
c3a79b2
fix: remove comments
Nov 16, 2022
09fc506
test: added tests for cert-generator
Nov 16, 2022
f5593a7
test: wip
Nov 20, 2022
4ccf30e
test: fixed table testing for k8sClient
Nov 20, 2022
8161364
test: cert-generator tests
Nov 21, 2022
1e4697b
fix: structure packages in webhook-init to not use one another, only …
Nov 21, 2022
a550115
fix: updated makefile
Nov 21, 2022
52ce8b7
test: ensure empty env variables has defaults
Nov 21, 2022
4b57836
fix: bump images versions, working product on minikube.
Nov 21, 2022
f893b31
fix: make code build succesfully
Nov 21, 2022
b3867fe
fix: fix import position to remove changes
Nov 21, 2022
93c7cd2
fix: remove unused script
Nov 21, 2022
5f8e5ee
fix: remove unnessecary changes in gitignore
Nov 21, 2022
c0a13b7
fix: remove luanch.json
Nov 21, 2022
55bacc5
fix: merge manifests conflicts
Nov 22, 2022
e8a9f87
fix: user logger correctly
Nov 22, 2022
3dbba7d
fix: pass env var for webhook pods selector
Nov 22, 2022
65fe7f0
test: test main flow of cer-generator
Nov 22, 2022
8ecc47c
fix: added deploy in makefile
Nov 27, 2022
7f36170
fix: bump images version in chart
Nov 27, 2022
9938af2
fix Makefile to deploy all binaries for awsmp. Ensure TOKEN isnt requ…
Nov 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
test: wip
  • Loading branch information
Noaa Barki authored and Noaa Barki committed Nov 21, 2022
commit f5593a7e800db4d9f97020058bc972c7950bac5d
14 changes: 7 additions & 7 deletions charts/datree-admission-webhook-awsmp/values.yaml
Original file line number Diff line number Diff line change
@@ -44,21 +44,21 @@ datree:
# The Datree webhook-server image to use.
image:
# Image repository
repository: 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/awsmp-datree-admission-webhook #localhost:5000/datree-admission-webhook
repository: localhost:5000/datree-admission-webhook #709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/awsmp-datree-admission-webhook
# Image tag
tag: 1.0-rc.2
# Image pull policy
pullPolicy: Always

initContainer:
repository: 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/datree-cert-generator #localhost:5000/cert-generator
tag: 0.1.1-rc.1
repository: localhost:5000/gen-cert #709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/datree-cert-generator
tag: 0.1.2-rc.1 #0.1.1-rc.1

imageWebhook:
# Image repository
repository: 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/datree-webhook-init #localhost:5000/webhook-init
repository: localhost:5000/webhook-init #709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/datree-webhook-init
# Image tag
tag: 0.1.1-rc.2 #1.0-rc.19
tag: 0.1.1-rc.2

# Security context for the containers
securityContext:
@@ -82,8 +82,8 @@ hooks:
timeoutTime:
# The image for running kubectl commands
image:
repository: 709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/bitnami-kubectl #bitnami/kubectl
sha: sha256:2c963c1aae87f544a53242348d64dce4bc39739211989394833322d29364b73c #sha256:1841cb16fddd9660c34f9b5bcec51f8c77bedba76c56381537753976b24649df
repository: bitnami/kubectl #709825985650.dkr.ecr.us-east-1.amazonaws.com/datree/bitnami-kubectl
sha: sha256:1841cb16fddd9660c34f9b5bcec51f8c77bedba76c56381537753976b24649df #sha256:2c963c1aae87f544a53242348d64dce4bc39739211989394833322d29364b73c
pullPolicy: IfNotPresent

# AWS Marketplace configuration
188 changes: 188 additions & 0 deletions cmd/webhook-init/k8s-client/k8sclient.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
package k8sclient

import (
"context"
"fmt"
"time"

"github.com/datreeio/admission-webhook-datree/cmd/webhook-init/utils"
"github.com/datreeio/admission-webhook-datree/pkg/loggerUtil"

v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"

admissionregistrationV1 "k8s.io/api/admissionregistration/v1"
ctrl "sigs.k8s.io/controller-runtime"
)

type K8sClient struct {
clientset kubernetes.Interface
}

func New(c kubernetes.Interface) *K8sClient {
if c != nil {
return &K8sClient{
clientset: c,
}
}

c, err := kubernetes.NewForConfig(ctrl.GetConfigOrDie())
if err != nil {
return nil
}

return &K8sClient{
clientset: c,
}

}

type ValidatingWebhookOpts struct {
MetaName string
CaBundle []byte
ServiceName string
Selector string
WebhookName string
}

func (k *K8sClient) CreateValidatingWebhookConfiguration(namespace string, cfg *ValidatingWebhookOpts) (*admissionregistrationV1.ValidatingWebhookConfiguration, error) {
if cfg == nil {
return nil, fmt.Errorf("invalid ValidatingWebhookOpts")
}

path := "/validate"
sideEffects := admissionregistrationV1.SideEffectClassNone

vw := &admissionregistrationV1.ValidatingWebhookConfiguration{
ObjectMeta: metav1.ObjectMeta{
Name: cfg.MetaName,
},
Webhooks: []admissionregistrationV1.ValidatingWebhook{{
Name: cfg.WebhookName,
ClientConfig: admissionregistrationV1.WebhookClientConfig{
CABundle: cfg.CaBundle, // CA bundle created earlier
Service: &admissionregistrationV1.ServiceReference{
Name: cfg.ServiceName, // datree-webhook-server
Namespace: namespace,
Path: &path,
},
},
Rules: []admissionregistrationV1.RuleWithOperations{{Operations: []admissionregistrationV1.OperationType{
admissionregistrationV1.Create,
admissionregistrationV1.Update,
},
Rule: admissionregistrationV1.Rule{
APIGroups: []string{"*"},
APIVersions: []string{"*"},
Resources: []string{"*"},
},
}},
SideEffects: &sideEffects,
AdmissionReviewVersions: []string{"v1", "v1beta1"},
TimeoutSeconds: &[]int32{30}[0],
NamespaceSelector: &metav1.LabelSelector{
MatchExpressions: []metav1.LabelSelectorRequirement{
{ // only validate pods in namespaces with the label "admission.datree/validate"
Key: cfg.Selector,
Operator: metav1.LabelSelectorOpDoesNotExist,
},
},
},
}},
}

vw, err := k.clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Create(context.Background(), vw, metav1.CreateOptions{})
if err != nil {
return nil, err
}

loggerUtil.Debugf("created validating webhook configuration: %v", vw)
return vw, nil
}

// search for validating webhook and delete if exists
func (k *K8sClient) DeleteExistingValidatingWebhook(name string) error {
vw := k.GetValidatingWebhookConfiguration(name)
if vw != nil {
return k.DeleteValidatingWebhookConfiguration(name)
}
return nil
}

func (k *K8sClient) DeleteValidatingWebhookConfiguration(name string) error {
return k.clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Delete(context.Background(), name, metav1.DeleteOptions{})
}

func (k *K8sClient) GetValidatingWebhookConfiguration(name string) *admissionregistrationV1.ValidatingWebhookConfiguration {
vw, err := k.clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Get(context.Background(), name, metav1.GetOptions{})
if (vw != nil && vw.Name == name) && err == nil {
return vw
}

return nil
}

func (k *K8sClient) CreatePodWatcher(ctx context.Context, namespace string, selector string) (watch.Interface, error) {
labelSelector := fmt.Sprint(selector)

opts := metav1.ListOptions{
TypeMeta: metav1.TypeMeta{},
LabelSelector: labelSelector,
FieldSelector: "",
}

return k.clientset.CoreV1().Pods(namespace).Watch(ctx, opts)
}

func (k *K8sClient) WaitUntilPodsAreRunning(ctx context.Context, namespace string, selector string, replicas int) error {
loggerUtil.Debugf("creating watcher for POD with label:%s ...", selector)
watcher, err := k.CreatePodWatcher(ctx, namespace, selector)
if err != nil {
return err
}

loggerUtil.Debug("watch out! Succuessfuly created watcher for PODs.")
defer watcher.Stop()

count := 0
for {
select {
case event := <-watcher.ResultChan():
pod := event.Object.(*v1.Pod)

if pod.Status.Phase == v1.PodRunning {
if k.IsPodReady(pod) {
count++
loggerUtil.Debugf("the POD \"%s\" is running", selector)
if count == replicas {
loggerUtil.Debug("all PODs are running")
return nil
}
}

}

case <-time.After(180 * time.Second):
loggerUtil.Debug("exit from waitPodRunning for POD \"%s\" because the time is over")
return nil

case <-ctx.Done():
loggerUtil.Debugf("exit from waitPodRunning for POD \"%s\" because the context is done", selector)
return nil
}
}
}

func (k *K8sClient) IsPodReady(pod *v1.Pod) bool {
checkPodReadyCondition := func(condition v1.PodCondition) bool {
return condition.Type == v1.PodReady && condition.Status == "True"
}

checkPodContainersCondition := func(condition v1.PodCondition) bool {
return condition.Type == v1.ContainersReady && condition.Status == "True"
}

return utils.FindIndex(checkPodReadyCondition, pod.Status.Conditions) > 0 && utils.FindIndex(checkPodContainersCondition, pod.Status.Conditions) > 0
}
91 changes: 91 additions & 0 deletions cmd/webhook-init/k8s-client/k8sclient_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
package k8sclient

import (
"testing"

"github.com/stretchr/testify/assert"
admissionregistrationV1 "k8s.io/api/admissionregistration/v1"
testclient "k8s.io/client-go/kubernetes/fake"
)

type condition[T any] struct {
compareFn func(actual T) bool
msg string
}

type output struct {
err *condition[error]
webhook *condition[*admissionregistrationV1.ValidatingWebhookConfiguration]
}

type input struct {
namespace string
cfg *ValidatingWebhookOpts
}

type test struct {
name string
args input
expected output
}

func TestCreateValidatingWebhookConfiguration(t *testing.T) {
tests := []test{
{
name: "should create validating webhook configuration",
args: input{
namespace: "datree",
cfg: &ValidatingWebhookOpts{
MetaName: "datree-webhook",
ServiceName: "webhook-server",
CaBundle: []byte("caBundle"),
Selector: "app=webhook-server",
WebhookName: "webhook-server.datree.svc",
}},
expected: output{
err: &condition[error]{compareFn: func(actual error) bool { return actual == nil }, msg: "should not return error"},
webhook: &condition[*admissionregistrationV1.ValidatingWebhookConfiguration]{compareFn: func(actual *admissionregistrationV1.ValidatingWebhookConfiguration) bool {
return actual != nil && actual.Name == "datree-webhook" && actual.Webhooks[0].Name == "webhook-server.datree.svc" && actual.Webhooks[0].ClientConfig.Service.Name == "webhook-server" && actual.Webhooks[0].ClientConfig.Service.Namespace == "datree" && actual.Webhooks[0].ClientConfig.Service.Path != nil && *actual.Webhooks[0].ClientConfig.Service.Path == "/validate" && actual.Webhooks[0].ClientConfig.CABundle != nil && string(actual.Webhooks[0].ClientConfig.CABundle) == "caBundle"
}, msg: "should return webhook configuration"},
},
},
{
name: "should not return error when namespace is empty",
args: input{
namespace: "",
cfg: &ValidatingWebhookOpts{
MetaName: "datree-webhook",
ServiceName: "webhook-server",
CaBundle: []byte("caBundle"),
Selector: "app=webhook-server",
WebhookName: "webhook-server.datree.svc",
},
},
expected: output{err: &condition[error]{compareFn: func(actual error) bool { return actual == nil }, msg: "should return error"}},
},
{
name: "should return error when cfg is nil",
args: input{
namespace: "datree",
cfg: nil,
},
expected: output{err: &condition[error]{compareFn: func(actual error) bool { return actual != nil }, msg: "should return error"}},
},
}

for _, ts := range tests {
t.Run(ts.name, func(t *testing.T) {
client := testclient.NewSimpleClientset()

k8sClient := New(client)

actualVW, actualErr := k8sClient.CreateValidatingWebhookConfiguration(ts.args.namespace, ts.args.cfg)
if ts.expected.err != nil {
assert.Condition(t, func() bool { return ts.expected.err.compareFn(actualErr) }, ts.expected.err.msg)
}
if ts.expected.webhook != nil {
assert.Condition(t, func() bool { return ts.expected.webhook.compareFn(actualVW) }, ts.expected.webhook.msg)
}
})
}
}
Loading
Oops, something went wrong.