Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dat feat awsmp sleek rebase #193

Draft
wants to merge 85 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
e7990ea
feat: new chart for aws marketplace product. The product supports lic…
Sep 6, 2022
6397cdd
fix: qoute productSKU value; add logs for checkout license
Sep 6, 2022
b658876
refactor: upgrade values.yaml with updated image pushed to aws mp pro…
Sep 6, 2022
0640270
fix: support region, product, fingerprint and debug env variables; re…
Sep 7, 2022
fdc41c2
fix: run license cron job on @daily. fix values naming bug
Sep 7, 2022
8ebf020
fix: upgrade image in aws mp helm chart values
Sep 7, 2022
17e6251
fix: remove awsmp-... chart. datree-admission-webhook chart should ho…
Sep 12, 2022
e0a348b
fix: add * to ignore luanch.json file
Sep 12, 2022
5289408
fix: remove ** to ignore luanch.json file
Sep 12, 2022
275e001
fix: accidently deleted gh-pages values file with awsmp values file. …
Sep 21, 2022
eeaa96b
fix: added aws.values
Sep 21, 2022
5e73163
feat: added validation for value.yaml file
Sep 12, 2022
2abeacc
fix: combine charts and added values.yaml for each option
Sep 19, 2022
cc12656
feat: script for packaing
Sep 19, 2022
2ccc1f3
fix: added script package for free-datree
Sep 19, 2022
595a75e
feat: added chart
Sep 20, 2022
42fbd5e
feat: charts file structure
Sep 20, 2022
f896fba
fix: combine charts and added values.yaml for each option
Sep 19, 2022
6489875
feat: script for packaing
Sep 19, 2022
150fba7
fix: added script package for free-datree
Sep 19, 2022
002513b
feat: added chart
Sep 20, 2022
f650e64
fix: move tempaltes to lib template
shmu3l Sep 28, 2022
448011f
fix: remove templates use include from lib
shmu3l Sep 28, 2022
b43ef0f
fix: remove templates use include from lib
shmu3l Sep 28, 2022
771c57a
fix: add dynamic webhook server alt name to cert
shmu3l Oct 2, 2022
2002a8d
fix: clear datree webhook values and update schema
shmu3l Oct 2, 2022
9b446e7
feat: awsmp datree use datree lib
shmu3l Oct 2, 2022
af85f57
fix: awsmp file structure
shmu3l Oct 2, 2022
d1ef809
feat: add release admission webhook script
shmu3l Oct 3, 2022
121763b
fix: release admission webhook
shmu3l Oct 3, 2022
37a828d
release chart 0.1.3
shmu3l Oct 3, 2022
8ddb66f
fix: release admission webhook helm script update
shmu3l Oct 6, 2022
23a42d7
fix: update .gitignore
Oct 6, 2022
30c96e4
fix: change messages
Oct 6, 2022
572c8c5
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
a13fabc
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
57a809a
fix: release admission webhook path scripts
shmu3l Oct 6, 2022
b79e351
fix: rename script
Oct 6, 2022
edfa70f
fix: change the order of helm file creations
Oct 6, 2022
88d5416
fix: remove files
Oct 6, 2022
7e60bf9
fix: release file index yaml
Oct 6, 2022
2d54188
fix: dont stash pop
Oct 6, 2022
56e619f
feat: certificate alternative names use datree.namespace template.
Oct 18, 2022
c02ba66
fix: bump lib chart version
Oct 18, 2022
7132337
fix: added dynamic namespace installation. Rename chart according to …
Oct 24, 2022
883f25c
fix: aligned charts with aws-marketplace ECR repositories.
Nov 3, 2022
d4c0867
feat: pull bitnami-kubectl from private ecr registry
Nov 3, 2022
a09986d
fix: updated bitnami in free offer chart
Nov 3, 2022
378ef6a
feat: added new binary in cmd folder for init container
Nov 7, 2022
a5e0e21
wip
Nov 8, 2022
7d9411c
wip: poc of webhook race condition
Nov 9, 2022
076b597
fix: working product on minikube. NOTE: very slow
Nov 10, 2022
d4cd131
fix: working version
Nov 13, 2022
a1d8ba3
fix: wokring production version on minikube
Nov 13, 2022
4e504da
fix: working production version. Minikube. 3minutes sleep time
Nov 13, 2022
956b584
fix: working version ECR repositories on Faragate.
Nov 13, 2022
3cfe527
feat: support uninstall Datree product
Nov 13, 2022
d2eab5f
fix: working version in AWS Marketplace. Datree product
Nov 13, 2022
58e430c
feat: working version. Datree Free product. version 1.0.1-rc.1
Nov 13, 2022
7ba108f
feat: check for webhook existence, wait for running pods. Refactor
Nov 15, 2022
1f1d134
fix: wait for all deployment replicas to be ready. updated ecr images
Nov 15, 2022
07f0a82
fix: update aws mp Chart
Nov 16, 2022
d51dfde
fix: remove .vscode luanch.json
Nov 16, 2022
cb90d7b
fix: update Dockerfiles
Nov 16, 2022
c3a79b2
fix: remove comments
Nov 16, 2022
09fc506
test: added tests for cert-generator
Nov 16, 2022
f5593a7
test: wip
Nov 20, 2022
4ccf30e
test: fixed table testing for k8sClient
Nov 20, 2022
8161364
test: cert-generator tests
Nov 21, 2022
1e4697b
fix: structure packages in webhook-init to not use one another, only …
Nov 21, 2022
a550115
fix: updated makefile
Nov 21, 2022
52ce8b7
test: ensure empty env variables has defaults
Nov 21, 2022
4b57836
fix: bump images versions, working product on minikube.
Nov 21, 2022
f893b31
fix: make code build succesfully
Nov 21, 2022
b3867fe
fix: fix import position to remove changes
Nov 21, 2022
93c7cd2
fix: remove unused script
Nov 21, 2022
5f8e5ee
fix: remove unnessecary changes in gitignore
Nov 21, 2022
c0a13b7
fix: remove luanch.json
Nov 21, 2022
55bacc5
fix: merge manifests conflicts
Nov 22, 2022
e8a9f87
fix: user logger correctly
Nov 22, 2022
3dbba7d
fix: pass env var for webhook pods selector
Nov 22, 2022
65fe7f0
test: test main flow of cer-generator
Nov 22, 2022
8ecc47c
fix: added deploy in makefile
Nov 27, 2022
7f36170
fix: bump images version in chart
Nov 27, 2022
9938af2
fix Makefile to deploy all binaries for awsmp. Ensure TOKEN isnt requ…
Nov 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: make code build succesfully
  • Loading branch information
Noaa Barki authored and Noaa Barki committed Nov 21, 2022
commit f893b31b8afee2b08bddce5e399ed8b08a3197c3
12 changes: 6 additions & 6 deletions cmd/cert-generator/main.go
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import (
"os"

"github.com/datreeio/admission-webhook-datree/cmd/cert-generator/renewer"
"github.com/datreeio/admission-webhook-datree/pkg/loggerUtil"
"github.com/datreeio/admission-webhook-datree/pkg/logger"
)

type fileWriter struct{}
@@ -28,29 +28,29 @@ func (fw *fileWriter) WriteFile(filepath string, sCert *bytes.Buffer) error {
func main() {
tlsDir, isFound := os.LookupEnv("WEBHOOK_CERTS_DIR")
if !isFound {
loggerUtil.Log("required directory for certificates is missing, verify env varaible WEBHOOK_CERTS_DIR in deployment")
logger.LogUtil("required directory for certificates is missing, verify env varaible WEBHOOK_CERTS_DIR in deployment")
return
}

err := os.MkdirAll(tlsDir, 0666)
if err != nil {
loggerUtil.Log(err.Error())
logger.LogUtil(err.Error())
return
}

renewer := renewer.NewCertRenewer(&fileWriter{})

caPrivKey, caCert, err := renewer.RenewCA(tlsDir)
if err != nil {
loggerUtil.Log(err.Error())
logger.LogUtil(err.Error())
return
}

err = renewer.RenewTLS(tlsDir, caCert, caPrivKey)
if err != nil {
loggerUtil.Log(err.Error())
logger.LogUtil(err.Error())
return
}

loggerUtil.Log("horray! successfully generated self-signed CA and signed webhook server certificate using this CA!")
logger.LogUtil("horray! successfully generated self-signed CA and signed webhook server certificate using this CA!")
}
17 changes: 8 additions & 9 deletions cmd/init-webhook/k8s-client/k8sclient.go
Original file line number Diff line number Diff line change
@@ -5,8 +5,7 @@ import (
"fmt"
"time"

"github.com/datreeio/admission-webhook-datree/pkg/loggerUtil"

"github.com/gardener/controller-manager-library/pkg/logger"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
@@ -97,7 +96,7 @@ func (k *K8sClient) CreateValidatingWebhookConfiguration(namespace string, cfg *
return nil, err
}

loggerUtil.Debugf("created validating webhook configuration: %v", vw)
logger.Debugf("created validating webhook configuration: %v", vw)
return vw, nil
}

@@ -136,13 +135,13 @@ func (k *K8sClient) CreatePodWatcher(ctx context.Context, namespace string, sele
}

func (k *K8sClient) WaitUntilPodsAreRunning(ctx context.Context, namespace string, selector string, replicas int) error {
loggerUtil.Debugf("creating watcher for POD with label:%s ...", selector)
logger.Debugf("creating watcher for POD with label:%s ...", selector)
watcher, err := k.CreatePodWatcher(ctx, namespace, selector)
if err != nil {
return err
}

loggerUtil.Debug("watch out! Succuessfuly created watcher for PODs.")
logger.Debug("watch out! Succuessfuly created watcher for PODs.")
defer watcher.Stop()

count := 0
@@ -154,21 +153,21 @@ func (k *K8sClient) WaitUntilPodsAreRunning(ctx context.Context, namespace strin
if pod.Status.Phase == v1.PodRunning {
if k.IsPodReady(pod) {
count++
loggerUtil.Debugf("the POD \"%s\" is running", selector)
logger.Debugf("the POD \"%s\" is running", selector)
if count == replicas {
loggerUtil.Debug("all PODs are running")
logger.Debug("all PODs are running")
return nil
}
}

}

case <-time.After(180 * time.Second):
loggerUtil.Debug("exit from waitPodRunning for POD \"%s\" because the time is over")
logger.Debug("exit from waitPodRunning for POD \"%s\" because the time is over")
return nil

case <-ctx.Done():
loggerUtil.Debugf("exit from waitPodRunning for POD \"%s\" because the context is done", selector)
logger.Debugf("exit from waitPodRunning for POD \"%s\" because the context is done", selector)
return nil
}
}
13 changes: 7 additions & 6 deletions cmd/init-webhook/main.go
Original file line number Diff line number Diff line change
@@ -2,10 +2,11 @@ package main

import (
"context"
"fmt"

k8sclient "github.com/datreeio/admission-webhook-datree/cmd/init-webhook/k8s-client"
webhookinfo "github.com/datreeio/admission-webhook-datree/cmd/init-webhook/webhook-info"
"github.com/datreeio/admission-webhook-datree/pkg/loggerUtil"
"github.com/datreeio/admission-webhook-datree/pkg/logger"
admissionregistrationV1 "k8s.io/api/admissionregistration/v1"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/watch"
@@ -14,14 +15,14 @@ import (
func main() {
k8sClient := k8sclient.New(nil)
if k8sClient == nil {
loggerUtil.Logf("failed to set k8s go -client, err")
logger.LogUtil("failed to set k8s go -client, err")
}

err := InitWebhook(k8sClient)
if err != nil {
loggerUtil.Logf("failed to init webhook, err: %v", err)
logger.LogUtil(fmt.Sprint("failed to init webhook, err: %v", err))
}
loggerUtil.Log("horray! succesfully created datree validating admission webhook")
logger.LogUtil("horray! succesfully created datree validating admission webhook")

// wait forever to prevent the container from restrating
waitForever()
@@ -39,13 +40,13 @@ type k8sClientInterface interface {
func InitWebhook(k8sClient k8sClientInterface) error {
err := k8sClient.DeleteExistingValidatingWebhook("datree-webhook")
if err != nil {
loggerUtil.Logf("failed to delete existed validation webhook config, err: %v", err)
logger.LogUtil(fmt.Sprint("failed to delete existed validation webhook config, err: %v", err))
return err
}

err = k8sClient.WaitUntilPodsAreRunning(context.Background(), webhookinfo.GetWebhookNamespace(), webhookinfo.GetWebhookSelector(), webhookinfo.GetWebhookServerReplicas())
if err != nil {
loggerUtil.Logf("failed to wait for pods, err: %v", err)
logger.LogUtil(fmt.Sprint("failed to wait for pods, err: %v", err))
return err
}

13 changes: 7 additions & 6 deletions cmd/init-webhook/webhook-info/webhookInfo.go
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
package webhookinfo

import (
"fmt"
"os"
"path/filepath"
"strconv"

"github.com/datreeio/admission-webhook-datree/pkg/loggerUtil"
"github.com/datreeio/admission-webhook-datree/pkg/logger"
)

func GetWebhookServiceName() string {
webhookServiceName, isFound := os.LookupEnv("WEBHOOK_SERVICE")
if !isFound || webhookServiceName == "" {
loggerUtil.Log("required environment variable WEBHOOK_SERVICE is missing")
logger.LogUtil("required environment variable WEBHOOK_SERVICE is missing")
return "datree-webhook-server"
}

@@ -21,7 +22,7 @@ func GetWebhookServiceName() string {
func GetWebhookNamespace() string {
webhookNamespace, isFound := os.LookupEnv("WEBHOOK_NAMESPACE")
if !isFound || webhookNamespace == "" {
loggerUtil.Log("required environment variable WEBHOOK_NAMESPACE is missing")
logger.LogUtil("required environment variable WEBHOOK_NAMESPACE is missing")
return "datree"
}

@@ -31,7 +32,7 @@ func GetWebhookNamespace() string {
func GetWebhookSelector() string {
webhookSelector, isFound := os.LookupEnv("WEBHOOK_SELECTOR")
if !isFound || webhookSelector == "" {
loggerUtil.Log("required environment variable WEBHOOK_SELECTOR is missing")
logger.LogUtil("required environment variable WEBHOOK_SELECTOR is missing")
return "admission.datree/validate"
}

@@ -51,12 +52,12 @@ func GetWebhookCABundle() ([]byte, error) {
func GetWebhookServerReplicas() int {
replicasStr, isFound := os.LookupEnv("WEBHOOK_SERVER_REPLICAS")
if !isFound || replicasStr == "" {
loggerUtil.Log("required environment variable WEBHOOK_SERVER_REPLICAS is missing")
logger.LogUtil("required environment variable WEBHOOK_SERVER_REPLICAS is missing")
return 2
}
replicas, err := strconv.Atoi(replicasStr)
if err != nil {
loggerUtil.Logf("invalid value for env variable WEBHOOK_SERVER_REPLICAS, err: %v", err)
logger.LogUtil(fmt.Sprintf("invalid value for env variable WEBHOOK_SERVER_REPLICAS, err: %v", err))
return 2
}

64 changes: 2 additions & 62 deletions cmd/webhook-server/main.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
package main

import (
"context"
"fmt"

"github.com/datreeio/admission-webhook-datree/pkg/config"
"github.com/datreeio/admission-webhook-datree/pkg/logger"
"github.com/datreeio/admission-webhook-datree/pkg/services"
"github.com/robfig/cron/v3"
"k8s.io/client-go/kubernetes"

"net/http"
"os"
@@ -17,7 +15,6 @@ import (
"github.com/datreeio/admission-webhook-datree/pkg/controllers"
"github.com/datreeio/admission-webhook-datree/pkg/errorReporter"
"github.com/datreeio/admission-webhook-datree/pkg/k8sMetadataUtil"
"github.com/datreeio/admission-webhook-datree/pkg/loggerUtil"
"github.com/datreeio/admission-webhook-datree/pkg/server"
"github.com/datreeio/datree/pkg/cliClient"
"github.com/datreeio/datree/pkg/deploymentConfig"
@@ -50,7 +47,7 @@ func start(port string) {
}
}()

loggerUtil.Log("initializing k8s metadata")
logger.LogUtil("initializing k8s metadata")
k8sMetadataUtil.InitK8sMetadataUtil()
initMetadataLogsCronjob()
server.InitServerVars()
@@ -71,7 +68,7 @@ func start(port string) {
// start server
err = http.ListenAndServeTLS(":"+port, certPath, keyPath, nil)
if err != nil {
loggerUtil.Log(err.Error())
logger.LogUtil(err.Error())
http.ListenAndServe(":"+port, nil)
}
}
@@ -81,60 +78,3 @@ func initMetadataLogsCronjob() {
cornJob.AddFunc("@every 1h", services.SendMetadataInBatch)
cornJob.Start()
}

func createValidationWebhookConfig(caCert []byte) error {
config := ctrl.GetConfigOrDie()
kubeClient, err := kubernetes.NewForConfig(config)
if err != nil {
return err // panic("failed to set go -client")
}
// webhookNamespace, _ := os.LookupEnv("WEBHOOK_NAMESPACE")
validationCfgName := "datree-webhook"

path := "/validate"
sideEffects := admissionregistrationv1.SideEffectClassNone

validationWebhookConfig := &admissionregistrationv1.ValidatingWebhookConfiguration{
ObjectMeta: metav1.ObjectMeta{
Name: validationCfgName,
},
Webhooks: []admissionregistrationv1.ValidatingWebhook{{
Name: "webhook-server.datree.svc",
ClientConfig: admissionregistrationv1.WebhookClientConfig{
CABundle: caCert, // CA bundle created earlier
Service: &admissionregistrationv1.ServiceReference{
Name: "datree-webhook-server", // datree-webhook-server
Namespace: "datree",
Path: &path,
},
},
Rules: []admissionregistrationv1.RuleWithOperations{{Operations: []admissionregistrationv1.OperationType{
admissionregistrationv1.Create,
admissionregistrationv1.Update,
},
Rule: admissionregistrationv1.Rule{
APIGroups: []string{"*"},
APIVersions: []string{"*"},
Resources: []string{"*"},
},
}},
SideEffects: &sideEffects,
AdmissionReviewVersions: []string{"v1", "v1beta1"},
TimeoutSeconds: &[]int32{30}[0],
NamespaceSelector: &metav1.LabelSelector{
MatchExpressions: []metav1.LabelSelectorRequirement{
{ // only validate pods in namespaces with the label "admission.datree/validate"
Key: "admission.datree/validate",
Operator: metav1.LabelSelectorOpDoesNotExist,
},
},
},
}},
}

if _, err = kubeClient.AdmissionregistrationV1().ValidatingWebhookConfigurations().Create(context.Background(), validationWebhookConfig, metav1.CreateOptions{}); err != nil {
return err
}

return nil
}
16 changes: 10 additions & 6 deletions go.mod
Original file line number Diff line number Diff line change
@@ -4,7 +4,9 @@ go 1.18

require (
github.com/datreeio/datree v1.6.6
github.com/gardener/controller-manager-library v0.2.0
github.com/ghodss/yaml v1.0.0
github.com/go-openapi/runtime v0.24.2
github.com/lithammer/shortuuid v3.0.0+incompatible
github.com/stretchr/testify v1.8.0
go.uber.org/zap v1.21.0
@@ -29,13 +31,15 @@ require (
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.7.3 // indirect
github.com/stretchr/objx v0.1.1 // indirect
github.com/stretchr/objx v0.4.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
k8s.io/apiextensions-apiserver v0.25.0 // indirect
k8s.io/component-base v0.25.0 // indirect
)

require (
github.com/Masterminds/semver v1.4.2 // indirect
github.com/sirupsen/logrus v1.8.1 // indirect
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.6.0 // indirect
)
@@ -53,8 +57,8 @@ require (
github.com/go-logr/logr v1.2.3 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.19.5 // indirect
github.com/go-openapi/swag v0.19.14 // indirect
github.com/go-openapi/jsonreference v0.19.6 // indirect
github.com/go-openapi/swag v0.21.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/gnostic v0.5.7-v3refs // indirect
@@ -67,16 +71,16 @@ require (
github.com/kyokomi/emoji v2.2.4+incompatible // indirect
github.com/lufia/plan9stats v0.0.0-20220517141722-cf486979b281 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/mailru/easyjson v0.7.6 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/mattn/go-isatty v0.0.14 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/mitchellh/mapstructure v1.4.1 // indirect
github.com/mitchellh/mapstructure v1.4.3 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/pelletier/go-toml v1.2.0 // indirect
github.com/pelletier/go-toml v1.7.0 // indirect
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
Loading
Oops, something went wrong.