test: new test framework for AKV provider #242

aramase · 2020-09-28T06:33:47Z

Reason for Change:

Add new test framework based on ginkgo
Uses e2eteam/busybox:1.29 for testing in windows. This is a smaller image.
New test framework reduces the tests run time from 45m to ~25m for windows clusters.
This framework will also allow us to setup soak tests.

Follow up after PR is merged:

Add doc for how to run E2E tests locally
Add script for setting up all artifacts required for tests (cc @bdlb77)
Remove bats tests from the repo

Requirements

squashed commits
included documentation
added unit tests and e2e tests (if applicable).

Issue Fixed:

Please answer the following questions with yes/no:

Does this change contain code from or inspired by another project? If so, did you notify the maintainers and provide attribution?

Special Notes for Reviewers:

aramase · 2020-09-28T16:50:01Z

@bdlb77 This is the new test framework I was working on last week. PTAL when you get a chance!

bdlb77 · 2020-09-29T17:20:28Z

@aramase Will do! For splitting #45 Would you say for the testing section I should wait for this to be merged before making the PR to update the tests for local / remote container development or open a PR using bats and then a follow-up which will add any possible tests needed to make it compatible in Golang test suite?

aramase · 2020-09-30T16:09:44Z

@aramase Will do! For splitting #45 Would you say for the testing section I should wait for this to be merged before making the PR to update the tests for local / remote container development or open a PR using bats and then a follow-up which will add any possible tests needed to make it compatible in Golang test suite?

@bdlb77 I think we can start with PR using bats and then follow up with a new PR for the golang test suite. WDYT?

bdlb77 · 2020-09-30T21:13:59Z

@bdlb77 I think we can start with PR using bats and then follow up with a new PR for the golang test suite. WDYT?

@aramase sounds good!

test/e2e/framework/certificates/certificates.go

test/e2e/framework/convenience.go

bdlb77

looks really good and awesome to see the test suite in Go! Just have a couple of questions mainly for clarity :)

aristosvo · 2020-10-29T18:41:01Z

Just curious: these e2e tests still require a Azure KeyVault filled with secrets, keys and certs, right?

aramase · 2020-10-29T18:45:15Z

Just curious: these e2e tests still require a Azure KeyVault filled with secrets, keys and certs, right?

That's right. @bdlb77 has a script in a branch that can be run to initially setup all the required artifacts for testing. Once we get this framework change updated and merged, we can add the scripts as part of the repo.

aramase · 2021-01-06T22:12:33Z

test/e2e/framework/helm/helm.go

+		"install",
+		podIdentityChartName,
+		fmt.Sprintf("--namespace=%s", framework.NamespaceKubeSystem),
+		"aad-pod-identity/aad-pod-identity", "--set", "nmi.allowNetworkPluginKubenet=true", "--wait", "--timeout=5m", "--debug",


Current test clusters run kubenet. I'll update the cluster config in a separate PR to use Azure CNI and then remove this configuration from testing.

coverage.txt

ritazh · 2021-01-25T04:48:25Z

test/e2e/auto_rotation_test.go

+					"objects":                string(objects),
+					"usePodIdentity":         "false",
+					"useVMManagedIdentity":   "true",
+					"userAssignedIdentityID": config.UserAssignedIdentityID,


Where do you grant permissions to the kv resources for this identity?

Permissions and role assignments aren't done as part of the test suite. They should be done out of band as part of e2e setup.

In a followup PR, can you pls document prerequisite steps (or a script) for how someone can setup the e2e?

sounds good! this is the issue #151 to track the script and docs.

ritazh · 2021-01-25T05:13:11Z

test/e2e/framework/secretproviderclass/secretproviderclass.go

+	Config    *framework.Config
+	Name      string
+	Namespace string
+	Spec      v1alpha1.SecretProviderClassSpec


Might be good to create a json template for the spec to catch regressions for all the expected values in parameters. WDYT?

Based on offline discussion:

We'll keep the examples dir which will contain sample manifests for each supported configuration.

For schema validation, we'll need to ensure with thorough unit testing that the AKV provider handles all expected values.

test/e2e/framework/pod/pod.go

codecov-io · 2021-01-25T22:42:06Z

Codecov Report

Merging #242 (cbf5d72) into master (a87ef7e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #242   +/-   ##
=======================================
  Coverage   62.50%   62.50%           
=======================================
  Files           6        6           
  Lines         472      472           
=======================================
  Hits          295      295           
  Misses        145      145           
  Partials       32       32

.pipelines/e2e-tests-template.yaml

azure-pipelines.yml

sozercan · 2021-02-05T18:50:40Z

test/e2e/Makefile

+
+.PHONY: run
+run:
+	cd $(TEST_E2E_DIR); GODEBUG=x509ignoreCN=0 go test -tags=e2e -timeout=90m -v -ginkgo.v \


does tests require x509ignoreCN=0?

The x509: certificate relies on legacy Common Name field. If we use SANs in the certificate, then we don't need this. With the current artifacts we have in keyvault for testing we enable Common Name matching with GODEBUG=x509ignoreCN=0 to resolve the error.

refactor certs util for tests add test for public key configure pod identity test

aramase · 2021-02-05T21:23:30Z

/azp run pr-e2e-azure

azure-pipelines · 2021-02-05T21:23:47Z

Azure Pipelines successfully started running 1 pipeline(s).

aramase added the do-not-merge/work-in-progress label Sep 28, 2020

aramase force-pushed the go-tests branch 10 times, most recently from c551d59 to e913374 Compare September 28, 2020 16:38

bdlb77 reviewed Oct 2, 2020

View reviewed changes

test/e2e/framework/certificates/certificates.go Show resolved Hide resolved

bdlb77 reviewed Oct 2, 2020

View reviewed changes

test/e2e/framework/convenience.go Outdated Show resolved Hide resolved

bdlb77 reviewed Oct 2, 2020

View reviewed changes

mitsutaka mentioned this pull request Oct 5, 2020

Replace e2e test with onsi/ginkgo kubernetes-sigs/secrets-store-csi-driver#245

Closed

aramase added this to In progress in Secret Store CSI Azure Provider Roadmap Oct 6, 2020

aramase added the test label Oct 21, 2020

aramase force-pushed the go-tests branch from e913374 to 773372d Compare November 12, 2020 00:30

aramase added this to the 0.0.12 milestone Dec 11, 2020

aramase force-pushed the go-tests branch 2 times, most recently from 2ad538e to b12403e Compare December 17, 2020 22:18

aramase force-pushed the go-tests branch 3 times, most recently from 5782572 to 8c9db7a Compare January 5, 2021 17:43

aramase force-pushed the go-tests branch from 2de72ca to 492ad6d Compare January 6, 2021 21:57

aramase changed the title ~~[WIP] update test framework~~ update to new test framework Jan 6, 2021

aramase removed the do-not-merge/work-in-progress label Jan 6, 2021

aramase changed the title ~~update to new test framework~~ test: new test framework for AKV provider Jan 6, 2021

aramase commented Jan 6, 2021

View reviewed changes

aramase requested a review from ritazh January 6, 2021 22:25

aramase requested a review from sozercan January 20, 2021 18:05

aramase modified the milestones: 0.0.12, 0.0.13 Jan 20, 2021

ritazh reviewed Jan 25, 2021

View reviewed changes

coverage.txt Outdated Show resolved Hide resolved

ritazh reviewed Jan 25, 2021

View reviewed changes

test/e2e/framework/pod/pod.go Outdated Show resolved Hide resolved

aramase force-pushed the go-tests branch 3 times, most recently from fccd4ef to 4b5c081 Compare January 25, 2021 22:38

aramase force-pushed the go-tests branch from 4b5c081 to 54dd11b Compare January 26, 2021 23:02

aramase requested a review from ritazh January 26, 2021 23:03

sozercan reviewed Feb 5, 2021

View reviewed changes

aramase force-pushed the go-tests branch from cbf5d72 to c819ad4 Compare February 5, 2021 21:04

add new test framework

d2445bb

refactor certs util for tests add test for public key configure pod identity test

aramase force-pushed the go-tests branch from c819ad4 to d2445bb Compare February 5, 2021 21:04

aramase requested a review from sozercan February 5, 2021 22:17

ritazh approved these changes Feb 5, 2021

View reviewed changes

aramase merged commit 584551e into Azure:master Feb 5, 2021

Secret Store CSI Azure Provider Roadmap automation moved this from In progress to Done Feb 5, 2021

aramase deleted the go-tests branch February 5, 2021 23:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: new test framework for AKV provider #242

test: new test framework for AKV provider #242

aramase commented Sep 28, 2020 •

edited

aramase commented Sep 28, 2020

bdlb77 commented Sep 29, 2020

aramase commented Sep 30, 2020

bdlb77 commented Sep 30, 2020 •

edited

bdlb77 left a comment

aristosvo commented Oct 29, 2020

aramase commented Oct 29, 2020

aramase Jan 6, 2021 •

edited

ritazh Jan 25, 2021

aramase Jan 25, 2021

ritazh Feb 5, 2021

aramase Feb 5, 2021

ritazh Jan 25, 2021

aramase Jan 26, 2021

codecov-io commented Jan 25, 2021 •

edited

sozercan Feb 5, 2021

aramase Feb 5, 2021

aramase commented Feb 5, 2021

azure-pipelines bot commented Feb 5, 2021

test: new test framework for AKV provider #242

test: new test framework for AKV provider #242

Conversation

aramase commented Sep 28, 2020 • edited

aramase commented Sep 28, 2020

bdlb77 commented Sep 29, 2020

aramase commented Sep 30, 2020

bdlb77 commented Sep 30, 2020 • edited

bdlb77 left a comment

Choose a reason for hiding this comment

aristosvo commented Oct 29, 2020

aramase commented Oct 29, 2020

aramase Jan 6, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Jan 25, 2021 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aramase commented Feb 5, 2021

azure-pipelines bot commented Feb 5, 2021

aramase commented Sep 28, 2020 •

edited

bdlb77 commented Sep 30, 2020 •

edited

aramase Jan 6, 2021 •

edited

codecov-io commented Jan 25, 2021 •

edited