Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addressing request to extend testing options for functional-test automation #45

Merged
merged 6 commits into from
Sep 30, 2020

Conversation

ilrudie
Copy link
Contributor

@ilrudie ilrudie commented Sep 29, 2020

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind bug
/kind cleanup
/kind deprecation

/kind design

/kind documentation
/kind feature

What this PR does / why we need it:
The PR addresses a request to allow running code specific to a test or set of tests as required to setup, teardown or reset (run in between tests) the environment. This would allow arbitrary adjustment to the test namespace such that tests pass/fail as expected. An example might be adding a label which a policy looks for before allowing/denying the request.

To achieve this the functional-tests.yaml format was significantly altered. There is no backward compatibility to the old format however the existing tests were copied over into the new format to ensure they continue to run in CI.

Which issue(s) this PR fixes:

Fixes #38

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE


Additional documentation e.g., usage docs, etc.:
This gist has been created as an example/template for how to have code run by the functional testing framework if your tests require it. It is optional.

If you do not require any special scripts to adjust the environment for your tests you may leave the script field blank in the manifest or omit the script: line entirely and the functional testing will run as it did prior to this PR.

The script should be placed in testing/resource/scripts/user_script and then be reference in the yaml by name not by full path.

example: for a deployment script you might have a script like testing/deployments/scripts/deploy-pass.sh and the resulting manifest should look like:

resources:
  - kind: deployments
    desired: pass
    script: deploy-pass.sh
    manifests:
      - test-deploy01.yaml
      - test-deploy03.yaml

@ilrudie ilrudie marked this pull request as ready for review September 29, 2020 01:51
Copy link
Contributor

@phenixblue phenixblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Looks mostly good. A couple of generic cleanup things and some consistency/style nits that I've pointed out in the comments. Looks like you need to rebase as well.

hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved

else
#check if we're doing all desired results or if the resoured desired result at $i matches the requested desired result
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consistency nit, space after # for comments.

hack/run-functional-tests.sh Outdated Show resolved Hide resolved
hack/run-functional-tests.sh Outdated Show resolved Hide resolved
Copy link
Contributor

@phenixblue phenixblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a rebase, but otherwise LGTM

@phenixblue
Copy link
Contributor

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extend testing options for functional-test automation
2 participants