Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gk-test] Run case #1446

Merged
merged 4 commits into from Jul 27, 2021
Merged

[gk-test] Run case #1446

merged 4 commits into from Jul 27, 2021

Conversation

willbeason
Copy link
Member

Depends on #1445.

Add logic for running individual test cases. Now that we've compiled the
constraints, all we need to do is Review() the specified objects and
compare the Results with what the user specified.

This PR does not add Assertion functionality - we'll do that for alpha,
not pre-alpha.

Also add unit tests that verify we are, in fact, running the test cases.

@willbeason willbeason added the authorship-tooling Tools for authors of Gatekeeper Constraints, Templates, Mutators label Jul 15, 2021
@willbeason willbeason added this to the gk-test cli alpha milestone Jul 15, 2021
@willbeason willbeason self-assigned this Jul 15, 2021
@codecov-commenter
Copy link

codecov-commenter commented Jul 15, 2021

Codecov Report

Merging #1446 (5e4032d) into master (ff826b2) will increase coverage by 0.23%.
The diff coverage is 66.88%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1446      +/-   ##
==========================================
+ Coverage   51.64%   51.87%   +0.23%     
==========================================
  Files          78       80       +2     
  Lines        7288     7429     +141     
==========================================
+ Hits         3764     3854      +90     
- Misses       3172     3206      +34     
- Partials      352      369      +17     
Flag Coverage Δ
unittests 51.87% <66.88%> (+0.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/gktest/printer_go.go 49.35% <49.35%> (ø)
pkg/gktest/result.go 72.72% <72.72%> (ø)
pkg/gktest/runner.go 90.47% <85.36%> (-4.98%) ⬇️
pkg/gktest/filter.go 66.66% <100.00%> (ø)
pkg/gktest/read_constraints.go 83.60% <100.00%> (+0.55%) ⬆️
pkg/watch/replay.go 79.21% <0.00%> (-2.25%) ⬇️
pkg/readiness/list.go 91.17% <0.00%> (+11.76%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff826b2...5e4032d. Read the comment docs.

@willbeason willbeason changed the title Run case [gk-test] Run case Jul 15, 2021
willbeason pushed a commit to willbeason/gatekeeper that referenced this pull request Jul 15, 2021
This lets us read Suites from the disk.

Per open-policy-agent#1446, we aren't doing Assertions for pre-alpha so I haven't added
those annotations yet.

Signed-off-by: Will Beason <willbeason@google.com>
willbeason pushed a commit to willbeason/gatekeeper that referenced this pull request Jul 15, 2021
This lets us read Suites from the disk.

Per open-policy-agent#1446, we aren't doing Assertions for pre-alpha so I haven't added
those annotations yet.

Signed-off-by: Will Beason <willbeason@google.com>
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pkg/gktest/runner_test.go Show resolved Hide resolved
willbeason pushed a commit to willbeason/gatekeeper that referenced this pull request Jul 16, 2021
This lets us read Suites from the disk.

Per open-policy-agent#1446, we aren't doing Assertions for pre-alpha so I haven't added
those annotations yet.

Signed-off-by: Will Beason <willbeason@google.com>
@willbeason willbeason force-pushed the run-case branch 2 times, most recently from e68a8c8 to 5f3234c Compare July 20, 2021 14:48
willbeason pushed a commit to willbeason/gatekeeper that referenced this pull request Jul 20, 2021
This lets us read Suites from the disk.

Per open-policy-agent#1446, we aren't doing Assertions for pre-alpha so I haven't added
those annotations yet.

Signed-off-by: Will Beason <willbeason@google.com>
Will Beason added 2 commits July 22, 2021 11:57
A first pass at printing output similar to what is shown for "go test".

Sadly we can't just use testing.T without asking users to write their
own unit tests. There is no accessible printer that automatically does
this formatting for us, so we must replicate it.

This does not print output as tests are run even though the result
output appears to suggest this. In the future we will do so, but as we
don't yet run tests that would be premature. The point here is to make
the output consistent with "go test".

Signed-off-by: Will Beason <willbeason@google.com>
Add logic for running individual test cases. Now that we've compiled the
constraints, all we need to do is Review() the specified objects and
compare the Results with what the user specified.

This PR does not add Assertion functionality - we'll do that for alpha,
not pre-alpha.

Also add unit tests that verify we are, in fact, running the test cases.

Signed-off-by: Will Beason <willbeason@google.com>
willbeason pushed a commit to willbeason/gatekeeper that referenced this pull request Jul 22, 2021
This lets us read Suites from the disk.

Per open-policy-agent#1446, we aren't doing Assertions for pre-alpha so I haven't added
those annotations yet.

Signed-off-by: Will Beason <willbeason@google.com>
@willbeason willbeason merged commit b6d1d60 into open-policy-agent:master Jul 27, 2021
@willbeason willbeason deleted the run-case branch July 27, 2021 19:45
willbeason pushed a commit that referenced this pull request Jul 27, 2021
* Print output of tests similar go go test

A first pass at printing output similar to what is shown for "go test".

Sadly we can't just use testing.T without asking users to write their
own unit tests. There is no accessible printer that automatically does
this formatting for us, so we must replicate it.

This does not print output as tests are run even though the result
output appears to suggest this. In the future we will do so, but as we
don't yet run tests that would be premature. The point here is to make
the output consistent with "go test".

Signed-off-by: Will Beason <willbeason@google.com>

* Run test cases

Add logic for running individual test cases. Now that we've compiled the
constraints, all we need to do is Review() the specified objects and
compare the Results with what the user specified.

This PR does not add Assertion functionality - we'll do that for alpha,
not pre-alpha.

Also add unit tests that verify we are, in fact, running the test cases.

Signed-off-by: Will Beason <willbeason@google.com>

* Add json annotations for Suites

This lets us read Suites from the disk.

Per #1446, we aren't doing Assertions for pre-alpha so I haven't added
those annotations yet.

Signed-off-by: Will Beason <willbeason@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
authorship-tooling Tools for authors of Gatekeeper Constraints, Templates, Mutators
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants