Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type for matching Constraints #193

Merged
merged 14 commits into from
Feb 14, 2022

Conversation

willbeason
Copy link
Member

Prep work for compiler sharding. This adds a store of the matchers
corresponding to Constraints by-target and by-kind.

We could do constraint matching per-target, but it's better for us to
fail fast rather than partially review objects when we're just going to
return errors. Thus, ConstraintsFor iterates over all targets itself
rather than requiring its caller to.

The triple-nested maps are ugly - I would not need much convincing
to replace them with types.

Signed-off-by: Will Beason willbeason@google.com

Will Beason added 5 commits January 27, 2022 10:25
Targets sometimes need to maintain state about the system. For example,
the Gatekeeper target needs to track the set of current Namespaces on
the cluster in order to properly match objects to Constraints when Audit
is called.

This commit adds a Cache interface which Targets may choose to
implement. If they implement this interface, Client attempts to add and
remove objects from the Target cache just as it does for Driver caches.

These operations are not atomic, so it is possible for systems to get
into an inconsistent state. There isn't a good solution to this now -
I've opened open-policy-agent#189 to solve this in the future. The implications are quite
complex and there's a lot of edge cases.

This commit also modifies the test target handler matchers - they now
access the test target's cache in order to function. These matchers
aren't called yet - we don't want to break Gatekeeper since Gatekeeper
Golang matchers are not yet implemented.

Signed-off-by: Will Beason <willbeason@google.com>
Signed-off-by: Will Beason <willbeason@google.com>
Otherwise it is easy to get into inconsistent cache states. There's lots
of edge cases that can cause unpredictable behaviors that we don't want
to allow.

Signed-off-by: Will Beason <willbeason@google.com>
Since adding data can fail in the target cache, remove data from the
driver cache.

Note that addition/deletion occur in opposite orders for AddData and
RemoveData - this is because we want to prioritize reversible over
potentially-irreversible operations. Removing data from the handler
cache can't fail, so it is safe to add it first.

Signed-off-by: Will Beason <willbeason@google.com>
Otherwise we can easily end up in very annoying inconsistent states. If
deleteion really, really needs to fail then the application should panic
rather than allow things to get in an inconsistent state.

Per discussion

Signed-off-by: Will Beason <willbeason@google.com>
@willbeason willbeason added this to the Rego Environment Sharding milestone Feb 1, 2022
@willbeason willbeason self-assigned this Feb 1, 2022
@willbeason
Copy link
Member Author

My reason for the current implementation as maps-of-maps-of-maps is that this messiness is contained to the package - the type is fully contained in the client package and it is unexported. If we were to move this functionality to its own package, or to need this behavior to be extensible somehow, we'd want to use special types for the per-target and per-kind layers.

@willbeason
Copy link
Member Author

Fixes #176

Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit, one concern around where/when the HandleReview hook is called.

constraint/pkg/client/matchers.go Outdated Show resolved Hide resolved
constraint/pkg/client/matchers.go Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Feb 3, 2022

Codecov Report

Merging #193 (b23d39e) into master (2c8fe2d) will increase coverage by 1.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #193      +/-   ##
==========================================
+ Coverage   47.88%   48.90%   +1.01%     
==========================================
  Files          59       60       +1     
  Lines        2865     2926      +61     
==========================================
+ Hits         1372     1431      +59     
- Misses       1243     1244       +1     
- Partials      250      251       +1     
Flag Coverage Δ
unittests 48.90% <ø> (+1.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...agent/frameworks/constraint/pkg/client/matchers.go 96.72% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c8fe2d...b23d39e. Read the comment docs.

constraint/pkg/client/matchers.go Outdated Show resolved Hide resolved
constraint/pkg/client/matchers.go Outdated Show resolved Hide resolved
Will Beason added 5 commits February 8, 2022 07:20
Prep work for compiler sharding. This adds a store of the matchers
corresponding to Constraints by-target and by-kind.

We could do constriant matching per-target, but it's better for us to
fail fast rather than partially review objects when we're just going to
return errors. Thus, ConstraintsFor iterates over all targets itself
rather than requiring its caller to.

Signed-off-by: Will Beason <willbeason@google.com>
Each Handler potentially has its own review type to be matched against,
so this makes requests for matching happen per-target.

Signed-off-by: Will Beason <willbeason@google.com>
Signed-off-by: Will Beason <willbeason@google.com>
Signed-off-by: Will Beason <willbeason@google.com>
Signed-off-by: Will Beason <willbeason@google.com>
Will Beason added 2 commits February 8, 2022 07:32
Signed-off-by: Will Beason <willbeason@google.com>
Signed-off-by: Will Beason <willbeason@google.com>
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Will Beason added 2 commits February 10, 2022 10:36
Signed-off-by: Will Beason <willbeason@google.com>
Signed-off-by: Will Beason <willbeason@google.com>
@willbeason
Copy link
Member Author

Fixes #176

Copy link
Member

@ritazh ritazh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@willbeason willbeason merged commit 811b909 into open-policy-agent:master Feb 14, 2022
@willbeason willbeason deleted the constraint-matcher branch February 14, 2022 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants