Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

topdown: memoize partial set results #3492

Merged

Conversation

srenatus
Copy link
Contributor

@srenatus srenatus commented May 27, 2021

We can either cache individual elements (data.foo.p["bar"]), or the
full extent of a partial set/object. A cached full extent of the partial
would be used when evaluating individual elements of the partial.

If the first encounter with a partial set/object has to materialize the
full extent with a variable key, like data.foo.p[x], then we cache the
fully-evaluated result for data.foo.p.

Fixes #822.

@srenatus srenatus marked this pull request as ready for review May 27, 2021 11:20
@srenatus srenatus force-pushed the sr/issue-822/memoize-partial-sets branch from 5f42254 to 8b88a91 Compare May 27, 2021 11:55
@srenatus
Copy link
Contributor Author

I'll incorporate @tsandall's WIP from here https://github.com/tsandall/opa/commits/memoize-partial-sets to capture more cases. Back to draft mode.

@srenatus srenatus marked this pull request as draft June 24, 2021 07:37
@srenatus srenatus force-pushed the sr/issue-822/memoize-partial-sets branch 3 times, most recently from 66699fd to 612ab09 Compare June 29, 2021 19:54
@srenatus srenatus marked this pull request as ready for review July 1, 2021 19:36
@srenatus
Copy link
Contributor Author

srenatus commented Jul 1, 2021

So now, if a full extent eval of data.foo.p is required because of a query data.foo.p[x] with non-ground X, the evaluation will code path will switch to evaluating it completely, and caching the result.

@srenatus srenatus force-pushed the sr/issue-822/memoize-partial-sets branch from 93a0547 to 892bf9f Compare July 12, 2021 09:14
tsandall
tsandall previously approved these changes Jul 13, 2021
@tsandall
Copy link
Member

tsandall commented Jul 13, 2021

LGTM. Just squash and merge when you're ready. Closing out a 3-year old issue feels great :D

We can either cache individual elements (`data.foo.p["bar"]`), or the
full extent of a partial set/object. A cached full extent of the partial
would be used when evaluating individual elements of the partial.

If the first encounter with a partial set/object has to materialize the
full extent with a variable key, like `data.foo.p[x]`, then we cache the
fully-evaluated result for `data.foo.p`.

Fixes open-policy-agent#822.

Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
@srenatus srenatus force-pushed the sr/issue-822/memoize-partial-sets branch from 892bf9f to 50f96e5 Compare July 14, 2021 07:51
@srenatus srenatus merged commit f3284cf into open-policy-agent:main Jul 14, 2021
@srenatus srenatus deleted the sr/issue-822/memoize-partial-sets branch July 14, 2021 08:09
dolevf pushed a commit to dolevf/opa that referenced this pull request Nov 4, 2021
We can either cache individual elements (`data.foo.p["bar"]`), or the
full extent of a partial set/object. A cached full extent of the partial
would be used when evaluating individual elements of the partial.

If the first encounter with a partial set/object has to materialize the
full extent with a variable key, like `data.foo.p[x]`, then we cache the
fully-evaluated result for `data.foo.p`.

Fixes open-policy-agent#822.

Co-authored-by: Torin Sandall <torinsandall@gmail.com>
Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>
Signed-off-by: Dolev Farhi <farhi.dolev@gmail.com>
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memoize partial set results
2 participants