-
Notifications
You must be signed in to change notification settings - Fork 393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trouble with DML/overlays and ensure_source_rvar #3030
Comments
msullivan
added a commit
that referenced
this issue
Oct 13, 2021
msullivan
added a commit
that referenced
this issue
May 24, 2022
Many of these were found by putting a trivial access policy on std::Object and running the test suite. * Disable overlays when compiling type rewrites: they aren't really in scope. * Work around issue #3030 by looking through assert_exists when looking for DML. * Fix a bad interaction with materialization that can generate invalid queries * Fix an issue with UNLESS CONFLICT by compiling constraints without query rewrites enabled to avoid introducing extra dependencies that broke our constraint key detection * Fix ordering in some tests that broke when the std::Object policy perturbed their ordreing
msullivan
added a commit
that referenced
this issue
May 24, 2022
Many of these were found by putting a trivial access policy on std::Object and running the test suite. * Disable overlays when compiling type rewrites: they aren't really in scope. * Work around issue #3030 by looking through assert_exists when looking for DML. * Fix a bad interaction with materialization that can generate invalid queries * Fix an issue with UNLESS CONFLICT by compiling constraints without query rewrites enabled to avoid introducing extra dependencies that broke our constraint key detection * Fix ordering in some tests that broke when the std::Object policy perturbed their ordreing
msullivan
added a commit
that referenced
this issue
Sep 28, 2023
Because of how casts are inserted in the func code, this required some tweaking to casts: - The empty array cast needs to be able to look through a Set in order to be efficient - The empty set to object cast needs to be able to look through a Set in order to not generate an IR cast that messes things up because it doesn't provide a source and thus causes the #3030 overlay issue to pop up.
msullivan
added a commit
that referenced
this issue
Sep 28, 2023
As we've discussed (and, tragically, recommended to users), we can rewrite the conditionals into for loops: for b in COND union ( { (for _ in (select () filter b) union (IF_BRANCH)), (for _ in (select () filter not b) union (ELSE_BRANCH)), } ) The main fiddly part is preserving the correct cardinality/multiplicity inference. I did this by adding a card_inference_override field to Set that specifies another set that should determine the cardinality. I don't love this, but I haven't thought of a cleaner approach that doesn't give up the benefits of the desugaring approach. We need more testing but I wanted to get something up for people to look at / we can catch up on testing after the feature freeze if needed. Fixes #4437. * Support writing a bare {} in the else branch Because of how casts are inserted in the func code, this required some tweaking to casts: - The empty array cast needs to be able to look through a Set in order to be efficient - The empty set to object cast needs to be able to look through a Set in order to not generate an IR cast that messes things up because it doesn't provide a source and thus causes the #3030 overlay issue to pop up.
This was referenced Sep 30, 2023
msullivan
added a commit
that referenced
this issue
Oct 3, 2023
Queries like: ``` with noobs := { ((insert Person { name := "foo" }), "bar"), ((insert Person { name := "spam" }), "eggs"), }, select noobs; ``` would return an empty set, since (for annoying reasons that could possibly be avoided in this simple case but not in the general case), we need to do an `ensure_source_rvar` on `noobs`, but `get_nearest_dml_stmt` is highly limited and can only find one DML statement. This means we can't produce the proper overlay that includes both Persons. Upgrade `get_nearest_dml_stmt` to be able to collect all of the relevant DML sources. Fixes #6057. That bug was just a special case of #3030; this fix thus represents a combination of forward progress on #3030 as well as some lateral movement: as discussed in that issue, some queries are now *differently* wrong (see test_edgeql_update_union_overlay_02, which previously would have returned the old value twice instead of the new value twice). I am going to close #3030 in favor of an updated #6222 once this is merged.
msullivan
added a commit
that referenced
this issue
Oct 4, 2023
) Queries like: ``` with noobs := { ((insert Person { name := "foo" }), "bar"), ((insert Person { name := "spam" }), "eggs"), }, select noobs; ``` would return an empty set, since (for annoying reasons that could possibly be avoided in this simple case but not in the general case), we need to do an `ensure_source_rvar` on `noobs`, but `get_nearest_dml_stmt` is highly limited and can only find one DML statement. This means we can't produce the proper overlay that includes both Persons. Upgrade `get_nearest_dml_stmt` to be able to collect all of the relevant DML sources. Fixes #6057. That bug was just a special case of #3030; this fix thus represents a combination of forward progress on #3030 as well as some lateral movement: as discussed in that issue, some queries are now *differently* wrong (see test_edgeql_update_union_overlay_02, which previously would have returned the old value twice instead of the new value twice). I am going to close #3030 in favor of an updated #6222 once this is merged.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In #3023 I added an xfailed insert test. A simplified version of it is
and it returns an empty set. The issue turns out to be:
Z.1.0
, and so the overlay isn't applied. This is becauseget_nearest_dml
just looks down the main path.I think this particular case might be easy to fix by making sure that enumerate can preserve source rvars for its elements, but there are plenty of operations that won't be able to preserve source rvars, like:
Part 2 could be addressed by taking a wider view of finding dml sources; perhaps by grabbing any dml source in the ir, at all. But this made me realize that there is a more general problem here.
Consider
We will output
One object is from the select, and reflects the old data, and one from the update, and reflects the new.
If we try something where we lose track of the source rvars, though:
... then we trip an assertion. But if we remove that assertion, then we get
with both objects reflecting the old data, since we don't use the overlay.
But if we did arrange to use the overlay, then both objects would reflect the new data, which is also wrong!
Getting this right in cases like this seems pretty tricky. Possible approaches:
I'm not sure if option 2 scales to dealing with links that are also changed, and the like, though.
This issue seems very tricky to get right.
The text was updated successfully, but these errors were encountered: