Enabling both the java and protobuf.python backends at once causes a rule graph error
#12889
Comments
|
When fixing, we should probably expand |
Apparently I forgot my discrete math / stats classes 😅 we have 24 backends currently. If we just wanted to test every combination where r=12, that's Testing every combination is not feasible. At the same time, only no backends + all backends + each backend on its own seems too limited. Perhaps we randomly select an additional n combinations to test, say 100? Normally random is bad in tests due to flakiness, but here, we would intentionally have it be random so that over time we get better coverage. That's still pretty dang limited in coverage, but better than nothing? cc @benjyw , iirc you enjoy questions like this |
|
I'm hopeful that fixing the underlying rule graph issue will make testing every combination unnecessary. But we'll see. |
|
=Stu.
I don't think fuzzing is the right approach.
The problem isn't "some combinations of our existing backends cause
errors". The problem is "it is possible for this class of errors to
happen". Testing all possible combinations is meaningless if a user's
custom backend can introduce the issue. We have to understand the
underlying issue, fix it, and regression test it.
…On Thu, Sep 16, 2021 at 9:34 PM Stu Hood ***@***.***> wrote:
I'm hopeful that fixing the underlying rule graph issue will make testing
every combination unnecessary. But we'll see.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#12889 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAD5F7GXCTB25V7FTB32OTTUCLAOPANCNFSM5EA752RA>
.
|
|
I'm fairly optimistic that I have an idea of how to do this. Opened #12934. |
As described in #12934, we would like plugins to have better defined interfaces, and to further clarify what is available while satisfying a `@union`. Without making any API changes for `@rule` authors, we can take a first step in this direction by implementing the `Get`s for `@union` types using `Query`s instead. The effect of this is that a `@union` usage may _only_ use the explicitly provided `Param`, and no others. When compared to the usual usage of `Get` (which can consume any `Param`s which are in scope at the "call site"), this makes for a much better defined API boundary. In order to complete #12934, we will likely want to make interface changes to allow more than the single `@union`-member-`Param` specified to the `Get` to be consumed by a plugin (see the examples in that issue's description). But that is not necessary today, and this change also has the benefit of fixing the blocking issue behind #12889 and #12929 by significantly simplifying the rule graph computation (since plugin boundaries are now "hardcoded", and don't need the `Param` detection executed for `Get`s). [ci skip-build-wheels]
Enabling both the
javaandprotobuf.pythonbackends at once causes a completely inscrutable rule graph error. This has been the case since #12781, which is when the@ruleswere first exposed as a backend.Repro:
The text was updated successfully, but these errors were encountered: