Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break loops when result can no longer change #728

Merged
merged 1 commit into from
Feb 7, 2023

Conversation

XN137
Copy link
Contributor

@XN137 XN137 commented Feb 2, 2023

No description provided.

Copy link
Contributor Author

@XN137 XN137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general note:

please let me know in case you consider these unnecessary micro optimizations

for me, when reading the code i always expected the loop bodies to have "meaningful" side-effects but from reading the inner code i think it was always just doing redundant work, with no observable difference, but maybe i am missing something

@XN137 XN137 marked this pull request as ready for review February 2, 2023 07:04
Copy link
Collaborator

@msridhar msridhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall these changes LGTM. I don't know if we're consistent about breaking out of loops whenever possible, but it seems good to do so. @lazaroclapp if you have time you can take a quick look before we merge

@@ -184,7 +184,10 @@ public NullnessHint onDataflowVisitMethodInvocation(
NullnessHint n =
h.onDataflowVisitMethodInvocation(
node, state, apContext, inputs, thenUpdates, elseUpdates, bothUpdates);
nullnessHint = nullnessHint.merge(n);
nullnessHint = nullnessHint.mergeByPriority(n);
if (nullnessHint.isHighestPriority()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly related to this PR, but in general, I feel like our use of CompositeHandler is set up for a potentially-bad future bug where multiple handlers implement a particular method and their interaction is unexpected or order-dependent. @lazaroclapp do you know if we have known cases where multiple handlers try to "handle" the same construct and we expect one to win? If not, we may want to implement some kind of "handler debug mode" (off by default) where we actually check in CompositeHandler that only one contained handler is returning a non-default value for each case. It will make the code messier but may save us some pain in the future.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would not work:

check in CompositeHandler that only one contained handler is returning a non-default value for each case

Handlers are meant to override and use the computations of one another, which is why they execute in a chain with meaningful order between them. Each Handler API method defines how results from different handlers are combined, in the case of onDataflowVisitMethodInvocation (admittedly one of the most complex) it's based on the priority given by the NullnessHint enum. This definitely leads to weird corner cases, often, but it also gives us a lot of flexibility in deciding which "source" should win for any given nullness fact (e.g. between jar annotations and library models).

@627721565
Copy link

627721565 commented Feb 5, 2023 via email

Copy link
Collaborator

@lazaroclapp lazaroclapp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple safe optimizations here (marked them with 👍 ), but in general breaking out early of handler chain iteration is not a safe optimization. The contract is that for a given event, the corresponding method will be called for all handlers in the chain, and these methods can have side effects beyond the value returned.

nullnessHint = nullnessHint.merge(n);
nullnessHint = nullnessHint.mergeByPriority(n);
if (nullnessHint.isHighestPriority()) {
break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this will currently break anything, but this is not a safe optimization.

It means that calls to .onDataflowVisitMethodInvocation(...) might be skipped when a handler early in the chain "wins" in terms of determining the NullnessHint. But keep in mind that handler methods are not meant to be pure, they are meant to be callbacks for events of the analysis. In fact, .onDataflowVisitMethodInvocation(...) specifically is often called for its effects on its AccessPathNullnessPropagation.Updates paramenters.

For example, AssertionHandler will always return NullnessHint.UNKNOWN, because it doesn't actually care about the nullability of the statement assertThat(foo).bar(), but will instead add whatever facts that assertion statement indicates about foo into bothUpdates.

Copy link
Contributor Author

@XN137 XN137 Feb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point - thanks for the explanation, not sure how i missed this 😓

so for assertThat(x).isNotNull() when the handlers examine the isNotNull() method invocation part, we dont really care about the nullness hint on the handlers return for isNotNull() but we want to give them the opportunity to examine the whole method-call-chain and realize that x is non-null afterwards (i.e. update bothUpdates)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost correct. It's not about the method call chain, but the chain of handlers. The way these handlers are used is, whenever you call a method like .isNotNull(), each handler in turn has the opportunity to process it. Some handlers, like LibraryModelsHandler will process it by i.e. looking up some information and deciding whether or not the return of method call should @Nullable or @NonNull. But that's not the only action to be taken on such call. As you know, AssertionHandler will instead look at the receiver of .isNotNull() and unpack x within a assertThat(x) receiver and then set the nullability of x (within dataflow after the current statement).

With your change, if a handler early in the chain computes a FORCE_NONNULL nullability for the method call's return, then no handler after that has a chance to process the call and perhaps set other state or any of the updates passed as arguments based on the call.

@@ -231,6 +231,7 @@ public NullnessStore getNullnessInfoBeforeNewContext(
if (!e.getKind().equals(ElementKind.FIELD)
|| !e.getModifiers().contains(Modifier.FINAL)) {
allAPNonRootElementsAreFinalFields = false;
break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -207,6 +207,7 @@ public NullnessHint onDataflowVisitMethodInvocation(
for (int idx : nullImpliesNullIndexes) {
if (!inputs.valueOfSubNode(node.getArgument(idx)).equals(NONNULL)) {
anyNull = true;
break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Collaborator

@lazaroclapp lazaroclapp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining two optimizations LGTM

@lazaroclapp lazaroclapp merged commit a2efa6e into uber:master Feb 7, 2023
@XN137 XN137 deleted the early-loop-break branch April 5, 2023 16:27
msridhar added a commit to msridhar/NullAway that referenced this pull request Jul 18, 2023
msridhar added a commit to msridhar/NullAway that referenced this pull request Jul 18, 2023
msridhar added a commit to msridhar/NullAway that referenced this pull request Jul 19, 2023
msridhar added a commit to msridhar/NullAway that referenced this pull request Jul 19, 2023
msridhar added a commit to msridhar/NullAway that referenced this pull request Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants