Skip to content

Conversation

@dianfu
Copy link
Contributor

@dianfu dianfu commented Dec 9, 2017

What is the purpose of the change

This pull request fix the issue that dangling reference generated after NFA clean up timed out SharedBufferEntry. Exception will be thrown when serializing NFA.

Verifying this change

This change added tests and can be verified as follows:

(example:)

  • Added tests NFATest#testTimeoutWindowPruning2

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

@dianfu
Copy link
Contributor Author

dianfu commented Dec 21, 2017

@dawidwys Could you help to take a look at this PR? This is a bug fix and the issue can be easily reproduced with the test case included in the PR. Thanks a lot.

Copy link
Contributor

@dawidwys dawidwys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch @dianfu! I had just one slight comment.


private transient Map<K, SharedBufferPage<K, V>> pages;

private final transient List<SharedBufferEntry<K, V>> prunedEntries;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary for it to be a field? Isn't a local variable sufficient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is for object reuse. What's your thought?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about the potential advantages. See e.g. this answer on SO: https://stackoverflow.com/a/18371813/4250114 . Plus the local variable is easier to reason about.

Personally I am for the local variable, but if you feel this version is better, will merge it as it is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, I will change it to local variable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the PR. Thanks a lot for the review. :)

for (Map.Entry<K, SharedBufferPage<K, V>> entry : pages.entrySet()) {
entry.getValue().removeEdges(prunedEntries);
}
prunedEntries.clear();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As prunedEntries is local now, you don't need to clear it any longer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@asfgit asfgit closed this in 9e3bac3 Jan 2, 2018
asfgit pushed a commit that referenced this pull request Jan 12, 2018
glaksh100 pushed a commit to lyft/flink that referenced this pull request Jun 6, 2018
@dianfu dianfu deleted the dangling_ref branch June 10, 2020 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants