Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8307538: Memory leak in TreeTableView when calling refresh #1129

Conversation

andy-goryachev-oracle
Copy link
Contributor

@andy-goryachev-oracle andy-goryachev-oracle commented May 5, 2023

Fixed a memory leak in TreeTableView by reverting to register**Listener (which is ok in this particular situation) - the leak is specific to TreeTableRowSkin.

Added a unit test.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

  • JDK-8307538: Memory leak in TreeTableView when calling refresh (Bug - "2")

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/1129/head:pull/1129
$ git checkout pull/1129

Update a local copy of the PR:
$ git checkout pull/1129
$ git pull https://git.openjdk.org/jfx.git pull/1129/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1129

View PR using the GUI difftool:
$ git pr show -t 1129

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/1129.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented May 5, 2023

👋 Welcome back angorya! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@andy-goryachev-oracle andy-goryachev-oracle marked this pull request as ready for review May 8, 2023 17:34
@openjdk openjdk bot added the rfr Ready for review label May 8, 2023
@mlbridge
Copy link

mlbridge bot commented May 8, 2023

@andy-goryachev-oracle
Copy link
Contributor Author

will separate Monkey Tester changes into a separate PR.

@kevinrushforth
Copy link
Member

/reviewers 2

@openjdk
Copy link

openjdk bot commented May 8, 2023

@kevinrushforth
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

@hjohn
Copy link
Collaborator

hjohn commented May 8, 2023

I've seen the problem with Views and Cell creation often enough now that I think we may want to talk about the fundamental problem that is causing these issues. The lifecycle of a Cell or Row is not the same as the View that contains them. Cells are created on demand, and should either be destroyed or unlinked.

In JavaFX, we have chosen to not have an explicit destroy method for cells, but we do have a way to unlink cells. For a TreeTableRow this is a simple as calling TreeTableRow#updateTreeTableView(null). The call already does exactly what you would expect: it unregisters all troublesome listeners (ones that were registered on the view). The skin also responds to this setting to null as it adds listeners to the treeTableViewProperty of its skinnable.

In other words, a TreeTableView which creates cells on demand with a different lifecycle than itself should also be responsible to clean them up when it no longer needs them -- this is where the real bug is. It is managing the lifecycle of these cells and should not rely on these cells doing this indirectly by using a lot of weak listeners. If TreeTableView would correctly unlink cells and rows it doesn't need, it would remove the need for all weak listeners (and also references) in both TreeTableRow and TreeTableRowSkin, making their code much simpler, more predictable and easier to test.

Specifically for this PR, only the listeners registered as part of setupTreeTableViewListeners need to be weak, the others are not the issue.

@andy-goryachev-oracle
Copy link
Contributor Author

andy-goryachev-oracle commented May 8, 2023

This is a very good suggestion, thanks!

edit:
I agree that the idea of explicitly clearing the cells when they are not needed would be a much better solution.
It will, however, require non-trivial changes in VirtualFlow, would impact three controls (List|Tree|TableView) and would require some careful testing.

For the purposes of fixing the regression bug, I would like to proceed with this fix as is.

@kevinrushforth
Copy link
Member

I agree that the idea of explicitly clearing the cells when they are not needed would be a much better solution. It will, however, require non-trivial changes in VirtualFlow, would impact three controls (List|Tree|TableView) and would require some careful testing.

Yes, and we wouldn't do this as part of a regression bug fix like this. It seems worth filing a follow-up bug for this. An explicit life-cycle that eliminates the need for weak listeners is generally a good thing.

For the purposes of fixing the regression bug, I would like to proceed with this fix as is.

Agreed.

Copy link
Member

@kevinrushforth kevinrushforth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I run just the TreeTableRowSkinTest test class, I now get a test failure:

$ gradle --info -PTEST_ONLY=true :controls:test --tests TreeTableRowSkinTest
TreeTableRowSkinTest > treeTableRowWithFixedCellSizeShouldIgnoreVerticalPadding() FAILED
    org.opentest4j.AssertionFailedError: expected: <18.0> but was: <23.0>
        at app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
        at app//org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
        at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:86)
        at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:81)
        at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1010)
        at app//test.javafx.scene.control.skin.TreeTableRowSkinTest.treeTableRowWithFixedCellSizeShouldIgnoreVerticalPadding(TreeTableRowSkinTest.java:251)

Given that the GHA run passes, as does a full controls test run on my system, that suggests something odd is going on with that test, possibly it is affected by tests in another class. If so, then it could fail randomly anyway, since the tests are run in an unspecified (and unpredictable) order.

@kevinrushforth
Copy link
Member

@aghaisas can you review this?

@hjohn
Copy link
Collaborator

hjohn commented May 10, 2023

For the purposes of fixing the regression bug, I would like to proceed with this fix as is.

I think that's fine, but I do think you should remove the changes that don't contribute to the fix (I'm pretty sure the weak listeners in the constructor are not a problem).

Copy link
Member

@kevinrushforth kevinrushforth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a couple inline comments.

I also wonder whether the index and treeItem properties need to be weak (I don't think they do).

The failing test is something that will need to be tracked down. If I revert your changes to treeTableRowWithFixedCellSizeShouldIgnoreVerticalPadding, the test passes when that class is run by itself, but fails when the entire controls test suite is run. The modified test fails with that class is run by itself, but passes when the entire controls test suite is run.

This suggests a possible problem where the (now weak) listeners are getting GCed sometimes but not others, and that whether or not it is GCed causes differences that are visible to the test. So either the test is relying on an implementation detail, or there is a functional problem caused by the listeners being GCed too early or not early enough (the latter could point to the need to remove them in dispose).

@andy-goryachev-oracle
Copy link
Contributor Author

it looks like the test is unstable - it works fine in Eclipse, but sometimes fails with gradle. investigating.

@kevinrushforth
Copy link
Member

it looks like the test is unstable - it works fine in Eclipse, but sometimes fails with gradle. investigating.

It's definitely affected by GC, so I'd start by looking there. I can change whether the the test passes or fails by changing the heap size or adding a call to gc before each test.

@andy-goryachev-oracle andy-goryachev-oracle marked this pull request as draft May 11, 2023 15:17
@openjdk openjdk bot removed the rfr Ready for review label May 11, 2023
@andy-goryachev-oracle andy-goryachev-oracle marked this pull request as ready for review May 11, 2023 22:20
@openjdk openjdk bot added the rfr Ready for review label May 11, 2023
Copy link
Collaborator

@hjohn hjohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed the tests fail before the changes in setupTreeTableViewListeners and pass with those changes. Removing the constructor and dispose changes did not fail any tests.

Comment on lines 101 to 103
ListenerHelper lh = ListenerHelper.get(this);

lh.addChangeListener(control.indexProperty(), (ev) -> {
registerChangeListener(control.indexProperty(), (x) -> {
updateCells = true;
});

lh.addChangeListener(control.treeItemProperty(), (ev) -> {
registerChangeListener(control.treeItemProperty(), (obs) -> {
updateTreeItem();
// There used to be an isDirty = true statement here, but this was
// determined to be unnecessary and led to performance issues such as
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes don't fail the test when I undo them. As said before, they're unnecessary.

Reasoning: they register on TreeTableRow, which is associated with the skin directly. If their lifecycles don't match, then dispose will take care of unregistering. If their lifecycles do match, then they go out of scope at the same time.

Unless you prefer using the register functions, I think this change should be undone.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though I don't particularly like register*Listener() because of its asymmetric nature (when it comes to removing listeners), here we do need to create weak listeners that get unregistered upon dispose(). We need weak listeners because TreeTableView does not explicitly "disconnect" unused cells (e.g. refresh()), and we need dispose() due to skin life cycle.

So, in this particular case, I think this change should be ok.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I think we may be misunderstanding each other. I'm specifically talking about the two listeners added in TreeTableRowSkin's constructor on indexProperty and treeItemProperty. These are part of the TreeTableRow which is also discarded when refresh is called.

  1. The test passes if these changes are reverted. This is a clear indication that either the test is insufficient, or that your assumption, that these must be weak, is incorrect. If you can construct a test that requires these listeners to be weak, then I think the changes are warranted.

  2. At the risk of stating the obvious, the listeners in the constructor added using ListenerHelper are also correctly removed in dispose. ListenerHelper does this in a similar way to the register methods.

  3. The indexProperty and treeItemProperty listeners are attached to the TreeTableRow, not the TreeTableView. The refresh function replaces the entire row, including the skin. There is no need to use weak listeners for this purpose. Compare this to the listeners that are attached to the TreeTableView or the VirtualFlow; these have a much longer lifecycle and can survive multiple refreshes and row factory replacements, and hence should be weak (for now).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right, of course. and I should know - ListenerHelper is my baby. I guess the fear of another regression overtook me.

reverting.

thank you for your patience and persistence!

}
}
}
}

private void updateCachedFixedSize() {
if (getSkinnable() != null) {
TreeTableView<T> t = getSkinnable().getTreeTableView();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is a short method, but I would rather see a more descriptive variable name here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer not to drag long names if a very descriptive type is right there: TreeTableView<T> t

@kevinrushforth kevinrushforth self-requested a review May 22, 2023 21:27
Copy link
Collaborator

@aghaisas aghaisas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks good to me!

Copy link
Member

@kevinrushforth kevinrushforth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@hjohn Did you want to re-review (there was only one cleanup change since you approved)?

@openjdk
Copy link

openjdk bot commented Jun 5, 2023

@andy-goryachev-oracle This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8307538: Memory leak in TreeTableView when calling refresh

Reviewed-by: kcr, aghaisas, mhanl

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 21 new commits pushed to the master branch:

  • 05548ac: 8301312: Create implementation of NSAccessibilityButton protocol
  • 10f41b7: 8293836: Rendering performance degradation at bottom of TableView with many rows
  • 1a0f6c7: 8306447: Adding an element to a long existing list may cause the first visible element to jump
  • 8fc1a25: 8308308: Update to Visual Studio 2022 version 17.5.0 on Windows
  • 0005f65: 8299756: Minor updates in CSS Reference
  • 7825137: 8306990: The guarantees given by Region's floor and ceiling functions should work for larger values
  • 3fa02ee: 8304959: Public API in javafx.css.Match should not return private API class PseudoClassState
  • 2a6e48f: 8308017: [Mac] Update deprecated constants in GlassWindow code
  • f8c8a8a: 8308191: [macOS] VoiceOver decorations are shifted on second monitor
  • 6334032: 8223373: Remove IntelliJ IDEA specific files from the source code repository
  • ... and 11 more: https://git.openjdk.org/jfx/compare/e7974bc84618c9f954e075935cc2ff324c741aad...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Ready to be integrated label Jun 5, 2023
TreeTableView<T> treeTableView = getSkinnable().getTreeTableView();
if (treeTableView == null) {
lh.addInvalidationListener(getSkinnable().treeTableViewProperty(), (ev) -> {
registerInvalidationListener(getSkinnable().treeTableViewProperty(), (x) -> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question here: Why does this prevent the leak but the ListenerHelper does not?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference is that registerInvalidationListener() adds a weak listener, while ListenerHelper adds a strong listener.

It is possible to use ListenerHelper here, at the expense of more complicated code since we'd need to explicitly disconnect the listener when tableViewProperty value gets set.

Another solution would involve adding a method to add a weak listener to the ListenerHelper to avoid explicit cleanup, or

Go back to the original code which used register/unregister*Listener in this particular case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, makes sense.

Copy link
Member

@Maran23 Maran23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@andy-goryachev-oracle
Copy link
Contributor Author

thank you all for your feedback and effort!

@andy-goryachev-oracle
Copy link
Contributor Author

/integrate

@openjdk
Copy link

openjdk bot commented Jun 6, 2023

Going to push as commit 17ed2e7.
Since your change was applied there have been 21 commits pushed to the master branch:

  • 05548ac: 8301312: Create implementation of NSAccessibilityButton protocol
  • 10f41b7: 8293836: Rendering performance degradation at bottom of TableView with many rows
  • 1a0f6c7: 8306447: Adding an element to a long existing list may cause the first visible element to jump
  • 8fc1a25: 8308308: Update to Visual Studio 2022 version 17.5.0 on Windows
  • 0005f65: 8299756: Minor updates in CSS Reference
  • 7825137: 8306990: The guarantees given by Region's floor and ceiling functions should work for larger values
  • 3fa02ee: 8304959: Public API in javafx.css.Match should not return private API class PseudoClassState
  • 2a6e48f: 8308017: [Mac] Update deprecated constants in GlassWindow code
  • f8c8a8a: 8308191: [macOS] VoiceOver decorations are shifted on second monitor
  • 6334032: 8223373: Remove IntelliJ IDEA specific files from the source code repository
  • ... and 11 more: https://git.openjdk.org/jfx/compare/e7974bc84618c9f954e075935cc2ff324c741aad...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Jun 6, 2023
@openjdk openjdk bot closed this Jun 6, 2023
@openjdk openjdk bot removed ready Ready to be integrated rfr Ready for review labels Jun 6, 2023
@openjdk
Copy link

openjdk bot commented Jun 6, 2023

@andy-goryachev-oracle Pushed as commit 17ed2e7.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@andy-goryachev-oracle andy-goryachev-oracle deleted the 8307538.refresh branch June 6, 2023 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated
Development

Successfully merging this pull request may close these issues.

5 participants