EngineLayer::dispose

[dnfield]

Part of flutter/flutter#81514

In effect, we end up with three layer trees:

• The Flutter framework (dart) Layer tree, which is how the framework abstracts SceneBuilder. Those layers may or may not hold refernces to EngineLayers. Some framework layers have no corresponding engine layer even though they have a corresponding flow layer (e.g. PictureLayer). Some have no corresponding engine or flow layer.
• The raster/flow layer tree, which is created on the native side via SceneBuilder and referred to by EngineLayer objects. It may or may not create additional layers that the framework doesn't know about. These layers contain pointers to other layers (child layers), and may also contain pointers to Skia objects (SkShader/SkPicture in particular).
• The EngineLayer tree, which links between framework and flow layers. It has no notions of parent/sibling/child relationships, and exists solely to help the framework with retained rendering.

The main problem is that the linkage between the framework layers means that flow layers cannot be fully cleaned up until a GC is run. This in turn means that some Skia objects which are no longer necessary get stuck until a GC.

Once this patch has landed and the framework consumes the dispose API correctly, we won't need to trigger GCs on image disposal anymore. We will also much more efficiently clean up Skia and flow Layer* related memory before waiting for a GC, even with the engine holding on to the last_layer_tree at times.

This still needs tests. Marking as draft to start running more tests than I've done locally.

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene wiki page, which explains my responsibilities.
• I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
• I listed at least one issue that this PR fixes in the description above.
• I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on
  writing and running engine tests.
• I updated/added relevant documentation (doc comments with ///).
• I signed the CLA.
• All existing and new tests are passing.
• The reviewer has submitted any presubmit flakes in this PR using the engine presubmit flakes form before re-triggering the failure.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[dnfield]

@goderbauer FYI since I'll be asking you to review framework side changes related to this.

[jason-simmons]

LGTM

[jason-simmons]

Would it make sense to set _nativeLayer to null after disposing instead of maintaining the _debugDisposed flag?

[dnfield]

I guess we could make it nullable and avoid the flag. I can change that.

[dnfield]

Done - I was worried this would add a ton of null checks, but we basically treat it as nullable in most places anyway.

[goderbauer]

@dnfield Is there a better view of how this will be integrated into the framework? The diff posted in the issue (https://github.com/flutter/flutter/compare/master..dnfield:disp_pict) seems to contain a lot of unrelated stuff...

[dnfield]

@goderbauer - everything in that diff is related, even if it seems like it's not :. I'll be splitting that up into the render object portion and the layer portion soon though.

[dnfield]

Ahhh try https://github.com/flutter/flutter/compare/dnfield:disp_pict

[goderbauer]

Ahhh try https://github.com/flutter/flutter/compare/dnfield:disp_pict

That looks better. Added to my queue.

[zanderso]

This lgtm. @yjbanov and @chinmaygarde should review as well.

[dnfield]

One thing I want to think about some more is what to do with oldLayer on the SceneBuilder API. It seems like we should be disposing it after we use it, and documenting that the caller must not use it again after that. Or, we should be documenting that the caller has to dispose it, which seems strange.

[fluttergithubbot]

This pull request is not suitable for automatic merging in its current state.

• The status or check suite Linux Host Engine has failed. Please fix the issues identified (or deflake) before re-applying this label.
• The status or check suite Mac Host Engine has failed. Please fix the issues identified (or deflake) before re-applying this label.