Migrate camera/android from SurfaceTexture->SurfaceProducer.

[matanlurey]

WIP: We do not plan to land this PR until the next stable release (>= April 3rd 2024).

Work towards flutter/flutter#145930.

Details

Migrates uses of createSurfaceTexture to createSurfaceProducer, which is intended to have no change in behavior, but does change the backend rendering path, so it will require more testing (and we're also open to minor API renames or changes before it becomes stable).

Background

Android plugins previously requested a SurfaceTexture from the Android embedder, and used that to produce a Surface to render external textures on (i.e. video_player). This worked because 100% of Flutter applications on Android used OpenGLES (via our Skia backend), and SurfaceTexture is actually an (opaque) OpenGLES-texture.

Starting soon (roughly ~Q3, this is not a guarantee and just an estimate), Flutter on Android will start to use our new Impeller graphics backend, which on newer devices (>= API_VERSION_28), will default to the Vulkan, not OpenGLES. In other words, SurfaceTexture will cease to work (it is possible, but non-trivial, to map an OpenGLES texture over to Vulkan).

After consultation with the Android team, they helped us understand that vending SurfaceTexture (the consumer-side API) was never the right abstraction, and we should have been vending the producer-side API, or Surface directly. The new SurfaceProducer API is exactly that - it generates a Surface, and similar to our platform view strategy, picks the "right" consumer-side implementation details for the user/plugin packages.

The new SurfaceProducer API has 2 possible rendering types (as an implementation detail):

• SurfaceTexture, for older OpenGLES devices, which works exactly as it does today.
• ImageReader, for newer OpenGLES or Vulkan devices.

These are some subtle nuances in how these two APIs work differently (one example: flutter/flutter#144407), but our theory at this point is we don't expect these changes to be observed by any users, and we have other ideas if necessary.

Note

These invariants are tested on CI in flutter/engine.

Points of contact:

• @matanlurey or @jonahwilliams (Flutter Engine)
• @johnmccutchan or @reidbaker (Flutter on Android)

[matanlurey]

Due to a release hiccup, the stable branch (3.22) is temporarily ahead of the master branch (3.22.0-35.0.pre.34), leading to CI failures (see discord discussion). I've merged back into HEAD and updated the pubspec/CHANGELOG - the rest might be done by someone else while I'm on vacation for a couple of weeks.

[stuartmorgan]

Is this file a local conflict resolution mistake?

[jonahwilliams]

probably something went wrong applying the formatting diff.

[jonahwilliams]

hmm, that formatting didn't work either

[jonahwilliams]

Thank you @stuartmorgan !

[jonahwilliams]

Current test failure is:

java.lang.Exception: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PlatformException was thrown running a test:
PlatformException(error, Attempt to invoke virtual method 'int
android.hardware.camera2.CameraCaptureSession.capture(android.hardware.camera2.CaptureRequest,
android.hardware.camera2.CameraCaptureSession$CaptureCallback,
android.os.Handler)' on a null object reference, null,
java.lang.NullPointerException: Attempt to invoke virtual method
'int
android.hardware.camera2.CameraCaptureSession.capture(android.hardware.camera2.CaptureRequest,
android.hardware.camera2.CameraCaptureSession$CaptureCallback,
android.os.Handler)' on a null object reference
at
io.flutter.plugins.camera.Camera.runPrecaptureSequence(Camera.java:660)
at io.flutter.plugins.camera.Camera.takePicture(Camera.java:645)
at
io.flutter.plugins.camera.MethodCallHandlerImpl.onMethodCall(MethodCallHandlerImpl.java:112)
at
io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:267)
at
io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:292)
at
io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$io-flutter-embedding-engine-dart-DartMessenger(DartMessenger.java:319)
at
io.flutter.embedding.engine.dart.DartMessenger$$ExternalSyntheticLambda0.run(Unknown
Source:12)

[jonahwilliams]

@johnmccutchan @camsim99 PTAL

[camsim99]

LGTM