[pigeon] Kotlin implementation for ProxyApis #6371

bparrishMines · 2024-03-21T19:24:05Z

Kotlin portion of flutter/flutter#134777

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

bparrishMines · 2024-03-22T20:30:15Z

...tform_tests/test_plugin/android/src/main/kotlin/com/example/test_plugin/ProxyApiTests.gen.kt

+abstract class PigeonProxyApiBaseCodec(
+    val binaryMessenger: BinaryMessenger,
+    val instanceManager: PigeonInstanceManager
+) : StandardMessageCodec() {
+  /**
+   * An implementation of [PigeonApiProxyApiTestClass] used to add a new Dart instance of
+   * `ProxyApiTestClass` to the Dart `InstanceManager`.
+   */
+  abstract fun getPigeonApiProxyApiTestClass(): PigeonApiProxyApiTestClass
+
+  /**
+   * An implementation of [PigeonApiProxyApiSuperClass] used to add a new Dart instance of
+   * `ProxyApiSuperClass` to the Dart `InstanceManager`.
+   */
+  abstract fun getPigeonApiProxyApiSuperClass(): PigeonApiProxyApiSuperClass
+
+  /**
+   * An implementation of [PigeonApiProxyApiInterface] used to add a new Dart instance of
+   * `ProxyApiInterface` to the Dart `InstanceManager`.
+   */
+  abstract fun getPigeonApiProxyApiInterface(): PigeonApiProxyApiInterface
+
+  fun setUpMessageHandlers() {
+    PigeonApiProxyApiTestClass.setUpMessageHandlers(
+        binaryMessenger, getPigeonApiProxyApiTestClass())
+    PigeonApiProxyApiSuperClass.setUpMessageHandlers(
+        binaryMessenger, getPigeonApiProxyApiSuperClass())
+  }
+
+  override fun readValueOfType(type: Byte, buffer: ByteBuffer): Any? {
+    return when (type) {
+      128.toByte() -> {
+        return instanceManager.getInstance(
+            readValue(buffer).let { if (it is Int) it.toLong() else it as Long })
+      }
+      else -> super.readValueOfType(type, buffer)
+    }
+  }
+
+  override fun writeValue(stream: ByteArrayOutputStream, value: Any?) {
+    if (value is ProxyApiTestClass) {
+      getPigeonApiProxyApiTestClass().pigeon_newInstance(value) {}
+    } else if (value is com.example.test_plugin.ProxyApiSuperClass) {
+      getPigeonApiProxyApiSuperClass().pigeon_newInstance(value) {}
+    } else if (value is ProxyApiInterface) {
+      getPigeonApiProxyApiInterface().pigeon_newInstance(value) {}
+    }
+
+    when {
+      instanceManager.containsInstance(value) -> {
+        stream.write(128)
+        writeValue(stream, instanceManager.getIdentifierForStrongReference(value))
+      }
+      else -> super.writeValue(stream, value)
+    }
+  }
+}


@camsim99 FYI

@stuartmorgan @tarrinneal This is one of the places where the design changes from the implementation in the doc. I wanted to improve the scenario of when a method returns a ProxyApi or a list/map of a ProxyApi:

class Basket { Apple giveMeAnApple(); List<Apple> giveMeApples(); }

Typically we solve this by manually handling adding a new instance when we implement the api:

class PigeonApiBasketImpl { Apple giveMeAnApple(Basket instance) { final Apple apple = instance.giveMeAnApple(); if (!instanceManager.containsInstance(apple)) { instanceManager.addHostCreatedInstance(apple); } return apple; } List<Apple> giveMeApples(Basket instance) { final List<Apple> apples = instance.giveMeApples(); for (Apple apple in apples) { if (!instanceManager.containsInstance(apple)) { instanceManager.addHostCreatedInstance(apple); } } return apples; } }

This change makes the codec the central class that contains all the proxy api implementations, so the writeValue can handle creating new dart instances automatically by calling pigeon_newInstance. And pigeon_newInstance would handle the logic shown in the methods above.

This also:

Creates a registrar like class that ensures a user implements each proxy api.

Provides access to a central class with every api implementation which prevents the need for multiple instances of the same api implementation.

This may end up overlapping with some of the codec work I'm about to do as well. I think it's generally a good idea to create a central codec though. We may end up with multiple codecs implementing each other.

My main concern at this point is that the number of custom inputs is limited to around 120, so if there are a lot of proxy apis, enums, and or data classes, we may not be able to accommodate them all.

As of right now, the generation of the ProxyApi codec is separate from Host/Flutter APIs codecs. Mainly because:

ProxyApis don't support data classes

ProxyApis support other ProxyApis in the codec and Host/Flutter APIs don't.

So they currently don't really share anything. If you create a central custom codec for all the APIs, then ProxyApis should only need one dedicated spot that uses the InstanceManager: https://github.com/flutter/packages/blob/main/packages/pigeon/platform_tests/shared_test_plugin_code/lib/src/generated/proxy_api_tests.gen.dart#L374.

~~Also, as a side note, since all data classes have an encode/decode could you create a base data class for all the data classes and only use one spot.~~ I realized this wouldn't work since the reader wouldn't know what class to call this on.

…_split

tarrinneal · 2024-04-10T20:54:07Z

Are you planning to add the full implementation and test suite to this pr?

bparrishMines · 2024-04-11T16:13:11Z

Are you planning to add the full implementation and test suite to this pr?

I wasn't planning on doing that for this PR, but I can if you prefer the full implementation. This PR just adds the Kotlin ProxyApi utility classes, including the InstanceManager and the tests for it. And also the bare minimum ProxyApi class declaration.

…_split

bparrishMines · 2024-04-17T19:51:06Z

@tarrinneal I updated the PR to have the full implementation and tests. Its ready for a review.

…_split

tarrinneal

quick little glance through, I think most of this pr is good as is. Any suggestions I've made here are generally open for discussion if you want to keep them.

tarrinneal · 2024-05-16T23:59:55Z

packages/pigeon/lib/ast.dart

@@ -4,6 +4,7 @@

 import 'package:collection/collection.dart' show ListEquality;
 import 'package:meta/meta.dart';
+import 'kotlin_generator.dart';


I don't love this being imported to this file. Can we move the kotlin options into the normal kotlin options class?

tarrinneal · 2024-05-17T18:28:09Z

packages/pigeon/lib/generator_tools.dart

@@ -307,6 +307,12 @@ const String seeAlsoWarning = 'See also: https://pub.dev/packages/pigeon';
 /// parameters.
 const String classNamePrefix = 'Pigeon';

+/// Prefix for apis generated for ProxyApi.


tarrinneal · 2024-05-17T18:29:01Z

packages/pigeon/lib/generator_tools.dart

@@ -494,6 +500,48 @@ Map<TypeDeclaration, List<int>> getReferencedTypes(
  return references.map;
 }

+/// Find the [TypeDeclaration] that has the highest api requirement and its


tarrinneal · 2024-05-17T18:55:01Z

packages/pigeon/lib/kotlin/templates.dart

+  private val identifiers = java.util.WeakHashMap<Any, Long>()
+  private val weakInstances = HashMap<Long, java.lang.ref.WeakReference<Any>>()
+  private val strongInstances = HashMap<Long, Any>()
+  private val referenceQueue = java.lang.ref.ReferenceQueue<Any>()
+  private val weakReferencesToIdentifiers = HashMap<java.lang.ref.WeakReference<Any>, Long>()


I'm sure you've already figured out this is the best way, but why is it that we need the java Weak version of these data structures? Is there no kotlin version that's usable?

tarrinneal · 2024-05-17T21:07:23Z

packages/pigeon/lib/kotlin/templates.dart

+    fun setUpMessageHandlers(binaryMessenger: BinaryMessenger, instanceManager: $instanceManagerClassName?) {
+      run {
+        val channel = BasicMessageChannel<Any?>(binaryMessenger, "$removeStrongReferenceName", codec)
+        if (instanceManager != null) {
+          channel.setMessageHandler { message, reply ->
+            val identifier = message as Number
+            val wrapped: List<Any?> = try {
+              instanceManager.remove<Any?>(identifier.toLong())
+              listOf<Any?>(null)
+            } catch (exception: Throwable) {
+              wrapError(exception)
+            }
+            reply.reply(wrapped)
+          }
+        } else {
+          channel.setMessageHandler(null)
+        }  
+      }
+      run {
+        val channel = BasicMessageChannel<Any?>(binaryMessenger, "$clearName", codec)
+        if (instanceManager != null) {
+          channel.setMessageHandler { _, reply ->
+            val wrapped: List<Any?> = try {
+              instanceManager.clear()
+              listOf<Any?>(null)
+            } catch (exception: Throwable) {
+              wrapError(exception)
+            }
+            reply.reply(wrapped)
+          }
+        } else {
+          channel.setMessageHandler(null)
+        }
+      }
+    }
+  }


a lot of this code is essentially duplicated with the host api. I'd prefer if that code could be gened from the same code. to avoid future work duplication

tarrinneal · 2024-05-17T21:21:53Z

packages/pigeon/lib/kotlin_generator.dart

+      () {
+        indent.writeln('val instanceManager: PigeonInstanceManager');
+        indent.format(
+          'private var _codec: StandardMessageCodec? = null\n'


This will need to change to use the new codec - once the codec pr lands of course

…_split

bparrishMines added 4 commits March 21, 2024 15:00

add other stuff

7d66bf7

add partial generator code

d64d2a2

add generated file

b64d723

finish additions

89d3c92

github-actions bot added p: pigeon platform-android labels Mar 21, 2024

some fixes

1f7d0ad

bparrishMines changed the title ~~Pigeon kotlin split~~ [pigeon] Initial kotlin implementation for ProxyApis Mar 22, 2024

bparrishMines marked this pull request as ready for review March 22, 2024 19:23

bparrishMines requested a review from tarrinneal as a code owner March 22, 2024 19:23

bparrishMines requested a review from stuartmorgan March 22, 2024 19:23

bparrishMines commented Mar 22, 2024

View reviewed changes

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

fa3c943

…_split

bparrishMines added 10 commits April 11, 2024 12:19

add onWriteBody

2665c22

use a common method for api math

cadf14c

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

2bf6738

…_split

add all stuff back

6437fc9

changelogg

aea71cd

formatting

0c09e91

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

f114c8c

…_split

fix proxy api separated suffix

d57ac0d

switch to a registrar

ff1e210

docs

7e345ac

bparrishMines changed the title ~~[pigeon] Initial kotlin implementation for ProxyApis~~ [pigeon] Kotlin implementation for ProxyApis Apr 16, 2024

bparrishMines mentioned this pull request Apr 16, 2024

[pigeon] Adds Kotlin implementation for @ProxyApi #6231

Closed

11 tasks

bparrishMines added 3 commits April 16, 2024 14:55

formatting

db2a198

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

a7c94e5

…_split

fix unit tests

7af4343

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

a7b20a0

…_split

bparrishMines added 4 commits April 17, 2024 16:07

improve helper method and add test

f2a1d5b

add a setter for clearFinalizedWeakReferencesInterval

f76265e

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

b8e70f6

…_split

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

4f2327d

…_split

huycozy mentioned this pull request Apr 29, 2024

How to use the ProxyApi annotation? flutter/flutter#147486

Closed

bparrishMines added 5 commits May 7, 2024 15:44

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

5ade010

…_split

return empty list in removeStrongReference

3b3a91b

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

ec37d62

…_split

update generated file

186fe47

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

894e642

…_split

tarrinneal reviewed May 17, 2024

View reviewed changes

bparrishMines added 3 commits May 20, 2024 11:55

make private codec accessible and use open for pigeonRegistrar

118606a

Merge branch 'main' of github.com:flutter/packages into pigeon_kotlin…

4e821cb

…_split

nits

4205418

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[pigeon] Kotlin implementation for ProxyApis #6371

[pigeon] Kotlin implementation for ProxyApis #6371

bparrishMines commented Mar 21, 2024 •

edited

bparrishMines Mar 22, 2024 •

edited

tarrinneal Apr 1, 2024

tarrinneal Apr 1, 2024

bparrishMines Apr 3, 2024 •

edited

tarrinneal commented Apr 10, 2024

bparrishMines commented Apr 11, 2024

bparrishMines commented Apr 17, 2024

tarrinneal left a comment

tarrinneal May 16, 2024

tarrinneal May 17, 2024

tarrinneal May 17, 2024

tarrinneal May 17, 2024

tarrinneal May 17, 2024

tarrinneal May 17, 2024

[pigeon] Kotlin implementation for ProxyApis #6371

Are you sure you want to change the base?

[pigeon] Kotlin implementation for ProxyApis #6371

Conversation

bparrishMines commented Mar 21, 2024 • edited

Pre-launch Checklist

bparrishMines Mar 22, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bparrishMines Apr 3, 2024 • edited

Choose a reason for hiding this comment

tarrinneal commented Apr 10, 2024

bparrishMines commented Apr 11, 2024

bparrishMines commented Apr 17, 2024

tarrinneal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bparrishMines commented Mar 21, 2024 •

edited

bparrishMines Mar 22, 2024 •

edited

bparrishMines Apr 3, 2024 •

edited