Binary compatibility and inline accessors

[nicolasstucki]

Compiler version

3.0, 3.1, 3.2, 3.3

Inline accessors

Whenever we refer to a definition that is not public in an inline definition, we generate an inline accessor to access that definition from where the code is inlined.

final class C:
  private def a: Int = 2
  inline def b: Int = a + C.d

object C:
  private[C] val d: Int = 4

is transformed into something like

final class C:
  private def a: Int = 2
  def inline$a = a
  def inline$d = C.d
  inline def b: Int = inline$a + inline$d
object C:
  private[C] val d: Int = 4

Note that the accessors are added to the class using the private definition. This can lead to duplication of accessors.

Issue 1: Changing any definition from private to public is a binary incompatible change

object C:
- private def a: Int = 2
+ def a: Int = 2
  inline def b: Int = a

This change would remove the need to generate C$$inline$a because a is publicly accessible. But any compiled code that inlined b before this change refers to C$$inline$a. This code can be located in different libraries, and we might be unable to recompile it with the new inline definition.

Issue 2: Changing the implementation of an inline definition can be a binary incompatible change

object C:
  private def a: Int = 2
-  inline def b: Int = a
+  inline def b: Int = 3

This change would remove the need to generate C$$inline$a because we never inline a. But any compiled code that inlined b before this change refers to C$$inline$a. This code can be located in different libraries, and we might be unable to recompile it with the new inline definition.

Issue 3: Removing final from a class is a binary incompatible change

- final class C:
+ class C:
  private def a: Int = 2
  inline def b: Int = a

If the class is final a will get the accessor inline$a but if it is not final it gets the accessor C$$inline$a.

[Kordyjan]

I think it is interesting how Kotlin deals with that. They could have the same problem, but they solved it by requiring every symbol used from the inline function to be either public or have a @PublishedApi annotation.

@PublishedApi makes annotated symbol look in the output bytecode the same way as if it was public. Of course, the proper visibility is still checked during the compiletime. Therefore

object C {
-   @PublishedApi internal fun sth() = TODO()
+   fun sth() = TODO()
}

is a safe and backward-compatible change.

The benefit of doing it only through annotation and not automatically is that all symbols used from the inline functions are documented, so the maintainers know they have the same compatibility guarantees as public symbols.

To solve our issue, I suggest taking inspiration from the Kotlin approach:
We would leave the accessors' generation as it is. We would also introduce a new annotation in the standard library (let's call it @api). In 3.3, each time we generate an inline accessor, we should check if the accessed symbol is annotated with the @api annotation; if not, we issue a warning. We should change the warning into a proper error in some future versions.

The @api annotation will force the generation of the accessor and leave a visible trace that the annotated symbol is part of the API and is subject to the same compatibility guarantees as any public symbol.

This still leaves

object C:
-  @api private def a: Int = 2
+  def a: Int = 2
}

as an unsafe change breaking binary compatibility, but it is at least well documented.

[nicolasstucki]

A possible solution would be to add a @inlineAccessible annotation to make a public inline accessor for that definition. Inline methods will use these inline accessors if they exist.

final class C:
  @inlineAccessible private def a: Int = 2
  def inline$a = a // generated by @inlineAccessible
  inline def b: Int = a // use inline$a because a has @inlineAccessible

We would also need to phase out the old inline accessor generation after this feature is added. We could start warning users when a dangerous inline accessor is generated.

[nicolasstucki]

I forgot to click on Comment and in meantime @Kordyjan #16983 (comment) proposed the same idea. I already coded a prototype.

[nicolasstucki]

A possible problem with the annotation approach is how to make top-level objects accessible

class C:
  inline def f: Int = C.n
@inlineAccessible private object C:
  def n: Int  = 3

Not sure if a static method in class C$ would work.

[sjrd]

We could take it a step further and decouple it from the inline use case. I would suggest to call it @binaryAPI (and perhaps a variant @binaryAPI(binaryAPI.Protected)). If such a method is accessed from an inline we call it as is instead of generating an inline accessor.

But it then has other uses. In particular, it can be used for binary compatibility scenarios. If I want to make a public method private in a new version, I can mark it as @binaryAPI to preserve binary compatibility. Currently, we use the private[package] trick for that, but it leads to issues because MiMa does not respect transitivity for package-private stuff.

[nicolasstucki]

@sjrd how would @binaryAPI work on private[this] definitions? For example

class A:
  @binaryAPI private val x = 1
  inline def f = x
class B extends A:
  @binaryAPI private val x = 2
  inline def g = x

We cannot make those definitions public or package private in the binary as they would clash. Maybe we don't allow those.

[sjrd]

That's a good point. I think we would have to refuse those, indeed. Not sure whether that makes sense 🤔