(almost) allocationless tags ⚡ #348

fwbrasil · 2024-05-10T22:30:40Z

Fixes #339.

Kyo uses tags for effect handling. Izumi's performance has been an important limitation, especially to provide sub-type checking it hot paths. This PR introduces a new Tag implementation that bundles the type information in a string format amenable to type comparisons without requiring allocations. Since the string is generated at compile-time, it becomes a constant in the class pool and doesn't require allocations.

The only scenario where this new Tag can allocate is when one of the type parameters has an implicit tag. For example:

def test[T: Tag] = Tag[List[T]]

In this case, the macro generates a tree that will concatenate the string representing Tag[T] into Tag[List[T]]. JIT compilers can many times avoid such allocation, though.

This change introduces an important restriction: only types with concrete class symbols (traits, classes, objects) are supported. We could extend the implementation to support intersection and union types as well but I think allowing these types can be confusing to users since they can be produced by inference when a user makes a mistake.

The performance is ~10x in comparison to Izumi:

The main reason is zero allocation:

https://jmh.morethan.io/?source=https://gist.githubusercontent.com/fwbrasil/7010fc054376ce94e3497de2fbeb3030/raw/0ba5e81e932185ef4b10c75201e0ace98aac4a33/jmh-result.json#details

fwbrasil · 2024-05-10T22:34:15Z

build.sbt

@@ -102,7 +102,6 @@ lazy val `kyo-core` =
        .in(file("kyo-core"))
        .settings(
            `kyo-settings`,
-            libraryDependencies += "dev.zio"       %%% "izumi-reflect"   % "2.3.9",


Izumi dependency only for benchmarks now.

fwbrasil · 2024-05-10T22:34:46Z

kyo-core/jvm/src/test/scala/kyoTest/coreBytecodeSizeTest.scala

@@ -38,7 +38,7 @@ class coreBytecodeSizeTest extends KyoTest:
        assert(map == Map(
            "test"        -> 28,
            "resultLoop"  -> 94,
-            "handleLoop"  -> 280,
+            "handleLoop"  -> 257,


fewer bytecodes to call =:=

fwbrasil · 2024-05-10T22:35:30Z

kyo-core/shared/src/main/scala/kyo/package.scala

-            inline expected: Tag[T]
+        inline def failTag(
+            inline actual: Tag[?],
+            inline expected: Tag[?]*


Since the new Tag doesn't support intersection types, I'm simulating them here for error reporting via a vararg param.

fwbrasil · 2024-05-10T22:35:58Z

kyo-core/shared/src/main/scala/kyo/tags.scala

 import scala.quoted.*

-opaque type Tag[T] = String
+case class Tag[T](tpe: String) extends AnyVal


Using AnyVal solved some issues I was having with type inference.

fwbrasil · 2024-05-10T22:36:54Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+
+    extension [T](t1: Tag[T])
+
+        def show = t1.tpe.takeWhile(_ != ';')


We should eventually improve this to show the entire type signature. This code only prints the outer class name, no type parameters. It's enough for the current error reporting use case, though.

Perhaps worth a pending test!

Makes sense! I've just added it.

fwbrasil · 2024-05-10T22:37:38Z

kyo-core/shared/src/test/scala/kyoTest/envsTest.scala

-            ()
-        }
+    "intersection type env" in {
+        assertDoesNotCompile("Envs.get[Int & Double]")


Doesn't compile anymore since tags don't allow intersection types.

fwbrasil · 2024-05-10T22:38:19Z

kyo-tapir/src/main/scala/kyo/routes.scala

@@ -9,7 +8,7 @@ import sttp.tapir.server.ServerEndpoint
 import sttp.tapir.server.netty.NettyKyoServer
 import sttp.tapir.server.netty.NettyKyoServerBinding

-type Route = ServerEndpoint[Any, KyoSttpMonad.M]
+case class Route(endpoint: ServerEndpoint[Any, KyoSttpMonad.M]) extends AnyVal


I had to change this since KyoSttpMonad.M isn't a concrete type.

fwbrasil · 2024-05-10T22:39:32Z

kyo-tapir/src/main/scala/kyo/routes.scala

        }
    end run

    def add[A: Tag, I, E: Tag: ClassTag, O: Flat](e: Endpoint[A, I, E, O, Any])(
        f: I => O < (Fibers & Envs[A] & Aborts[E])
    ): Unit < Routes =
-        Sums.add(List(


There was a bug here introduced by #267. The new Tag transformed it into a compilation error! \o/

fwbrasil · 2024-05-10T22:49:41Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+        //
+        // Tag[Test[Param1, Param2]]
+        //
+        // Test;Super;java.lang.Object;scala.Matchable;scala.Any;[-Param1;Super;java.lang.Object;scala.Matchable;scala.Any;,+Param2;Super;java.lang.Object;scala.Matchable;scala.Any;]


This format has some opportunity for optimization. We might be able to omit java.lang.Object;scala.Matchable;scala.Any and assume they're always present but I'm not sure about Matcheable.

fwbrasil · 2024-05-10T22:52:14Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+            end if
+        end checkParams
+
+        def nextParam(tag: String, idx: Int): Int =


This scan consumes a significant portion of the CPU usage in the benchmark. We can probably optimize it by encoding the length of elements in the string representation. Profiling session:

hearnadam

Very impressive performance improvement!

The change looks pretty good overall. Will take a closer look at the tests later.

Does izumi have tests we can copy? It would be good to highlight the overlap in functionality, as well as where Kyo tags may won't work (should be rare in practice).

I wasn't able to tell, but can you still invoke == on two tags? Can we somehow disallow this?

hearnadam · 2024-05-10T23:32:51Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+
+        def nextParam(tag: String, idx: Int): Int =
+            @tailrec def loop(opens: Int, idx: Int): Int =
+                tag.charAt(idx) match


Add a @switch annotation 🙏🏻

Thanks! I need to get used to this annotation. I had to avoid the pattern guards to add the annotation but it should still improve performance. I added it in a couple of other places as well.

hearnadam · 2024-05-11T00:52:45Z

kyo-core/shared/src/main/scala/kyo/tags.scala

-    given canEqual[T, U]: CanEqual[Tag[T], Tag[U]] = CanEqual.derived
+    import internal.*
+
+    inline given apply[T >: Nothing]: Tag[T] = ${ tagImpl[T] }


Will the >: Nothing be problematic for Aborts? (Or any effect)

Good question! I think we actually shouldn't allow Tag[Nothing]. In Kyo's usage of tags, it's essentially an inconsistent state since tags are meant to represent concrete types, and Nothing, although it has a class symbol, has no runtime representation. I've changed the macro to fail for Tag[Nothing] and added a test for it. It caught an inssue in streamsTest! I've removed the type constraint since it isn't strict, and the macro will detect it anyway. Nothing can still appear in nested type parameters. I'm not sure if we should constraint it too.

hearnadam · 2024-05-11T00:54:57Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+        def show = t1.tpe.takeWhile(_ != ';')
+
+        def <:<[U](t2: Tag[U]): Boolean =
+            t1.tpe == t2.tpe || isSubtype(t1, t2)


Do we need to check reference equality? Or does the default string == do so?

Nice catch! It doesn't, fixed.

hearnadam · 2024-05-11T00:57:24Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+        def =!=[U](t2: Tag[U]): Boolean =
+            t1.tpe != t2.tpe
+
+        def >:>[U](t2: Tag[U]): Boolean =


Can't this be defined in terms of the other direction subtype method? Do we need this method?

I prefer the freedom to order operands in both directions, it sometimes make the code more linear.

hearnadam · 2024-05-11T01:00:02Z

kyo-core/shared/src/test/scala/kyoTest/envsTest.scala

-            ()
-        }
-        "method inference" in pendingUntilFixed {
-            def test[T](v: T < (Envs[Int] & Envs[Double])) =


Isn't this still relevant?

Both scenarios fail because of the intersection tag and scenarios that test compilation failures are relatively fragile since they can break for different reasons as the codebase evolves. I don't see much value in keeping them. I'd say this is more of a wontfix for now but we can review if we explore supporting intersection and union types in tags.

fwbrasil · 2024-05-11T05:03:06Z

Thanks for the review! 🙏

Does izumi have tests we can copy? It would be good to highlight the overlap in functionality, as well as where Kyo tags may won't work (should be rare in practice).

Izumi can represent many more kinds types than Kyo's tags but I think it's actually a good thing to have something more constrained. There are a few tests for unsupported types.

I wasn't able to tell, but can you still invoke == on two tags? Can we somehow disallow this?

It's only possible to restrict it if the compiler is configured with strict equality enabled, which is the case in Kyo's codebase. If the flag is enabled, == won't be available since Tag doesn't provide a CanEqual.

hearnadam · 2024-05-11T16:27:35Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+                        case _ =>
+                            report.errorAndAbort(s"Unsupported Tag type ${tpe.show}")


Can we make the error messages unique to assist with debugging issues? When users submit tickets it may be useful to know which case they are hitting

hearnadam · 2024-05-11T16:29:13Z

kyo-core/shared/src/test/scala/kyoTest/streamsTest.scala

@@ -321,7 +321,7 @@ class streamsTest extends KyoTest:
        "none" in {
            assert(
                Streams.initSeq(Seq(1, 2, 3)).collect {
-                    case v if false => ???
+                    case v if false => (): Unit < Streams[Int]


Do we need a generalized unit method to improve inference?

object Kyo: def unit[S]: Unit < S = ()

I'd say this is an edge case for testing. I think this compilation error would happen most often when users make mistakes for example not composing effects correctly.

hearnadam · 2024-05-11T16:31:46Z

kyo-core/shared/src/main/scala/kyo/tags.scala

+
+    extension [T](t1: Tag[T])
+
+        def show = t1.tpe.takeWhile(_ != ';')


Perhaps worth a pending test!

fwbrasil · 2024-05-12T12:35:34Z

Thanks for the reviews @hearnadam! I'll merge this and work on support for union and intersection types in a separate PR.

fwbrasil commented May 10, 2024

View reviewed changes

hearnadam reviewed May 11, 2024

View reviewed changes

hearnadam approved these changes May 11, 2024

View reviewed changes

fwbrasil force-pushed the new-tag branch from 5c9bc89 to 6ae20bb Compare May 11, 2024 18:22

calvinlfer mentioned this pull request May 11, 2024

Using Streams, IOs and Fibers cause an exception to occur #339

Closed

hearnadam approved these changes May 12, 2024

View reviewed changes

fwbrasil added 7 commits May 12, 2024 05:27

(almost) allocationless tags

3573eea

don't allow Tag[Nothing]

c3ae809

optimization: check tag strings for referential equality first

f798e0e

optimization: @switch for char matches

475a165

fix rebase conflicts

8e7426a

tags: show tests with pending scenario

beb01b8

remove unused import

769dc9f

fwbrasil force-pushed the new-tag branch from ee822bd to 769dc9f Compare May 12, 2024 12:28

tags: improve error messages

9afd8c6

fwbrasil merged commit d50d2fd into main May 12, 2024
3 checks passed

fwbrasil deleted the new-tag branch May 12, 2024 12:56

fwbrasil mentioned this pull request May 12, 2024

move effect command from member type to type parameter #356

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(almost) allocationless tags ⚡ #348

(almost) allocationless tags ⚡ #348

fwbrasil commented May 10, 2024 •

edited

Loading

fwbrasil May 10, 2024

fwbrasil May 10, 2024

fwbrasil May 10, 2024

fwbrasil May 10, 2024

fwbrasil May 10, 2024 •

edited

Loading

hearnadam May 11, 2024

fwbrasil May 11, 2024

fwbrasil May 10, 2024

fwbrasil May 10, 2024

fwbrasil May 10, 2024 •

edited

Loading

fwbrasil May 10, 2024

fwbrasil May 10, 2024

hearnadam left a comment •

edited

Loading

hearnadam May 10, 2024

fwbrasil May 11, 2024

hearnadam May 11, 2024

fwbrasil May 11, 2024 •

edited

Loading

hearnadam May 11, 2024

fwbrasil May 11, 2024

hearnadam May 11, 2024

fwbrasil May 11, 2024

hearnadam May 11, 2024

fwbrasil May 11, 2024

fwbrasil commented May 11, 2024 •

edited

Loading

hearnadam May 11, 2024

hearnadam May 11, 2024

fwbrasil May 11, 2024

hearnadam May 11, 2024

fwbrasil commented May 12, 2024


		extension [T](t1: Tag[T])

		def show = t1.tpe.takeWhile(_ != ';')

		case _ =>
		report.errorAndAbort(s"Unsupported Tag type ${tpe.show}")

(almost) allocationless tags ⚡ #348

(almost) allocationless tags ⚡ #348

Conversation

fwbrasil commented May 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwbrasil May 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwbrasil May 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hearnadam left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwbrasil May 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwbrasil commented May 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fwbrasil commented May 12, 2024

fwbrasil commented May 10, 2024 •

edited

Loading

fwbrasil May 10, 2024 •

edited

Loading

fwbrasil May 10, 2024 •

edited

Loading

hearnadam left a comment •

edited

Loading

fwbrasil May 11, 2024 •

edited

Loading

fwbrasil commented May 11, 2024 •

edited

Loading