`IOLocal` propagation for unsafe access #3636

armanbilge · 2023-05-16T19:34:54Z

Still needs quite a bit of work, but wanted to sketch the basic idea.

Goal: to expose a fiber's IOLocals as ThreadLocals within a side-effecting block, so that they can be accessed and modified in unsafe land.

Motivation: basically telemetry Java interop.

Constraints: to do this as safely as possible 😅

armanbilge · 2023-05-16T19:36:50Z

core/jvm/src/main/java/cats/effect/IOFiberConstants.java

@@ -43,4 +43,6 @@ final class IOFiberConstants {
  static final byte CedeR = 6;
  static final byte AutoCedeR = 7;
  static final byte DoneR = 8;
+
+  static final boolean dumpLocals = Boolean.getBoolean("cats.effect.tracing.dumpLocals");


Bikesheddable configuration for opting-in. So the rest of us don't have to pay the penalty 😇

Is this specifically "tracing", even if that's the most obvious use case?

Oh woops, this was a very lazy copy-pasta. I copied it from the system properties we use to configure fiber tracing. We should rename it anyway, dumpLocals is not quite right I think 😅

What about cats.effect.localContextPropagation similar to Monix's monix.environment.localContextPropagation?

Thanks, I liked that! I went with cats.effect.ioLocalPropagation.

armanbilge · 2023-05-16T19:37:25Z

core/shared/src/main/scala/cats/effect/IOFiber.scala

+          var locals: IOLocals = null
+          if (dumpLocals) {
+            locals = new IOLocals(localState)
+            IOLocals.threadLocal.set(locals)
+          }


I did this just for delay, but I guess blocking and interruptible would want it too.

armanbilge · 2023-05-16T19:39:47Z

core/shared/src/main/scala/cats/effect/unsafe/IOLocals.scala

+// TODO handle defaults and lenses. all do-able, just needs refactoring ...
+final class IOLocals private[effect] (private[this] var state: IOLocalState) {


Ok, so I was extremely lazy with implementing this thing. But the main idea of this wrapper API is that it should only give the user access to IOLocals that they know about i.e. they should not able to clear out other locals that happen to be present.

core/shared/src/main/scala/cats/effect/IOFiber.scala

core/shared/src/main/scala/cats/effect/unsafe/IOLocals.scala

armanbilge · 2023-05-17T01:04:51Z

core/shared/src/main/scala/cats/effect/IO.scala

    val fiber = new IOFiber[A](
-      Map.empty,
+      if (IOFiberConstants.dumpLocals) unsafe.IOLocals.getState else Map.empty,


We can even go in the opposite direction for IO#unsafeRun* 😁

It's less clear if/how to do this for fibers started in a Dispatcher, since they should be inheriting locals from the fiber backing the Dispatcher.

rossabaker

Interesting. I will try to give a snapshot of this a shot over on the otel4s side.

rossabaker · 2023-05-18T02:16:21Z

core/jvm/src/main/java/cats/effect/IOFiberConstants.java

@@ -43,4 +43,6 @@ final class IOFiberConstants {
  static final byte CedeR = 6;
  static final byte AutoCedeR = 7;
  static final byte DoneR = 8;
+
+  static final boolean dumpLocals = Boolean.getBoolean("cats.effect.tracing.dumpLocals");


Is this specifically "tracing", even if that's the most obvious use case?

core/shared/src/main/scala/cats/effect/unsafe/IOLocals.scala

armanbilge · 2023-05-21T03:00:26Z

core/jvm/src/main/java/cats/effect/IOFiberConstants.java

 // defined in Java since Scala doesn't let us define static fields
-final class IOFiberConstants {
+public final class IOFiberConstants {


This is not good. To avoid this we'll either have to replicate it at both the cats.effect and cats.effect.unsafe levels, or move the thread-local IOLocals accessors into cats.effect.

kevin-lee · 2023-06-23T14:31:18Z

I apologise for jumping in and asking this question but, can it be used as an alternative to Local from Monix?
I need it specifically for logging, much like what's explained in the post, "Better logging with Monix 3, part 1: MDC" but with cats-effect 3. This has been a blocker for my company's transition from cats-effect 2 with Monix to cats-effect 3. The issue has become increasingly critical as more and more libraries cease to support cats-effect 2, choosing to support only cats-effect 3 instead. So it would be great to see if there's any upcoming solution.

armanbilge · 2023-06-23T15:03:08Z

@kevin-lee no problem! At a glance, yes, this does look like an alternative/replacement to that Monix feature. Perhaps @alexandru can confirm :)

kevin-lee · 2023-06-28T14:15:10Z

@kevin-lee no problem! At a glance, yes, this does look like an alternative/replacement to that Monix feature. Perhaps @alexandru can confirm :)

@armanbilge Thank you. That's great!
It looks quite similar and looks like it can be used for the same purpose, but I can see the methods in IOLocals taking IOLocal[A] whereas the methods in Local from Monix don't. So I'm wondering if it can be used for the same purpose. I need to use it in a logging library like Logback, which doesn't have IO or any effect.
Yeah, it would be nice if @alexandru can confirm this.

armanbilge · 2023-06-28T14:31:49Z

but I can see the methods in IOLocals taking IOLocal[A] whereas the methods in Local from Monix don't. So I'm wondering if it can be used for the same purpose. I need to use it in a logging library like Logback, which doesn't have IO or any effect.

@kevin-lee the IOLocal[A] is just the "key". Once you have that key (and you can build one unsafely outside of the effect) then you don't need any more effects to read and write it.

For an example integration, see this PR which implements a Java SPI using this IOLocals API.

Experimental IOLocalContextStorage otel4s#214

kevin-lee · 2023-06-28T14:39:02Z

@kevin-lee the IOLocal[A] is just the "key". Once you have that key (and you can build one unsafely outside of the effect) then you don't need any more effects to read and write it.

For an example integration, see this PR which implements a Java SPI using this IOLocals API.

Experimental IOLocalContextStorage otel4s#214

@armanbilge Oh... got it. It looks very promising then. Thank you!

armanbilge · 2023-09-30T16:45:33Z

I did a quick benchmark and this feature is definitely not free 😕

ioLocalPropagation=false

[info] Benchmark                (size)   Mode  Cnt     Score     Error  Units
[info] DeepBindBenchmark.delay   10000  thrpt   20  4161.322 ± 553.643  ops/s

ioLocalPropagation=true

[info] Benchmark                (size)   Mode  Cnt     Score     Error  Units
[info] DeepBindBenchmark.delay   10000  thrpt   20  3382.086 ± 279.254  ops/s

Edit: sad, even simply adding the toggle seems to have had an impact. Here are control numbers from series/3.x before the changes in this PR.

[info] Benchmark                (size)   Mode  Cnt     Score     Error  Units
[info] DeepBindBenchmark.delay   10000  thrpt   20  4952.134 ± 353.248  ops/s

NthPortal · 2023-11-15T21:45:40Z

@armanbilge what's the status of this?

armanbilge · 2023-11-15T22:21:47Z

I intend to land this in the upcoming v3.6.0 release (no ETA for this yet, but I'll make a milestone). The benchmarks I posted above for the initial implementation were disappointing, but I still need to benchmark the new implementation that I pushed in 3589db4.

kevin-lee · 2023-11-29T13:39:38Z

@armanbilge That's great. Thank you! I'm sure you can optimize it afterwards, and in the meantime, it would be extremely useful for people who need this feature.

armanbilge · 2023-11-29T16:57:25Z

I'm sure you can optimize it afterwards

Unfortunately it's not so simple. The performance impact I reported before was for all users of Cats Effect, including those who are not using this feature. We want to avoid slowing down the entire runtime for everyone, because that would be a serious regression. So before we can release this, we need to minimize its impact on existing users who do not need this feature.

Still I'm optimistic about the performance of the revised implementation. Will report updated benchmarks, soon ...

henricook · 2024-01-02T20:34:46Z

How'd it go @armanbilge ? If you've had time

djspiewak

I think I've come round to the opinion this is a necessary evil, it's just bikeshedding some of the API and vetting out a few concerns about the implementation.

core/jvm/src/main/scala/cats/effect/unsafe/WorkerThread.scala

core/shared/src/main/scala/cats/effect/IO.scala

core/shared/src/main/scala/cats/effect/IOLocal.scala

core/shared/src/main/scala/cats/effect/unsafe/IOLocals.scala

core/shared/src/main/scala/cats/effect/IOFiber.scala

djspiewak · 2024-01-30T03:08:42Z

core/shared/src/main/scala/cats/effect/IOFiber.scala

+    if (ioLocalPropagation) {
+      IOFiber.setCurrentIOFiber(null)
+    }


We definitely need to do some careful benchmarking here. This particular bit is very sensitive to inliner shenanigans.

We meaning you 😛

Yeah yeah yeah… Give me a pair of shas! :)

core/jvm/src/main/scala/cats/effect/unsafe/WorkerThread.scala

armanbilge · 2024-06-05T00:37:57Z

core/js-native/src/main/scala/cats/effect/IOFiberConstants.scala

@@ -46,6 +46,8 @@ private object IOFiberConstants {
  final val AutoCedeR = 7
  final val DoneR = 8

+  final val ioLocalPropagation = false


Since JS/Native currently lack a mechanism to eliminate branches based on system properties at link- or run-time we just hard-code to false to DCE at compile-time.

armanbilge · 2024-06-05T00:45:48Z

core/jvm/src/main/scala/cats/effect/IOLocalPlatform.scala

+  /**
+   * Returns a [[java.lang.ThreadLocal]] view of this [[IOLocal]] that allows to unsafely get,
+   * set, and remove (aka reset) the value in the currently running fiber. The system property
+   * `cats.effect.ioLocalPropagation` must be `true`, otherwise throws an
+   * [[java.lang.UnsupportedOperationException]].
+   */
+  def unsafeThreadLocal(): ThreadLocal[A] = if (ioLocalPropagation)


This is the new API based on the ThreadLocal interface. Thoughts?

I made it JVM-only. We could definitely support this on JS/Native but I'm not yet convinced that there is a good reason to do so: no Java libs to interop with and performance implications.

If ioLocalPropagation == false it throws an UnsupportedOperationException. Libraries can check IOLocal.isPropagating before deciding to call this method.

Alternatively we could return a no-op ThreadLocal that never sets and always returns the default value.

If we decide not to throw, we could make this a lazy val (or similar) instead. Creating the ThreadLocal "view" isn't observably effectual.

I think this is great, all of it.

djspiewak · 2024-07-04T03:39:56Z

Finished benchmarking. @armanbilge let's see your armchair performance reasoning explain this result. I'm at a loss.

Before

[info] Benchmark                                             (cpuTokens)   (size)   Mode  Cnt      Score     Error    Units
[info] DeepBindBenchmark.async                                       N/A    10000  thrpt   10   2786.655 ±   3.536    ops/s
[info] DeepBindBenchmark.delay                                       N/A    10000  thrpt   10   9613.496 ± 234.106    ops/s
[info] DeepBindBenchmark.pure                                        N/A    10000  thrpt   10  11168.222 ± 465.787    ops/s
[info] MapCallsBenchmark.batch120                                    N/A      N/A  thrpt   10    338.430 ±   3.116    ops/s
[info] MapCallsBenchmark.batch30                                     N/A      N/A  thrpt   10     86.709 ±   0.563    ops/s
[info] MapCallsBenchmark.one                                         N/A      N/A  thrpt   10      2.944 ±   0.017    ops/s
[info] MapStreamBenchmark.batch120                                   N/A      N/A  thrpt   10   5561.435 ±  14.217    ops/s
[info] MapStreamBenchmark.batch30                                    N/A      N/A  thrpt   10   2524.134 ±   2.286    ops/s
[info] MapStreamBenchmark.one                                        N/A      N/A  thrpt   10   3233.048 ±   4.027    ops/s
[info] ParallelBenchmark.parTraverse                               10000     1000  thrpt   10    886.610 ±   0.865    ops/s
[info] ParallelBenchmark.traverse                                  10000     1000  thrpt   10     70.460 ±   0.067    ops/s
[info] ShallowBindBenchmark.async                                    N/A    10000  thrpt   10   2031.729 ±   3.593    ops/s
[info] ShallowBindBenchmark.delay                                    N/A    10000  thrpt   10   9778.303 ±  62.426    ops/s
[info] ShallowBindBenchmark.pure                                     N/A    10000  thrpt   10  11973.770 ±  32.027    ops/s
[info] WorkStealingBenchmark.alloc                                   N/A  1000000  thrpt   10     14.152 ±   0.090  ops/min
[info] WorkStealingBenchmark.manyThreadsSchedulingBenchmark          N/A  1000000  thrpt   10     31.868 ±   5.072  ops/min
[info] WorkStealingBenchmark.runnableScheduling                      N/A  1000000  thrpt   10    870.238 ±   3.475  ops/min
[info] WorkStealingBenchmark.runnableSchedulingScalaGlobal           N/A  1000000  thrpt   10   2245.542 ±   7.645  ops/min
[info] WorkStealingBenchmark.scheduling                              N/A  1000000  thrpt   10     29.631 ±   2.092  ops/min

After

[info] Benchmark                                             (cpuTokens)   (size)   Mode  Cnt      Score     Error    Units
[info] DeepBindBenchmark.async                                       N/A    10000  thrpt   10   2814.793 ±   6.982    ops/s
[info] DeepBindBenchmark.delay                                       N/A    10000  thrpt   10   9589.880 ±  27.294    ops/s
[info] DeepBindBenchmark.pure                                        N/A    10000  thrpt   10  11377.434 ±  37.080    ops/s
[info] MapCallsBenchmark.batch120                                    N/A      N/A  thrpt   10    336.483 ±   2.160    ops/s
[info] MapCallsBenchmark.batch30                                     N/A      N/A  thrpt   10     85.833 ±   0.404    ops/s
[info] MapCallsBenchmark.one                                         N/A      N/A  thrpt   10      2.925 ±   0.007    ops/s
[info] MapStreamBenchmark.batch120                                   N/A      N/A  thrpt   10   5684.510 ±   8.954    ops/s
[info] MapStreamBenchmark.batch30                                    N/A      N/A  thrpt   10   2522.568 ±   7.291    ops/s
[info] MapStreamBenchmark.one                                        N/A      N/A  thrpt   10   3284.312 ±   2.895    ops/s
[info] ParallelBenchmark.parTraverse                               10000     1000  thrpt   10    891.486 ±   0.846    ops/s
[info] ParallelBenchmark.traverse                                  10000     1000  thrpt   10     70.303 ±   0.060    ops/s
[info] ShallowBindBenchmark.async                                    N/A    10000  thrpt   10   1985.424 ±   2.771    ops/s
[info] ShallowBindBenchmark.delay                                    N/A    10000  thrpt   10   9655.691 ± 182.608    ops/s
[info] ShallowBindBenchmark.pure                                     N/A    10000  thrpt   10  10075.531 ±  22.884    ops/s
[info] WorkStealingBenchmark.alloc                                   N/A  1000000  thrpt   10     14.044 ±   0.078  ops/min
[info] WorkStealingBenchmark.manyThreadsSchedulingBenchmark          N/A  1000000  thrpt   10     48.611 ±   1.642  ops/min
[info] WorkStealingBenchmark.runnableScheduling                      N/A  1000000  thrpt   10   2987.595 ±   5.299  ops/min
[info] WorkStealingBenchmark.runnableSchedulingScalaGlobal           N/A  1000000  thrpt   10   2249.480 ±  48.568  ops/min
[info] WorkStealingBenchmark.scheduling                              N/A  1000000  thrpt   10     52.240 ±   3.133  ops/min

armanbilge marked this pull request as draft May 16, 2023 19:35

POC thread-local iolocals

0b88c01

armanbilge force-pushed the topic/thread-local-iolocal branch from dc9b74c to 0b88c01 Compare May 16, 2023 19:35

armanbilge commented May 16, 2023

View reviewed changes

armanbilge mentioned this pull request May 16, 2023

Unable to start span as current typelevel/otel4s#202

Open

Simplify and optimize

db743e2

armanbilge commented May 16, 2023

View reviewed changes

core/shared/src/main/scala/cats/effect/IOFiber.scala Outdated Show resolved Hide resolved

core/shared/src/main/scala/cats/effect/unsafe/IOLocals.scala Outdated Show resolved Hide resolved

Special-case for WorkerThread

716ef32

armanbilge commented May 17, 2023

View reviewed changes

core/shared/src/main/scala/cats/effect/unsafe/IOLocals.scala Outdated Show resolved Hide resolved

Load locals in unsafeRunFiber

0a69caf

armanbilge commented May 17, 2023

View reviewed changes

rossabaker reviewed May 18, 2023

View reviewed changes

rossabaker mentioned this pull request May 18, 2023

Experimental IOLocalContextStorage typelevel/otel4s#214

Draft

rossabaker added a commit to typelevel/otel4s that referenced this pull request May 18, 2023

Try typelevel/cats-effect#3636

682abb6

armanbilge added 4 commits May 18, 2023 06:12

Dump locals in more places

2775064

Refactor IOLocal

270764f

Use new IOLocal APIs in IOLocals

d55489d

Mark IOLocal methods as final

2cf72a5

armanbilge commented May 21, 2023

View reviewed changes

Add IOLocalsSpec

cb3859d

armanbilge changed the base branch from series/3.5.x to series/3.x June 27, 2023 00:28

armanbilge marked this pull request as ready for review June 27, 2023 00:29

armanbilge changed the title ~~Proof-of-concept thread-local IOLocals~~ thread-local IOLocals Jun 27, 2023

armanbilge added 2 commits June 28, 2023 18:15

Rename property to ioLocalPropagation and fixes

7dce01c

Bump base version

5e171ac

armanbilge added 2 commits September 30, 2023 07:41

Use real instead of ticked

7517755

Formatting

8d8e004

armanbilge added 2 commits September 30, 2023 17:54

Try keeping the current fiber as a thread-local instead

3589db4

Revert spurious whitespace changes

522677e

armanbilge added this to the v3.6.0 milestone Nov 15, 2023

nima-taheri-hs mentioned this pull request Nov 17, 2023

Is it safe to use MDC with cats-effect? outr/scribe#506

Closed

djspiewak requested changes Jan 30, 2024

View reviewed changes

gaeljw mentioned this pull request Feb 19, 2024

Support context propagation in Cats Effect library (Scala) open-telemetry/opentelemetry-java-instrumentation#10599

Open

armanbilge added 4 commits May 27, 2024 14:42

Merge branch 'series/3.x' into topic/thread-local-iolocal

6cc4d38

Update headers

ac88480

Update platform headers

49e5c30

Remove unused class

925f504

djspiewak mentioned this pull request May 29, 2024

Consolidate duplicate "currently executing fiber" pointers #4083

Open

armanbilge added 2 commits June 4, 2024 20:30

Expose IOLocal propagation as a ThreadLocal

d63a6ff

unsafeToThreadLocal() throws if propagation disabled

d4549fb

armanbilge marked this pull request as draft June 4, 2024 20:56

armanbilge added 3 commits June 5, 2024 00:12

Add scaladoc

2502045

Factor out to JVM-only

535fc8a

Bikeshed API and docs

d854799

armanbilge marked this pull request as ready for review June 5, 2024 00:34

armanbilge added 2 commits June 5, 2024 00:38

Formatting

f070552

Delete dead code

2cf1d8a

armanbilge commented Jun 5, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`IOLocal` propagation for unsafe access #3636

`IOLocal` propagation for unsafe access #3636

armanbilge commented May 16, 2023

armanbilge May 16, 2023

rossabaker May 18, 2023

armanbilge May 18, 2023

kevin-lee Jun 28, 2023

armanbilge Jun 28, 2023

armanbilge May 16, 2023

armanbilge May 16, 2023

armanbilge May 17, 2023

rossabaker left a comment

rossabaker May 18, 2023

armanbilge May 21, 2023 •

edited

Loading

kevin-lee commented Jun 23, 2023

armanbilge commented Jun 23, 2023

kevin-lee commented Jun 28, 2023

armanbilge commented Jun 28, 2023 •

edited

Loading

kevin-lee commented Jun 28, 2023

armanbilge commented Sep 30, 2023 •

edited

Loading

NthPortal commented Nov 15, 2023

armanbilge commented Nov 15, 2023

kevin-lee commented Nov 29, 2023

armanbilge commented Nov 29, 2023

henricook commented Jan 2, 2024

djspiewak left a comment

djspiewak Jan 30, 2024

armanbilge Jun 5, 2024

djspiewak Jun 7, 2024

armanbilge Jun 5, 2024

armanbilge Jun 5, 2024 •

edited

Loading

djspiewak Jun 7, 2024

djspiewak commented Jul 4, 2024

		// TODO handle defaults and lenses. all do-able, just needs refactoring ...
		final class IOLocals private[effect] (private[this] var state: IOLocalState) {

IOLocal propagation for unsafe access #3636

Are you sure you want to change the base?

IOLocal propagation for unsafe access #3636

Conversation

armanbilge commented May 16, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rossabaker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

armanbilge May 21, 2023 • edited Loading

Choose a reason for hiding this comment

kevin-lee commented Jun 23, 2023

armanbilge commented Jun 23, 2023

kevin-lee commented Jun 28, 2023

armanbilge commented Jun 28, 2023 • edited Loading

kevin-lee commented Jun 28, 2023

armanbilge commented Sep 30, 2023 • edited Loading

NthPortal commented Nov 15, 2023

armanbilge commented Nov 15, 2023

kevin-lee commented Nov 29, 2023

armanbilge commented Nov 29, 2023

henricook commented Jan 2, 2024

djspiewak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

armanbilge Jun 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djspiewak commented Jul 4, 2024

Before

After

`IOLocal` propagation for unsafe access #3636

`IOLocal` propagation for unsafe access #3636

armanbilge May 21, 2023 •

edited

Loading

armanbilge commented Jun 28, 2023 •

edited

Loading

armanbilge commented Sep 30, 2023 •

edited

Loading

armanbilge Jun 5, 2024 •

edited

Loading