FiberRef (#618)

[hanny24]

This is my initial take on #618. It's still very much in progress, contains no user documentation, the code itself is not in the best possible shape...nevertheless I would like you to review my approach. I'm not really sure if I have not overseen something as I'm not very familiar with ZIO code base.

What do you think about it? Do you see it as a way how #618 can be implemented? If we agree that this is the way, I'm more that happy to implement this for real.

[jdegoes]

@hanny24 Thank you for taking a stab at this!

This is quite complex topic, made more complex by the fact that #618 is not fully spec'd out.

I do have some thoughts but won't have time to post them for a bit. Hang tight!

[hanny24]

Is there anything I can do in order to help with this? Maybe prepare some sort of spec?

[jdegoes]

@hanny24 Working on a spec / unit test suite for FiberLocal would be phenomenal.

We want to ensure that:

1. When we fork a fiber, the child gets the parent locals.
2. When the child modifies a local, it only modifies it in the child, not the parent.
3. When a child forks another (grand)child, then the grandchild gets the value from the child, not from the parent of the child (grandfather).
4. Etc. with obvious constraints on parallelism (two children cannot modify each other's state).

Right now these tests for FiberLocal will fail, but they can inform implementation of the work.

[jdegoes]

I can share more ideas over the weekend.

[jdegoes]

My main idea for an API now is:

object ZIO {
  def inheritLocals(that: Fiber[_, _]): UIO[Unit]
}

This would be called automatically in fork so that every child inherits from its parent.

Locals would be stored per fiber in a weak map: WeakMap[FiberLocal[_], FiberId]. Reading fiber locals just reads them; but writing checks the FiberId and if it doesn't belong to the fiber doing the writing, it removes it from the weak map and adds a new entry.

This involves internalizing FiberLocal into the runtime system. We can probably improve the API as a result, making an initial value mandatory, for example, so that getting a fiber local always succeeds.

[jdegoes]

@hanny24 ^^^

[hanny24]

Seems pretty straight-forward to me, however I have two notes:

• We will effectively make FiberLocal inheritance built-in feature that can't be turned off. I'm perfectly fine with this approach as it is very reasonable default behavior.
• What I had in mind originally was some sort of copy-on-fork behavior. Which means that forked fiber sees the value from FiberLocal from the point in time when forked happened. If I understand it correctly, your proposal is really copy-on-write, meaning that child fiber sees all changes made by its ancestors (as long as it does not write some value itself). My point is that is that this behavior might lead be surprising to someone.

This involves internalizing FiberLocal into the runtime system. We can probably improve the API as a result, making an initial value mandatory, for example, so that getting a fiber local always succeeds.

I'm not sure just yet how this is related.

Nevertheless, I'll try to implement your proposal over the weekend.

[hanny24]

Hmm, after some deeper thought it's not that clear anymore. I'm not really sure about ZIO.inherticLocals semantics. Consider following example:

for {
      local <- FiberLocal.make[Int]
      f1    <- local.set(10).fork
      f2    <- local.set(20).fork
      f     = f1.zip(f2)
      _     <- ZIO.inheritLocals(f)
      v     <- local.get
    } yield v must_=== ???

What should v equal to? Any thoughts on this?

cc @jdegoes

[jdegoes]

@hanny24 Let me take a closer look at this and get back... apologies for the delay.

[hanny24]

@jdegoes ping: Any thoughts on this?

[jdegoes]

@hanny24 I'm back and read to help push this through to completion!

So I think we need the following ingredients:

1. A FiberLocalImpl inside internal that consists solely of a wrapper class around a value. Something like:

   private[internal] class FiberLocalImpl[A] private (@volatile var value: A, val id: Long) extends FiberLocal[A] { ... }

   The FiberLocal[A] inside scalaz.zio will become a pure interface (trait).
2. A new counter (AtomicLong) for fiber local variables, which can be stored in FiberContext and will be responsible for giving every new fiber local a fresh identity.
3. A new weak map inside FiberContext that holds local variables (weak maps can already be created inside FiberContext via Platform). It will have type Map[FiberLocalImpl[_], FiberId]. That is, it will be a map from fiber local to fiber id. If there are no references to the fiber local variable, then it will be garbage collected and removed from the map—this is key, because they will be shared on fork boundaries.
4. On every fork, we pass copy over the fiber locals into a new weak map into the child fiber. This means the constructor of FiberContext will need to take the weak map of fiber locals, and the private fork function inside FiberContext will need to copy over the fiber locals into the new weak map that is passed to the constructor of the new child FiberContext.
5. We add two new operations into the ZIO set of operations. One of them is: NewFiberLocal, which returns a FiberLocal of the specified type. Importantly, and unlike the previous fiber local implementation, it will take an initial value, meaning that you cannot create a fiber local with an undefined state. The other one is UpdateFiberLocal, which takes a FiberLocal and a function A => A to perform the update.
6. In the FiberContext main loop (evaluateNow), you have to add interpretation for the two new operations. The NewFiberLocal interpretation will create a new fiber local, initialized with the specified value and new fresh identity, and add the new local to the fiberLocals weak map, using the fiber id of the FiberContext. The UpdateFiberLocal interpretation will update the fiber local value, but before it does this, it will check to see if the fiber id in its fiber locals weak map is the same as the fiber id of the executing context: if it is the same, then it will update it directly; if it is not the same, then it will remove the old fiber local from the map, and create a new fiber local, with a new value, and the same fiber local identity (so the FiberLocalImpl will change, and get the new value, but keep the same id as the old FiberLocalImpl), and add that new fiber local into its own map. This will achieve copy-on-write semantics.
7. Finally, when one fiber joins another fiber, we have to replace all fiber locals of the joiner with the values of the joinee, using the identities of the fiber local. This implies we will need to add a new operation, something like InheritFiberLocals, which will take a fiber, and copy its fiber locals into its own, wherever they share the same id. This operation may be independently useful, and could be exposed in a new method added to the ZIO object, e.g. def inheritLocals(fiber: Fiber[_, _]): UIO[Unit]. This method can be used inside the implementation of Fiber.join. Thus we will achieve the following identity: forking an effect, which then modifies values, and then joining the fiber, will be equivalent to modifying the fiber locals on the original fiber. This is a useful algebraic law.

Please let me know if anything is unclear or if this doesn't come together as I hope. 😄

[hanny24]

Hi,
points 1 - 6 are pretty clear and I think I have a good idea how to implement it. However I'm still struggling with number 7. I'm not really sure how should implementation of inheritLocals look like for Fibers that are not FiberContext. By that I mean fibers created by map, orElse or zip combinators which don't have any (direct) access to their fiber locals. Should those somehow include reference to their original FiberContext from which they were derived from? From my point of view, this is currently the only way how this could be implemented, however it seems quite hacky.

Does my comment make any sense to you?

[jdegoes]

@hanny24 You are right. We should put the method on Fiber, not add a new standalone op. Then it will be perfectly clean and well-defined.

[jdegoes]

@hanny24 You are right. We should put the method on Fiber, not add a new op. Then it will be perfectly clean and well-defined.

[hanny24]

Maybe I have one more question related to inheritLocals. Assuming that inheritLocal is called on join, suppose following code:

for {
  local <- FiberLocal.create(42)
  fiber1 <- doSomething(local).fork
  fiber2 <- doSomethingElse(local).fork
  fiber = fiber1.orElse(fiber2)
  _ <- fiber.join
  result <- local.get
} yield result

Assuming that both doSomething and doSomethingElse modify local, what should be the value of result?

The best thing I can come up with is to accept local changes from both fiber1 and fiber2, possibly taking fiber1 taking precedence over fiber2. How does that sound to you?

[hanny24]

Oh, I just had another idea: we could make inheritLocals blocking in a sense that it completes only when fiber is completed. That would actually provide nice semantic for orElse Fiber as we could choose to inherit from the fiber that actually succeeded.

[jdegoes]

The best thing I can come up with is to accept local changes from both fiber1 and fiber2, possibly taking fiber1 taking precedence over fiber2. How does that sound to you?

fiber1 orElse fiber2 must allow fiber1 to take precedence over fiber2, because orElse is consistently left-biased in ZIO. In order to achieve the left-bias, you then would need to do: def inheritLocals = fiber2.inheritLocals *> fiber1.inheritLocals. This is the correct thing to do, actually the only correct thing to do.

inheritLocals should definitely not be blocking. It's a "snapshot in time" semantic.

[hanny24]

6. In the FiberContext main loop (evaluateNow), you have to add interpretation for the two new operations. The NewFiberLocal interpretation will create a new fiber local, initialized with the specified value and new fresh identity, and add the new local to the fiberLocals weak map, using the fiber id of the FiberContext. The UpdateFiberLocal interpretation will update the fiber local value, but before it does this, it will check to see if the fiber id in its fiber locals weak map is the same as the fiber id of the executing context: if it is the same, then it will update it directly; if it is not the same, then it will remove the old fiber local from the map, and create a new fiber local, with a new value, and the same fiber local identity (so the FiberLocalImpl will change, and get the new value, but keep the same id as the old FiberLocalImpl), and add that new fiber local into its own map. This will achieve copy-on-write semantics.

On a second thought this doesn't seems as really clear. How can we update FiberLocalImpl? If we were to create new instance of FiberLocalImpl (and use it as a key in weak map) on a write by a forked fiber, how do we make sure that:
a) it is not garbage collected?
b) how do we look it up later? We could probably solve it using equals/hashCode override that uses fiber local identity

What would make sense to me is to define FiberLocal as class FiberLocal[A](id: Long) and define weak map in FiberContext as Map[FiberLocal, (AnyRef,FiberId)].

[jdegoes]

On a second thought this doesn't seems as really clear. How can we update FiberLocalImpl?

FiberLocalImpl has a volatile variable value inside it, which can be updated mutably.

Do you see any problems with that approach?

[hanny24]

OK, I'll try to describe the issue I see better:) I see a problem with this:

if it is not the same, then it will remove the old fiber local from the map, and create a new fiber local, with a new value, and the same fiber local identity

I can create a new FiberLocalmpl, I can even store it in FiberContext's weak map. Now when I want to read the value of FiberLocal I only have a reference to the original FiberLocal. This is still doable (although not really nice) since we can somehow use FiberLocal identifier to lookup the value in the weak map. However the issue I see is that the weak map might not contain the FiberLocalmpl I created on the write as we did not store the reference to it anywhere so it might be garbage collected a thus removed from the map.

Is there anything I missed in your proposal? I don't see how newly created FiberLocalmpl is not garbage collected.

[jdegoes]

I created on the write as we did not store the reference to it anywhere so it might be garbage collected a thus removed from the map.

You are right! I think your proposal will work:

What would make sense to me is to define FiberLocal as class FiberLocal[A](id: Long) and define weak map in FiberContext as Map[FiberLocal, (AnyRef,FiberId)].

This way there is a single identity for the fiber local, even as it crosses fork boundaries, and it will be garbage collected only in the event it is not reachable from anywhere.

[jdegoes]

@hanny24 Let me know if you need any more help. I think we're getting close to something really nice!

Relatedly, I think we should "modernize" the FiberLocal interface, so that it resembles Ref and TRef. In fact, we should consider changing the name to FiberRef, for consistency. It's not widely used yet but I do think it will become very widely used post 1.0...

[hanny24]

FiberRef sounds good to me. I'll try to implement this over a few next days.

[hanny24]

I tried to implement this. However it turned out little bit different. I'll try to describes the changes I have done below.

The main issue is that some fiber can access FiberRef that was created by unrelated fiber (such as sibling). In that case there is no record in FiberContext's weak map. In order to provide some meaningful value, I used definition class FiberRef[A](initial: A) and I use that value. That's probably the biggest difference from your comment #665 (comment)

Also, in current implementation child fiber does not see any updates done by a parent. In order to implement this we can either:

• define weak map as util.Map[FiberRef[_], (AtomicReference[Any], FiberId)].
• make FiberRef a trait and provide two strategies: on that is simple where child doesn't see any updates; second strategy that is internally using Ref and does support this feature.

I kind of like the second approach as you only pay for what you need.

Any thoughts on this?

[jdegoes]

now it seems that FiberId is not needed at all in util.Map[FiberRef[_], (Any, FiberId)]

Great! I will inspect the logic in one last review when you are ready.

I would suggest to rename inheritLocals to inheritFiberRefs to keep the naming somehow consistent.

Great catch!

[jdegoes]

Love all these new methods, and by making them consistent, we're helping to make sure users don't have to re-learn a similar set of primitives for each new type of references (Ref, RefM, TRef, and FiberRef).

[jdegoes]

A few minor comments and then looks good to merge! 🎉

[hanny24]

I consider this ready to be merged. I implemented all your comments and I also added fiberref.md with some basic documentation. My English grammar is far from being flawless so feel free to make any necessary changes.

[jdegoes]

This is great work! I saw that we are not using FiberId in the Map, and I suggest we delete that. Otherwise, looks ready to merge! 🎉

[jdegoes]

Awesome and impressive work! FiberRef is now ready to take on ThreadLocal. 😄

Thank you for this contribution! 🙏

[hanny24]

I'm glad to help!