Can I share a lock between instances of Zio on different threads?

[psznm]

I am currently having some issues that I don't fully understand yet but I think it is because I am trying to share one lock (RwLock) between zio instances where each zio instance is on its own thread.

So the question is, can I even actually do that? Can I share a lock between instances of Zio on different threads? If yes, are there any rules to it? If no, what are the the alternatives?

[lalinsky]

There is nothing fundamental in the code that would make it incompatible with cross-thread communication.

All these sync primitives boil down to some internal logic and then either parking the current task/thread, or resuming the parked task/thread. All of those operations are safe across runtimes, or even foreign threads.

But there could he a specific bug in RwLock. I'll be honest I'm not super happy with the current RwLock in zio, it does thundering herd issues, which are technically okay, just inefficient.

[psznm]

Alright, thanks. I am using it to wait until I can free up a resource. Basically locking write only at program stop and rarely on configuration reload. I have been doing some larger refactors of my prototype and started having issues with write lock never being able to lock and hanging, even though all read locks should have been released. Also the circumstances of the run have changed, so I am not sure if it has been caused by refactoring or if there has been an issue that I haven't noticed earlier. But maybe some read lock isn't released after all. Will have some more debugging to do tomorrow.

[psznm]

So I have been debugging some more and running with thread sanitizer. The original issue was really some of my read locks "disappearing", so there was no issue with the RwLock itself and it seems to have been some issue with cancellation/deinitialization. I ahd to take a step back, and now I am running into data race in Mutex?

My code is

    {
        const lock_io = self.io_mgmt;
        self.mtx.lockUncancelable(lock_io); // Proxy.zig:235
        defer self.mtx.unlock(lock_io); // Proxy.zig:236
        try self.acceptors.append(gpa, accept);
    }

And the report is:

WARNING: ThreadSanitizer: data race (pid=99000)
  Atomic write of size 4 at 0x7ffff0667428 by thread T6:
    #0 atomic.Value(u32).fetchAdd /opt/zig-x86_64-linux-0.16.0/lib/std/atomic.zig:53 (driver+0x803126)
    #1 common.Waiter.signal /home/kmarek/git/cont-cont/release/proxy-experiments/zig-pkg/zio-0.16.0-xHbVVJQxIgAMdBlOCF65siVKJRVww9Z_ayfJanpk7VEK/src/common.zig:108 (driver+0x803126)
    #2 sync.Futex.wake /home/kmarek/git/cont-cont/release/proxy-experiments/zig-pkg/zio-0.16.0-xHbVVJQxIgAMdBlOCF65siVKJRVww9Z_ayfJanpk7VEK/src/sync/Futex.zig:244 (driver+0x802f22)
    #3 io.futexWakeImpl /home/kmarek/git/cont-cont/release/proxy-experiments/zig-pkg/zio-0.16.0-xHbVVJQxIgAMdBlOCF65siVKJRVww9Z_ayfJanpk7VEK/src/io.zig:398 (driver+0x979cce)
    #4 Io.futexWake__anon_11357 /opt/zig-x86_64-linux-0.16.0/lib/std/Io.zig:1579 (driver+0x67a400)
    #5 Io.Mutex.unlock /opt/zig-x86_64-linux-0.16.0/lib/std/Io.zig:1647 (driver+0x67a077)
    #6 app.Proxy.acceptor /home/kmarek/git/cont-cont/release/proxy-experiments/src/app/Proxy.zig:236 (driver+0x7c9743)

  Previous read of size 8 at 0x7ffff0667428 by thread T4:
    #0 __tsan_memcpy /opt/zig-x86_64-linux-0.16.0/lib/libtsan/tsan_interceptors_memintrinsics.cpp:27 (driver+0xf1ce8b)
    #1 common.Waiter.wait__anon_57057 /home/kmarek/git/cont-cont/release/proxy-experiments/zig-pkg/zio-0.16.0-xHbVVJQxIgAMdBlOCF65siVKJRVww9Z_ayfJanpk7VEK/src/common.zig:145 (driver+0x850987)
    #2 sync.Futex.wait /home/kmarek/git/cont-cont/release/proxy-experiments/zig-pkg/zio-0.16.0-xHbVVJQxIgAMdBlOCF65siVKJRVww9Z_ayfJanpk7VEK/src/sync/Futex.zig:102 (driver+0x973646)
    #3 io.futexWaitUncancelableImpl /home/kmarek/git/cont-cont/release/proxy-experiments/zig-pkg/zio-0.16.0-xHbVVJQxIgAMdBlOCF65siVKJRVww9Z_ayfJanpk7VEK/src/io.zig:394 (driver+0x979d2e)
    #4 Io.futexWaitUncancelable__anon_12971 /opt/zig-x86_64-linux-0.16.0/lib/std/Io.zig:1573 (driver+0x68557c)
    #5 Io.Mutex.lockUncancelable /opt/zig-x86_64-linux-0.16.0/lib/std/Io.zig:1637 (driver+0x685265)
    #6 app.Proxy.acceptor /home/kmarek/git/cont-cont/release/proxy-experiments/src/app/Proxy.zig:235 (driver+0x7c9600)

This one seems to be out of my control, or can I do something about that...? Note that I am always passing in zio.Runtime from main thread, but I also tried this with zio.Runtime from thread calling this. I think it doesn't really matter which instance gets passed in since zio discards the userdata anyway? Also tried regular .lock, instead of the uncancelable one.

EDIT: That is a false positive, due to futexes lock free code, right? So I should just suppress it.

[lalinsky]

It's false positive because this statement, const d = &self.mode.direct; actually makes a full copy of self.mode to check that the .direct tag is active. And TSan considers the copy as racing with the fetchAdd. Even non-tagged union actually does that. Only extern union is safe. I'm considering switching to multiple optionals to avoid the false positive. I'd possibly even consider it a bug in the compiler, but it is what it is.

https://ziggit.dev/t/full-copy-of-tagged-union-just-to-check-tag-in-debug-mode/16640