core/events: refactor the events.Target to use mutexes instead of a background goroutine

[calebdoxsey]

Summary

Currently the event target pushes an operation onto a channel for each of its methods. This is then handled by a goroutine. Since each of the operations goes through a separate channel, and the channels were defined with a size of 1, it was possible for call order to behave unpredictably. For example:

AddListener(li)
Dispatch(ctx, evt)

Could end up with the Dispatch occurring before AddListener.

By making these channels synchronous we can avoid this behavior. Now AddListener will block until the goroutine picks it up, and Dispatch will always happen after AddListener.

Refactor the events.Target to use mutexes instead of a background goroutine.

[coveralls]

coverage: 64.134% (+0.02%) from 64.112% when pulling 0a769f7 on cdoxsey/events-ordering into 0b79a28 on main.

[kenjenkins]

I'm not sure this is sufficient to order the execution of the AddListener() / Dispatch() operations.

Consider the following test case:

func TestDispatchOrder(t *testing.T) {
    var target events.Target[int64]
    t.Cleanup(target.Close)

    ch := make(chan error)
    
    var next atomic.Int64
    
    update := func(_ context.Context, i int64) {
        if n := next.Load(); i != n {
            ch <- fmt.Errorf("want %d, got %d", n, i)
        }
        next.Store(i + 1)
    }
    
    go func() {
        for i := int64(0); i < 1000; i++ {
            h := target.AddListener(update)
            target.Dispatch(context.Background(), i)
            target.RemoveListener(h)
        }   
    }()
    
    if err := <-ch; err != nil {
        t.Fatal(err)
    }   
}

Should it pass after this PR?

It seems to fail reliably on my machine at some point before all 1000 iterations.

(Let me know if I've made some mistake in the test case itself.)

[edit: indeed, this test is buggy in a few ways — most importantly, even if the dispatches all occur in order, the different listener goroutines will be racing to process the events, so there's no guarantee the update function will be called with each event in the order they were dispatched]

[calebdoxsey]

I will refactor this to use mutexes.