Release v16.0.2

Summary

An event store's outbox could only ever be delivered to one subscribing event store. When two or more event stores subscribe to the same source store's outbox, only the first-subscribed target received events — the others were silently starved, with no error.

Root cause

The cross-store outbox observer is resolved in EventStoreSubscriptionsManager.GetObserver:

new ObserverKey(definition.Identifier, definition.SourceEventStore, namespaceName, EventSequenceId.Outbox)

The subscription Identifier is set to the source store name (EventStoreSubscriptionsReactor: subscriptionId = eventContext.EventSourceId.Value), so the observer key is effectively (source, source, namespace, Outbox) — it does not vary by target. The subscription managers are per-target grains (eventstoresubscriptionsmanager/<target>), but they all resolve the same observer grain for a given source.

Consequently, when a second target subscribes to a source another target already subscribes to, RefreshSubscription finds observer.IsSubscribed() == true and short-circuits:

var subscribed = await observer.IsSubscribed();
if (subscribed) { /* already active — return, do not subscribe */ return; }

The shared observer keeps the first target's context.Metadata, so EventStoreSubscriptionObserverSubscriber.OnNext only ever appends to the first target's inbox. The second target's inbox-<source> is never populated.

Fix

Include the target event store in the ObserverId so each (source, target) pair gets its own outbox observer and independent delivery:

var observerId = new ObserverId($"{definition.Identifier.Value}->{_targetEventStoreName.Value}");
return GrainFactory.GetGrain<IObserver>(new ObserverKey(observerId, definition.SourceEventStore, namespaceName, EventSequenceId.Outbox));

Verified

Reproduced and fixed against a real 3-service topology where a Lobby store's outbox fans out to both a Studio and a StudioAdmin store:

• Before: the source store had a single shared Lobby outbox observer; only Studio received events, StudioAdmin's inbox-Lobby stayed empty across restarts and repeated appends.
• After: two observers exist — Lobby->Studio and Lobby->StudioAdmin — and both targets receive the outbox.

Suggested regression test

Extend for_EventStoreSubscriptionsManager (Orleans TestKit): create two managers with different target keys subscribing to the same source, and assert each resolves a distinct IObserver grain (observer id contains the target) and each receives a Subscribe call — rather than the second being skipped by the shared-observer IsSubscribed() guard.

Fixed

• Cross-store outbox subscriptions now deliver to every subscribing event store. Previously, when multiple event stores subscribed to the same source store's outbox, only the first to subscribe received events.