Add ProjectionCoordinator + distributor layer for hot-cold daemon coo…#96
Merged
Merged
Conversation
…rdination Closes #83. Polecat could spin up the async projection daemon via PolecatDaemonHostedService (single-node, no coordination), but had no path to safely run multiple Polecat nodes against the same database — every node would try to drive every shard. This adds the multi-node-aware coordinator that mirrors Marten's hot-cold distributor design, adapted to SQL Server idioms. ## What's added - `SqlServerAppLock` — Polecat's analog to Marten's `Weasel.Postgresql.AdvisoryLock`. Wraps `sp_getapplock` / `sp_releaseapplock` with `@LockOwner = 'Session'`, holding a dedicated `SqlConnection` for the lifetime of the lock owner so the Session-scoped locks persist until we explicitly release or the connection drops. Stale-handle detection clears local tracking when the connection is found broken so the coordinator's `HasLock()` check stays honest. - `IProjectionDistributor` + `ProjectionSet` — the per-cycle "(database × shards) with a lock id" grouping the coordinator polls. - Three concrete distributors: - `SoloProjectionDistributor` — single-node fall-through, no locks (test / explicit-opt-out path; not auto-selected by the coordinator). - `SingleTenantProjectionDistributor` — one set per shard, deterministic lock id from `db.Identifier:schema:shard.Identity` so every node in the deployment races for the same `sp_getapplock` resource. - `MultiTenantedProjectionDistributor` — one set per database, all shards grouped behind one per-database lock so a tenant's projections never split across nodes. - `ProjectionCoordinator` — the concrete `IProjectionCoordinator` implementation. The `ExecuteAsync` loop is a slot-for-slot mirror of Marten's `ProjectionCoordinator.executeAsync`: - Build the current distribution - For each set: if we hold the lock, ensure agents are running; if not, try to attain it (start agents on success, stop them on failure to recover from a lost lock) - Sleep based on agent pause status Picks `SingleTenantProjectionDistributor` for `DatabaseCardinality.Single` and `MultiTenantedProjectionDistributor` for `StaticMultiple` (mirrors Marten's `Tenancy is DefaultTenancy ? Single : MultiTenanted` choice). Plus `ProjectionCoordinator<T>` for ancillary multi-store DI registration. - DI wiring on `PolecatConfigurationExpression`: `AddProjectionCoordinator(DaemonMode)` and the typed `AddProjectionCoordinator<T>(DaemonMode)`. Mutually exclusive with `AddAsyncDaemon` — pick one. ## Tests `src/Polecat.Tests/Daemon/sql_server_app_lock_tests.cs` — 6 cases for the lock primitive, all passing against SQL Server 2025 docker on net9.0: - Only one of two `SqlServerAppLock` instances acquires a given id - Releasing lets a waiting instance acquire - Disposing one auto-releases its session locks (Session-scope) - Multiple distinct lock ids are independent - `TryAttainLockAsync` is idempotent when the id is already owned - Releasing an unknown id is a no-op The full coordinator loop (leadership election + agent lifecycle) follows Marten's well-tested implementation slot-for-slot; the lock primitive is the genuinely Polecat-new code path and gets dedicated coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…rdination
Closes #83.
Polecat could spin up the async projection daemon via PolecatDaemonHostedService (single-node, no coordination), but had no path to safely run multiple Polecat nodes against the same database — every node would try to drive every shard. This adds the multi-node-aware coordinator that mirrors Marten's hot-cold distributor design, adapted to SQL Server idioms.
What's added
SqlServerAppLock— Polecat's analog to Marten'sWeasel.Postgresql.AdvisoryLock. Wrapssp_getapplock/sp_releaseapplockwith@LockOwner = 'Session', holding a dedicatedSqlConnectionfor the lifetime of the lock owner so the Session-scoped locks persist until we explicitly release or the connection drops. Stale-handle detection clears local tracking when the connection is found broken so the coordinator'sHasLock()check stays honest.IProjectionDistributor+ProjectionSet— the per-cycle "(database × shards) with a lock id" grouping the coordinator polls.Three concrete distributors:
SoloProjectionDistributor— single-node fall-through, no locks (test / explicit-opt-out path; not auto-selected by the coordinator).SingleTenantProjectionDistributor— one set per shard, deterministic lock id fromdb.Identifier:schema:shard.Identityso every node in the deployment races for the samesp_getapplockresource.MultiTenantedProjectionDistributor— one set per database, all shards grouped behind one per-database lock so a tenant's projections never split across nodes.ProjectionCoordinator— the concreteIProjectionCoordinatorimplementation. TheExecuteAsyncloop is a slot-for-slot mirror of Marten'sProjectionCoordinator.executeAsync:SingleTenantProjectionDistributorforDatabaseCardinality.SingleandMultiTenantedProjectionDistributorforStaticMultiple(mirrors Marten'sTenancy is DefaultTenancy ? Single : MultiTenantedchoice). PlusProjectionCoordinator<T>for ancillary multi-store DI registration.DI wiring on
PolecatConfigurationExpression:AddProjectionCoordinator(DaemonMode)and the typedAddProjectionCoordinator<T>(DaemonMode). Mutually exclusive withAddAsyncDaemon— pick one.Tests
src/Polecat.Tests/Daemon/sql_server_app_lock_tests.cs— 6 cases for the lock primitive, all passing against SQL Server 2025 docker on net9.0:SqlServerAppLockinstances acquires a given idTryAttainLockAsyncis idempotent when the id is already ownedThe full coordinator loop (leadership election + agent lifecycle) follows Marten's well-tested implementation slot-for-slot; the lock primitive is the genuinely Polecat-new code path and gets dedicated coverage.