Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
# Alberto.* package makes the restore fail.
- name: Pack smoke
run: |
for proj in Alberto.Dcb Alberto.Dcb.Admin Alberto.Dcb.Commands Alberto.Dcb.EntityFramework Alberto.Dcb.InMemory Alberto.Dcb.Messaging Alberto.Dcb.Postgres Alberto.Dcb.Postgres.Messaging Alberto.Dcb.Telemetry Alberto.Dcb.Testing Alberto.Dcb.Testing.Xunit; do
for proj in Alberto.Dcb Alberto.Dcb.Commands Alberto.Dcb.EntityFramework Alberto.Dcb.InMemory Alberto.Dcb.Messaging Alberto.Dcb.Postgres Alberto.Dcb.Postgres.Messaging Alberto.Dcb.Telemetry Alberto.Dcb.Testing Alberto.Dcb.Testing.Xunit; do
dotnet pack "src/$proj/$proj.csproj" -c Release --version-suffix ci -o /tmp/alberto-pack
done
SMOKEDIR=$(mktemp -d)
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/publish-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,11 @@ jobs:
- name: Build libraries
run: |
dotnet build src/Alberto.Dcb/Alberto.Dcb.csproj -c Release
# Built but deliberately not packed: the admin surface is IsPackable=false until the
# front doors on feature/admin-surface ship. Building it here still catches a Release
# compile break, which the pack loop below no longer would.
dotnet build src/Alberto.Dcb.Admin/Alberto.Dcb.Admin.csproj -c Release
dotnet build src/Alberto.Dcb.Postgres.Admin/Alberto.Dcb.Postgres.Admin.csproj -c Release
dotnet build src/Alberto.Dcb.Commands/Alberto.Dcb.Commands.csproj -c Release
dotnet build src/Alberto.Dcb.EntityFramework/Alberto.Dcb.EntityFramework.csproj -c Release
dotnet build src/Alberto.Dcb.InMemory/Alberto.Dcb.InMemory.csproj -c Release
Expand All @@ -67,7 +71,7 @@ jobs:
- name: Pack
run: |
VERSION_SUFFIX="${{ steps.version.outputs.suffix }}"
for proj in Alberto.Dcb Alberto.Dcb.Admin Alberto.Dcb.Commands Alberto.Dcb.EntityFramework Alberto.Dcb.InMemory Alberto.Dcb.Messaging Alberto.Dcb.Postgres Alberto.Dcb.Postgres.Messaging Alberto.Dcb.Telemetry Alberto.Dcb.Testing Alberto.Dcb.Testing.Xunit; do
for proj in Alberto.Dcb Alberto.Dcb.Commands Alberto.Dcb.EntityFramework Alberto.Dcb.InMemory Alberto.Dcb.Messaging Alberto.Dcb.Postgres Alberto.Dcb.Postgres.Messaging Alberto.Dcb.Telemetry Alberto.Dcb.Testing Alberto.Dcb.Testing.Xunit; do
dotnet pack "src/$proj/$proj.csproj" -c Release --no-build --version-suffix "$VERSION_SUFFIX" -o artifacts
done
dotnet pack tools/Alberto.Cli/Alberto.Cli.csproj -c Release --no-build --version-suffix "$VERSION_SUFFIX" -o artifacts
Expand Down
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,12 @@ Note: `apps/Alberto.Payments` is in the solution and builds, but it has no host
- **GraphQL** (Orders example only): HotChocolate 15.x

### Admin surface
The operator surface is the CLI in `tools/Alberto.Cli`. There is no admin HTTP API. `src/Alberto.Dcb.Admin` is a packable project containing the `IAdminReader`/`IAdminOperator` abstraction the CLI's command files are built on; it serves no endpoint.
The operator surface is the CLI in `tools/Alberto.Cli`. There is no admin HTTP API. `src/Alberto.Dcb.Admin` contains the `IAdminReader`/`IAdminOperator` abstraction the CLI's command files are built on; it serves no endpoint.

**Both admin projects are `IsPackable=false`.** `Alberto.Dcb.Admin` and `Alberto.Dcb.Postgres.Admin` build, sit in the solution, are tested, and are referenced by the CLI as projects — they just do not ship to nuget.org, so 1.0 does not freeze `IAdminReader`/`IAdminOperator` under semver before the front doors on `feature/admin-surface` exist. `Alberto.Dcb.Postgres.Admin` was split out of `Alberto.Dcb.Postgres` (which **is** packable) precisely so that package carries no dependency on a parked one; its files keep `namespace Alberto.Dcb.Postgres`, so no consumer's usings changed.

- **Per-processor mutations** go through the core interfaces: `ICheckpointStore` (`SaveAsync`, `ResetAsync`, `RewindAsync`) and `IDeadLetterStore` (`CountAsync`, `ClearAsync`, `MarkForRetryAsync`).
- **`PostgresAdminDataAccess`** (`src/Alberto.Dcb.Postgres`) holds the inspection queries and the composite transactional mutations (`RetryByRewindAsync`, `ReleaseTenantLeasesAsync`) that span multiple tables and so cannot be composed from per-processor interfaces.
- **`PostgresAdminDataAccess`** (`src/Alberto.Dcb.Postgres.Admin`) holds the inspection queries and the composite transactional mutations (`RetryByRewindAsync`, `ReleaseTenantLeasesAsync`) that span multiple tables and so cannot be composed from per-processor interfaces.
- `SaveAsync` is monotonic by design (`GREATEST`). `RewindAsync` is the deliberate escape hatch for operator-initiated rewinds and is the only way to move a checkpoint backwards.

## Technology Stack
Expand Down
1 change: 1 addition & 0 deletions Alberto.slnx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
<Project Path="src/Alberto.Dcb.Testing/Alberto.Dcb.Testing.csproj" />
<Project Path="src/Alberto.Dcb.Testing.Xunit/Alberto.Dcb.Testing.Xunit.csproj" />
<Project Path="src/Alberto.Dcb.Admin/Alberto.Dcb.Admin.csproj" />
<Project Path="src/Alberto.Dcb.Postgres.Admin/Alberto.Dcb.Postgres.Admin.csproj" />
</Folder>
<Folder Name="/apps/">
<Project Path="apps/Alberto.AppHost/Alberto.AppHost.csproj" />
Expand Down
69 changes: 69 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ renamed.
`"{moduleKey}#{shardId}"` string. It now splits into two dimensions: `module` and `shard`.
Custom dashboards built on the old combined value need updating.

**`tenant.id` removed from the tenant-lock counters** — `alberto.tenant_locks_acquired` and
`alberto.tenant_lock_failures` are now tagged by `consumer.id` only. A tenant id is unbounded, and
every distinct tag combination is a time series the SDK holds for the life of the process, so the
old shape cost one series per tenant per replica per counter. Queries that grouped or filtered by
`tenant.id` need rewriting — aggregate by `consumer.id`, or use traces for the per-tenant question.

**Command pipeline reshape** — `Persist`/`PersistUnconditionally` renamed to `Commit`/`CommitUnconditionally`.
`NoValidation()` removed (validation was always optional). `Decide` is now synchronous; use the new
`Enrich` stage for async work before the consistency boundary. `LoadUnder` added for the case where
Expand Down Expand Up @@ -169,8 +175,71 @@ locks in different key spaces and do not serialize against each other**, so the
is unprotected for the length of the overlap. Drain or stop the old version before starting the
new one; see [UPGRADING.md](UPGRADING.md) for the rollout note.

**The admin surface does not ship in 1.0** — `Alberto.Dcb.Admin` is no longer published to nuget.org,
and the PostgreSQL implementation of it moved out of the `Alberto.Dcb.Postgres` package into a new,
also-unpublished `Alberto.Dcb.Postgres.Admin`. `IAdminReader`/`IAdminOperator` are the contract the
admin front doors on `feature/admin-surface` are built on, and shipping them at 1.0 would freeze that
abstraction under semver before anything that consumes it exists. Both projects still build, are in
the solution, and are referenced by `tools/Alberto.Cli` — the operator CLI is unaffected, and so is
every consumer that only ever used the CLI.

The concrete break is for anyone referencing `PostgresAdminDataAccess`, `PostgresAdminOperator` or
`AddAlbertoPostgresAdmin` from the `Alberto.Dcb.Postgres` **package**: those 33 members are gone from
it. Their namespace is unchanged (`Alberto.Dcb.Postgres`), so no `using` needs editing, but the types
now live in an assembly you can only get by project reference. Build the CLI from source, or wait for
the admin surface to be unparked after 1.0.

**Delivered outbox entries are now deleted after 7 days by default** — `WithOutbox` registers an
`OutboxRetentionService` that sweeps hourly and deletes `delivered` entries older than
`deliveredRetention`. Nothing removed them before, so **the first sweep after upgrading faces every
delivered entry the table has ever held**; on a large table that one DELETE can be long. Purge it
once from the CLI during a quiet window first (`alberto ops outbox purge --before <timestamp>`), or
start with a wide `deliveredRetention` and walk it down. Pass `Timeout.InfiniteTimeSpan` to keep
delivered entries forever — do that if the table is your integration audit trail. Only `delivered`
entries are ever eligible; `pending`, `processing` and `failed` are work, not history, and are never
removed by age. **Run migration 034 before deploying the new binary** — it adds the partial index on
`delivered_at` the sweep needs, without which the DELETE is a sequential scan.

### Added

- **The outbox no longer grows forever, and its ordering guarantee is written down.**
`WithOutbox` gains `deliveredRetention` (default 7 days, `Timeout.InfiniteTimeSpan` to disable)
and `retentionSweepInterval` (default 1 hour), honoured by a new `OutboxRetentionService`. It is
deliberately a separate hosted service rather than a step on the relay's loop: a purge that takes
seconds would otherwise be seconds in which nothing is published. It waits out a full interval
before its first sweep, several replicas sweeping at once is safe, and a failed sweep is logged
and retried rather than faulting the host. `alberto ops outbox purge --before <timestamp>` does
the same delete on demand and records an `admin-outbox-purged` audit event;
`IAdminOperator.PurgeOutboxAsync` is the operation behind it. Migration 034 adds the partial index
on `delivered_at` that keeps the sweep off a sequential scan.
Separately, [docs/reactors-and-outbox.md](docs/reactors-and-outbox.md#ordering-there-isnt-any) now
states plainly that outbox delivery is **unordered** — `created_at` is transaction-start time with
no tiebreaker, retries re-deliver late, and `FOR UPDATE SKIP LOCKED` hands concurrent relays
disjoint batches — and points at `ExternalMessage.RoutingHint` as the per-entity ordering hook for
transports that have partition keys, message-group ids or routing keys. No behaviour changed
there; it was true before and undocumented.
- **A conflict that outlives `Commit`'s retries now reaches the client as a coded error.**
`Commit` retries a `DcbConflictException` up to its attempt limit and then rethrows; the example
slices' `OrThrow` awaited that bare, so a boundary that stayed contended surfaced as an unhandled
exception — a 500 with no code on it, which is the worst error on the documented happy path.
`OrThrow` now catches it and raises the same `Problem` a `TryCommit` would have returned, so
`Handle → Load → Decide → Commit → OrThrow` stays the shape a slice is written in and
`TryCommit` stays what it was for: branching on the failure rather than reporting it.
`DcbConflictException` gains `ProblemCode` (`"dcb.conflict"`) and `ToProblem()` so both paths
render one shape and callers branch on a constant. `Problem.Details` now reach GraphQL as error
extensions instead of being dropped, which is how `expectedPosition` and `conflictingPosition`
get to a client deciding whether to retry.
- **Every destructive CLI command is now recorded.** Every mutation in `alberto ops` routes
through `IAdminOperator`, which appends an admin event to `alberto_events` in the same
transaction as the change. Previously `checkpoint set`, `checkpoint reset`, `checkpoint rename`,
`dead-letters dismiss`, `dead-letters retry` and `tenants release` reached past it to the
underlying stores and left no trace; only `retry-rewind` and the three rebuild verbs were
audited. `IAdminOperator` gains `RenameCheckpointAsync` and `MarkDeadLettersForRetryAsync` (and
two matching event types, `admin-checkpoint-renamed` and `admin-dead-letters-marked-for-retry`)
so the two commands that had no operator-level equivalent now have one. The recorded operator id
is the CLI's OS user name — attribution for a cooperating team, not authentication; database
credentials remain the access control. See
[docs/operations.md](docs/operations.md#what-a-mutation-records).
- `TelemetryOptions.RecordEventTagValues` (default `false`) — opts append spans back into
carrying tag values alongside tag keys, for a development environment or a collector inside
the same trust boundary as the database.
Expand Down
10 changes: 7 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,12 +97,16 @@ Note: `apps/Alberto.Payments` is in the solution and builds, but it has no host
- **GraphQL** (Orders example only): HotChocolate 15.x

### Admin surface
The operator surface is the CLI in `tools/Alberto.Cli`. There is no admin HTTP API. `src/Alberto.Dcb.Admin` is a package, but only the `IAdminReader`/`IAdminOperator` abstraction the CLI's 14 command files are built on — it serves no endpoint, and `AddAlbertoPostgresAdmin` in `src/Alberto.Dcb.Postgres` is its only implementation.
The operator surface is the CLI in `tools/Alberto.Cli`. There is no admin HTTP API. `src/Alberto.Dcb.Admin` holds the `IAdminReader`/`IAdminOperator` abstraction the CLI's 14 command files are built on — it serves no endpointand `AddAlbertoPostgresAdmin` in `src/Alberto.Dcb.Postgres.Admin` is its only implementation.

**The in-process front doors are parked, not missing.** A GraphQL admin API, an MCP server, a React console and a BFF live on `feature/admin-surface`, held out of 1.0 so their field and tool names are not frozen by semver. Do not rebuild them on main — extend that branch. Keep `IAdminReader`/`IAdminOperator` additive when changing them here, or the branch stops merging cleanly.
**The whole admin surface is parked, not missing — and that includes its two projects.** A GraphQL admin API, an MCP server, a React console and a BFF live on `feature/admin-surface`, held out of 1.0 so their field and tool names are not frozen by semver. `Alberto.Dcb.Admin` and `Alberto.Dcb.Postgres.Admin` are both `IsPackable=false` for the same reason: shipping `IAdminReader`/`IAdminOperator` at 1.0 would freeze the abstraction under semver before the things that consume it exist. They build, they are in the solution, they are tested, and the CLI references them by project — they just do not go to nuget.org. Unparking is `IsPackable=true` plus capturing `PublicAPI.Shipped.txt` (the analyzer is gated on `IsPackable`, so it is inert until then).

`Alberto.Dcb.Postgres.Admin` exists only because `Alberto.Dcb.Postgres` **is** packable. `PostgresAdminDataAccess`, `PostgresAdminOperator` and `PostgresAdminServiceCollectionExtensions` used to live there, which made its nupkg carry an unresolvable `Alberto.Dcb.Admin` dependency and 33 public members returning parked types. The three files keep `namespace Alberto.Dcb.Postgres` so no consumer's usings changed, and they reach back for internals (`SchemaQualifier`) via `InternalsVisibleTo`.

Do not rebuild the front doors on main — extend that branch. Keep `IAdminReader`/`IAdminOperator` additive when changing them here, or the branch stops merging cleanly.

- **Per-processor mutations** go through the core interfaces: `ICheckpointStore` (`SaveAsync`, `ResetAsync`, `RewindAsync`) and `IDeadLetterStore` (`CountAsync`, `ClearAsync`, `MarkForRetryAsync`).
- **`PostgresAdminDataAccess`** (`src/Alberto.Dcb.Postgres`) holds the inspection queries and the composite transactional mutations (`RetryByRewindAsync`, `ReleaseTenantLeasesAsync`) that span multiple tables and so cannot be composed from per-processor interfaces.
- **`PostgresAdminDataAccess`** (`src/Alberto.Dcb.Postgres.Admin`) holds the inspection queries and the composite transactional mutations (`RetryByRewindAsync`, `ReleaseTenantLeasesAsync`) that span multiple tables and so cannot be composed from per-processor interfaces.
- `SaveAsync` is monotonic by design (`GREATEST`). `RewindAsync` is the deliberate escape hatch for operator-initiated rewinds and is the only way to move a checkpoint backwards.
- **Sharded modules**: `ShardResolver` turns `--shard`/`--all-shards` plus `.alberto/config.json` into the databases a command runs against. Reads fan out by default; mutations refuse without a selection. `alberto shards list|where|assign` manages the catalog. Shard connection strings live in config, never in the catalog table.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ dotnet add package Alberto.Dcb --prerelease
| `Alberto.Dcb` | Event store abstractions, control loop, middleware, projections, tenancy |
| `Alberto.Dcb.Commands` | The `AlbertoStore` command pipeline (`Handle → Load → Decide → Commit`) |
| `Alberto.Dcb.InMemory` | In-memory backend, checkpoint, dead-letter and state stores — dev and tests |
| `Alberto.Dcb.Postgres` | PostgreSQL backend, migrations, leases, admin queries |
| `Alberto.Dcb.Postgres` | PostgreSQL backend, migrations, leases |
| `Alberto.Dcb.EntityFramework` | EF Core–backed projections |
| `Alberto.Dcb.Messaging` | Transactional outbox abstractions |
| `Alberto.Dcb.Postgres.Messaging` | PostgreSQL outbox store |
Expand Down
Loading
Loading