Skip to content

v1.6.0

Choose a tag to compare

@github-actions github-actions released this 19 Aug 17:08
· 9 commits to main since this release

The broker stops being mandatory. A dispatcher can now deliver into a table — the consumer's inbox
— and a producer that is not on pgx can write to the outbox.

Added

  • A postgres driver: delivery into a table instead of to a broker. The destination is the
    consumer's inbox, and the dispatcher only inserts into it — it does not create the table, read
    it, update it or clean it up. That boundary is what keeps this a destination rather than the
    beginnings of a consumer framework.

    It buys a guarantee no broker offers. Delivery stays at-least-once, because the insert and the
    write-back marking the row sent are two commits and a replica can die between them, but the
    inbox's primary key makes the repeat harmless: the driver inserts with ON CONFLICT (id) DO NOTHING and reports a conflict as a delivery. Deduplication stops being an obligation on the
    consumer's code and becomes a property of its schema.

    It also costs nothing to carry. pgx is already linked, so the driver adds zero modules
    the only candidate on the driver spec of which that is true, against
    +1.58 MB for NATS and +6.33 MB for Redis Streams. And it removes the broker from the list of
    things a deployment must run at all.

    An empty DSN means the database the dispatcher already reads its outbox from, which is the
    modular-monolith case. Two configurations are refused at startup: a table that does not exist,
    and a destination that is the dispatcher's own outbox table — which would deliver to itself,
    every published message becoming a new message to publish.

    A batch is one statement on the happy path. When the database refuses, the batch is replayed one
    message at a time, because a positional error slice is the contract and one malformed message
    must not condemn the rest; that second pass is skipped when the database could not be reached at
    all. Classification reads the server's own SQLSTATE, which makes this the least guessy of the
    three drivers.

    Example schema and the reasoning behind it:
    migrations/inbox/messages.sql. Use cases and the flows:
    docs/InboxSpec.ru.md; whether it was worth building at all:
    docs/PostgresDestination.ru.md.

    Three properties are measured rather than asserted. A batch is one statement — proved by a
    statement-level trigger on the inbox, which fires once per statement whatever the row count,
    rather than by timing. On the failure path the errors land positionally, including on the first
    and last message, where an off-by-one in the isolating pass would show. And there is no
    payload-size class at all: 32 MiB in one message and 64 MiB in one batch go in and come back byte
    for byte, where every broker driver has a permanent failure for exceeding a frame or a
    message.max.bytes.

    Cleaning the inbox is the consumer's. The janitor sweeps the outbox and never touches the
    destination, so the inbox grows until its owner cleans it — quietly, because nothing on the
    dispatcher's side is looking. It is the one failure of this driver that surfaces six months later
    rather than immediately, which is why it is stated in three places rather than one.

    No fan-out. An inbox is point-to-point, so one event reaching three consumers means the
    producer writing three rows. Where a fan-out is wanted, a broker is still the right tool, and the
    documentation says so rather than leaving it to be discovered on the second subscriber.

  • Three use cases for it, one page each:
    a modular monolith with no broker at all,
    two services delivering into each other's inbox, and
    dead letters in a table rather than a topic — which needs no code
    change, because the forwarder already publishes through the router.

Fixed

  • CI builds against the current Go patch release. go.mod says go 1.26, and without
    check-latest the action takes whatever patch the runner had cached — which is how the same
    commit went green on 1.26.6 and red on 1.26.5. Standard library advisories are fixed by patch
    releases, so which one a run gets decided whether govulncheck had anything to report. It now
    applies to the release workflow too, where the stake is higher: a release could otherwise ship
    binaries built against a known-vulnerable crypto/tls on a day the cache happened to be stale,
    and nothing in the pipeline would have said so.

Changed

  • config.DBConfig.ConnString replaces the unexported DSN assembly in internal/store. The pool
    is no longer the only thing that needs it: a driver delivering into PostgreSQL has to be able to
    say "the same database the dispatcher reads from" without repeating how that database is
    described.

  • pkg/outboxsql, the producer client for everybody not on pgx. pkg/outboxclient takes a
    pgx.Tx, which is the wrong dependency to force on a codebase that chose sqlx, gorm or the
    standard library. The new one takes anything with ExecContext and imports no driver at all.

    It is a package of its own rather than another constructor in outboxclient, because importing
    that package brings pgx with it — which is the thing being avoided. The two Message types are
    therefore duplicated, and a test compares them by reflection so the copies cannot drift apart
    unnoticed.

    It costs the daemon nothing: go list -deps ./cmd/outbox does not contain either package, and
    the binary measures 21.87 MB before and after. Integration tests cover both PostgreSQL drivers a
    database/sql user realistically holds — lib/pq and pgx/v5/stdlib — including a payload of
    arbitrary non-UTF-8 bytes and a jsonb round trip. The recipe is
    use case 6.

Changed

  • The use cases are one page each, in docs/usecases, with
    docs/UseCases.md as the index. The single document had reached seven hundred
    lines and six recipes, which is past the point where anybody reads it end to end. Every recipe is
    carried over unchanged.