Skip to content

v2.0.0

Latest

Choose a tag to compare

@dmitriy-iliyov dmitriy-iliyov released this 15 Oct 09:04
· 22 commits to main since this release

oncebox 2.0.0

The library is now named by its two mechanisms, outbox and inbox, every time is taken from the database
clock instead of the JVM, and the inbox gets a second, at-most-once consumer. Configuration keys, tables, metrics,
bean names and message headers change, so this is a major version.

Highlights

  • Two idempotent consumers. TransactionalIdempotentConsumer (effectively-once: the event id and the operation
    share one transaction) and NonTransactionalIdempotentConsumer (at-most-once: the id is committed before the
    operation runs, for side effects that must not repeat, such as sending an email). A listener chooses one with
    @TransactionalConsumer or @NonTransactionalConsumer.
  • Time zone independent. Stuck-batch recovery, retry scheduling and every cleanup compare against the database
    clock, so instances in different time zones agree on what is stuck or expired.
  • Fencing of captured batches. Each capture writes a lock token, and a poller that stalled past stuck recovery
    can no longer overwrite the outcome of the poller that captured the batch after it.
  • DLQ REST API errors as RFC 9457 ProblemDetail, with a type URI per error and validation errors listed by field.

Breaking changes

Naming

  • Properties: oncebox.publisher.* to oncebox.outbox.*, oncebox.consumer.* to oncebox.inbox.*.
  • Annotation: @EnableOutbox to @EnableOncebox.
  • Classes: OutboxIdempotentConsumer to IdempotentConsumer, OutboxEventIdExtractor to EventIdExtractor,
    OutboxHeaders to OnceboxHeaders, OutboxHeadersUtils to OnceboxHeadersUtils.
  • Packages: *.publisher to *.outbox, *.consumer to *.inbox; OutboxEvent, EventStatus and
    SenderResult move from core.outbox.domain to core.outbox.
  • Artifacts: oncebox-consumer-cache to oncebox-inbox-cache; oncebox-messaging is merged into oncebox-core.
  • Tables: outbox_consumed_events to inbox_events, outbox_jobs to oncebox_jobs.
  • Metrics: consumed_outbox_* to inbox_*; outbox_*_tasks_total, outbox_task_processing_duration and
    outbox_polling_delay_milliseconds to oncebox_*; the task_type value outbox-consumed-cleanup is now
    inbox-cleanup. The bundled Grafana dashboards are renamed accordingly.
  • Message headers: outbox_event_{id,type,payload_type} to oncebox_event_*.
  • Bean names: the listener container factories and message converters are inbox*, shared beans such as
    onceboxJdbcTemplate and onceboxScheduledExecutorService are oncebox*.

Inbox

  • IdempotentConsumer has two implementations, registered as inboxTransactionalIdempotentConsumer and
    inboxNonTransactionalIdempotentConsumer. Neither is @Primary: injecting IdempotentConsumer without
    @TransactionalConsumer, @NonTransactionalConsumer or @Qualifier fails the context.
  • An implementation is replaced through the beans inboxUndecoratedTransactionalIdempotentConsumer and
    inboxUndecoratedNonTransactionalIdempotentConsumer.
  • Within one service an event is handled by one consumer: the inbox records an event by its id alone, whichever
    consumer recorded it.

Database

  • Every time is set by the database. The time columns are PostgreSQL TIMESTAMPTZ, MySQL DATETIME(6) holding UTC
    and Oracle TIMESTAMP WITH TIME ZONE, with defaults from the database clock; inserts do not write them.
  • MySQL 8.0.13 or newer is required, for the expression defaults.
  • outbox_events has a nullable lock_token column.
  • The starter no longer needs a Clock bean. Only the DLQ API error handler reads one: the application Clock when
    there is exactly one, the UTC system clock otherwise.

API for custom implementations

  • The repositories, managers, DefaultOutboxDlqEventMapper, JacksonOutboxSerializer, DefaultOutboxProcessor,
    DefaultOnceboxJobCreateCommand and the repository factories no longer take a Clock.
  • Repository methods take a Duration age instead of an Instant threshold: deleteBatchOlderThan,
    deleteBatchByStatusOlderThan, updateBatchStatusByStatusOlderThan.
  • The outcome of a batch is written by updateBatchStatusByLockToken and partiallyUpdateBatchByLockToken;
    OutboxManager.loadBatch and finalizeBatch take the lock token.
  • OutboxManager.finalizeBatch takes a Function<Integer, Duration> giving the delay before the next retry;
    OutboxEvent carries it as retryDelay.
  • A new event is made by new OutboxEvent(id, eventType, payloadType, payload); its times are null until it is read
    from the database.
  • New methods: OutboxRepository.saveBatchWithCreatedAt and OutboxManager.saveBatchWithCreatedAt; in
    AbstractOutboxRepository updateBatchStatusByLockTokenSql, partiallyUpdateBatchByLockTokenSql and
    setCreatedAt; in AbstractOutboxDlqApiRepository currentTimestampSql.
  • SqlIdHelper is SqlUuidHelper. A MySQL repository built by hand takes MySqlResultSetMapper.
  • OutboxManager.loadBatch(EventStatus, int) and OutboxDlqManager.loadAndLockBatch require a transaction of the
    caller (MANDATORY); finalizeBatch rejects a negative maxRetryCount.
  • moved_at is set by the database: OutboxDlqRepository.saveBatch does not write it, and events made by
    OutboxDlqEventMapper.toDlqEvent carry a null movedAt.
  • InboxRepository.saveIfAbsent(Set) and OutboxDlqRepository.saveBatch and deleteBatch throw
    NullPointerException on null.

DLQ REST API

  • Errors are application/problem+json (RFC 9457) with a type under https://oncebox.io/errors/, the request path
    as instance and a timestamp. Validation errors come as a list of field and message pairs.
  • An event in IN_PROCESS is answered with 409 Conflict instead of 400, and IN_PROCESS is rejected as a target
    status.

Startup checks

  • The outbox refuses to start when sender.emergency-timeout is not below
    stuck-recovery.max-batch-processing-time: a batch still being sent would otherwise be recovered as stuck and
    published twice.

New

  • @TransactionalConsumer and @NonTransactionalConsumer qualifier annotations for a constructor or method
    parameter, a field or a setter.
  • NonTransactionalIdempotentConsumer with an at-most-once guarantee; the ids a failed call recorded are logged at
    error.
  • Events brought back from the DLQ keep their original created_at, start their retries over and are due at once.

Fixed

  • Recovery, retry scheduling and the cleanup of processed events, the DLQ and the inbox no longer depend on the time
    zone of the JVM: an instance in another zone no longer takes a fresh capture for stuck, and no longer deletes
    records too early or too late.
  • A stale poller can no longer overwrite the outcome of a newer capture, which could leave a delivered event FAILED
    and in the DLQ, or a failed one PROCESSED.
  • Durations below a second were truncated to whole seconds: a backoff.delay of 500ms retried at once, an
    emergency-timeout of 500ms failed every batch, a sub-second metrics cache ttl expired every value. A fractional
    backoff.multiplier lost its fraction (1.5 with 10s gave 10s, 20s, 30s instead of 15s, 22.5s, 33.75s).
  • A schedule strategy listener that threw stopped the polling of its event type until restart, and a failing
    onExecutionSucceeded was counted as a failed run.
  • The inbox cache never hit on in-memory caches (ConcurrentMapCacheManager, Caffeine): ids were read as strings and
    written as UUIDs.
  • The MySQL DLQ schema shipped by the starter lacked the index on dlq_status and moved_at used by the DLQ API.
  • The Kafka inbox warned about a non-manual ack-mode, which is not needed; it now warns when
    enable.auto.commit=true, which does lose events.
  • DLQ REST API: standard Spring MVC failures (unreadable body, missing parameter, unsupported method) keep their own
    status instead of 500, the 500 response no longer exposes the exception message, and a catch-all
    @ControllerAdvice of the application no longer answers DLQ API exceptions in its own format.

Improved

  • Logs carry their context: task, worker, event type, lock token and counts; an empty poll is logged at debug.
  • Configuration error messages name the property as the YAML key, for example 'sender.emergency-timeout'.
  • README: the two delivery guarantees, broker acknowledgement settings, cache name uniqueness across services sharing
    a cache store, and corrected property names and defaults.
  • The example stack provisions Grafana with the datasource and both dashboards.