1.0.0

The first production release of Cloud APIM - Event Brokers for Otoroshi: event brokers behind your
Otoroshi routes. Publish events with a plain HTTP call, consume them as newline-delimited JSON, as
server-sent events or over a websocket, push them to webhooks. The brokers stay private, and Otoroshi
decides who may reach them.

1.0.0 is a rewrite of the 0.0.x prototype: new core, new clients, new formats. Read
Upgrading from 0.0.x before you install it over a prototype.

Documentation: https://cloud-apim.github.io/otoroshi-event-brokers-extension/

Requirements

• Otoroshi 18 (built against 18.0.0) and Java 17 or later.
• One jar, otoroshi-event-brokers-extension_3-1.0.0.jar, to put on Otoroshi's classpath. It is an
  assembly that ships the RabbitMQ and MQTT clients. The Kafka, Pulsar, Redis and Artemis clients come
  from Otoroshi.

java -cp "./otoroshi-event-brokers-extension_3-1.0.0.jar:./otoroshi.jar" play.core.server.ProdServerStart

Brokers

broker	Broker	Client
kafka	Apache Kafka	pekko-connectors-kafka (otoroshi)
rabbitmq	RabbitMQ (AMQP 0.9.1): classic, quorum and stream queues	amqp-client 5
pulsar	Apache Pulsar	Pulsar client (otoroshi)
redis	Redis pub/sub, or Redis streams	Lettuce (otoroshi)
mqtt	any MQTT 5 or 3.1.1 broker, over TCP or websockets	HiveMQ MQTT client
artemis	ActiveMQ Artemis, on its core protocol (new)	Artemis core client (otoroshi)

What's in it

Connections every node keeps for you

• Every node connects in the background, with backoff and healthchecks, and rebuilds a client after
  too many failures. New settings connect before the old ones close (make before break), so requests
  are not dropped. Status API, status badges in the backoffice, connection events and alerts.
• Settings typed per broker and checked on write. Every error is reported with its path, and an
  unknown field is an error. ${vault://...} references are kept and resolved for the clients. The
  backoffice form follows the broker, and _template?broker= gives a valid starting point.
• TLS on every broker with the certificates of the Otoroshi store, mutual TLS included (Kafka
  through inline PEM, no files on disk).
• Authentication on every broker: Kafka SASL SCRAM and PLAIN, Pulsar tokens, Redis ACL users, and
  user / password for RabbitMQ, MQTT and Artemis.

Each broker at its best

• Kafka: keys, partitions, offsets, consumer groups, producer tuning (acks, idempotence,
  compression, batching, linger), server-sent events resumed by offset, consumer lag.
• RabbitMQ: publisher confirms, a declarative topology (exchanges, queues of every type,
  bindings), stream queues replayed from the first message, an offset or a date, server-sent events
  resumed on streams.
• Pulsar: subscriptions of every type, server-sent events resumed by message id, partitioned topics
  included.
• Redis: pub/sub, or streams with consumer groups, acks, claims of abandoned entries, replay,
  server-sent events resume and consumer lag.
• MQTT 5 and 3.1.1 through HiveMQ: shared subscriptions, user properties, websockets transport,
  reconnection.
• ActiveMQ Artemis: multicast and anycast, durable shared subscriptions, message groups, selectors,
  duplicate detection, delayed delivery.

Routes

• HTTP events ingester: one event, a batch or newline-delimited events, and one result per event
  once the broker took it.
• HTTP events consumer (newline-delimited JSON), SSE events consumer (heartbeats,
  Last-Event-ID resume), Websocket ingester / consumer (client acks, max_in_flight). All
  support groups, at-least-once delivery and replay.
• Events request reply (new): publishes the request and answers with the event that replies to
  it, over every broker.
• The route decides what its callers may ask:
  • options are fixed by the route, with Otoroshi's expression language
    (orders-${apikey.metadata.tenant});
  • allowed_overrides lists the options a caller may set, anything else is a 400;
  • allowed_topics holds the glob patterns a caller's topic must match.

CloudEvents everywhere

Every event is a CloudEvents 1.0 event, carried with the protocol binding of each broker:

• binary mode with ce_ headers on Kafka, Pulsar and Redis streams;
• application properties on RabbitMQ and Artemis;
• user properties on MQTT 5;
• a structured payload on Redis pub/sub and MQTT 3.1.1.

A message that came without CloudEvents attributes gets them, and broker metadata (partition, offset,
stream id...) is added as extensions. Consumers choose format=cloudevents (the default) or
format=raw.

Webhooks (new)

Events pushed to HTTP endpoints:

• three modes: structured, binary or raw;
• signed with an HMAC (X-Event-Brokers-Signature);
• retried with backoff, dead-lettered to another connection, or reported by an
  EventBrokerWebhookDeliveryFailed alert.

One node of a cluster delivers each webhook at a time. The nodes decide which one in a Redis they share
(webhooks.redis.uri), which is required in a cluster.

Integrations (new)

• Try a connection from the backoffice or the admin API: publish a test event, or peek at what a
  topic or a queue holds without acking anything.
• Data exporter: Otoroshi's own events (analytics, alerts, audit) exported to any connection, as
  CloudEvents.
• Workflow functions: extensions.com.cloud-apim.event-brokers.publish and .consume.

Observability

• EventBrokerConnectionEvent on each state change, EventBrokerConnectionUnavailableAlert once per
  outage.
• Traffic counters per connection and per route: published, failed, consumed, acked, open
  subscriptions, publish latency. They are shown in the status API and the backoffice, and sent to
  Otoroshi's metrics.
• The lag of consumer groups, where the broker tells it (Kafka, Redis streams).

0.0.20

chore(deps): bump otoroshi 17.16.0 -> 17.17.0

0.0.19

chore(deps): bump otoroshi 17.15.0 -> 17.16.0

0.0.18

chore(deps): bump otoroshi 17.14.0 -> 17.15.0

0.0.17

chore(deps): bump otoroshi 17.13.0 -> 17.14.0

0.0.16

chore(deps): bump otoroshi 17.12.0 -> 17.13.0

0.0.15

chore(deps): bump otoroshi 17.11.0 -> 17.12.0

0.0.14

chore(deps): bump otoroshi 17.10.0 -> 17.11.0

0.0.13

chore(deps): bump otoroshi 17.9.2 -> 17.10.0

0.0.12

chore(deps): bump otoroshi 17.9.1 -> 17.9.2