-
-
Notifications
You must be signed in to change notification settings - Fork 0
Packages
eventferry is a small monorepo of focused packages, each on independent SemVer. Pick exactly the ones you need; nothing is forced on you transitively. Every adapter declares its native client (pg, mysql2, kafkajs, @confluentinc/kafka-javascript, @kafkajs/confluent-schema-registry, aws-msk-iam-sasl-signer-js) as an optional peer dependency so your install footprint matches your actual surface area.
| Package | Role | Required peers | Optional peers | Latest |
|---|---|---|---|---|
@eventferry/core |
DB/broker-agnostic primitives — Relay, types, registry, error taxonomy |
— | — | 3.4.0 |
@eventferry/postgres |
PostgresStore + migration SQL + streaming/notify relays |
@eventferry/core |
pg, pg-logical-replication
|
3.3.2 |
@eventferry/mysql |
MysqlStore + migration SQL + binlog relay |
@eventferry/core |
mysql2, @vlasky/zongji
|
3.3.2 |
@eventferry/kafka |
KafkaPublisher + admin + consume subpath + tuning |
@eventferry/core |
kafkajs, @confluentinc/kafka-javascript
|
3.5.0 |
@eventferry/schema-registry |
Confluent SR serializer — Avro / Protobuf / JSON | @eventferry/core |
@kafkajs/confluent-schema-registry |
3.4.0 |
@eventferry/kafka-iam |
AWS MSK IAM SASL/OAUTHBEARER helper | @eventferry/kafka |
aws-msk-iam-sasl-signer-js |
0.2.1 |
@eventferry/all |
Meta-package — re-exports core + DB + Kafka + SR | (the four above) | — | 3.2.5 |
The DB-and-broker-agnostic primitives. Everything else builds on this.
npm i @eventferry/coreWhat you get:
-
Relay— the polling orchestrator. Callsstore.claimBatch→publisher.publish→store.markDone/markFailed/requeue. -
defineOutbox(registry)— Standard Schema-based typed event registry. Same registry on producer + consumer sides. -
OutboxStore,Publisher,Serializerinterfaces — what an adapter must satisfy. -
PublishErrorKindtaxonomy —retriable | fatal | poison | backpressure | quota | fenced. -
OutboxValidationError,OutboxRecord,PublishableMessage,PublishResulttypes. - Default JSON
Serializer.
You always install this. It has no native peers — pure JS / TS, runs anywhere Node 18+ runs.
When NOT to use alone: core has no store and no publisher. You always pair it with at least one DB adapter (@eventferry/postgres or @eventferry/mysql) and at least one publisher (@eventferry/kafka).
PostgreSQL 12+ store and migration helpers.
npm i @eventferry/postgres pg
# Optional, for the streaming relay:
npm i pg-logical-replicationWhat you get:
-
PostgresStore— theOutboxStoreimplementation. Batch claim viaSKIP LOCKED, server-side TZ-safe reaper, JSONB payload / headers,BIGINTids. -
createMigrationSql(table)— the table + indexes as a single SQL string. Run it via your existing migration tool (or justpool.query). -
PostgresStreamingRelay— sub-millisecond wake onINSERTviapgoutputlogical replication. Requireswal_level=logicalon the cluster. -
PostgresNotifyWaker— middle ground:LISTEN/NOTIFYwakes the pollingRelayso you skip the poll interval when nothing's pending. No cluster config required.
When to use:
- You're on Postgres 12+ (which is everyone in 2026 — even 12 is EOL, 16 is the current LTS).
- You want sub-second event-to-broker latency without polling-tax →
PostgresStreamingRelayorPostgresNotifyWaker.
When NOT:
- Your DB is MySQL/MariaDB → use
@eventferry/mysql. - You're on a managed Postgres without superuser access AND want logical replication → talk to your provider first (RDS / Cloud SQL both support it, but require flag toggles).
MySQL 8+ / MariaDB 10.6+ store.
npm i @eventferry/mysql mysql2
# Optional, for the binlog relay:
npm i @vlasky/zongjiWhat you get:
-
MysqlStore— batch claim viaSELECT ... FOR UPDATE SKIP LOCKED, server-sideINTERVAL SECONDreaper (no application-clock surprises),JSONcolumns for payload / headers,BIGINT AUTO_INCREMENTids. -
createMigrationSql(table)— DDL string, drop into your migration runner. -
MysqlBinlogRelay— sub-millisecond wake from the ROW-mode binlog. Requireslog-bin=mysql-bin,binlog-format=ROW,binlog-row-image=FULL, andREPLICATION SLAVE+REPLICATION CLIENTgrants on the relay user.
When to use:
- You're on MySQL 8+ or MariaDB 10.6+ (both support
SKIP LOCKED; older versions can't run the batch claim). - You want a binlog-driven low-latency relay without setting up Debezium.
When NOT:
- Older MySQL (5.7, MariaDB <10.6) → no
SKIP LOCKED. Consider upgrading first. - You want logical replication ergonomics in MySQL → look at Debezium + Kafka Connect; eventferry's binlog relay is intentionally lighter weight (no Connect cluster, no schema registry of its own).
The Kafka / Redpanda publisher.
npm i @eventferry/kafka
# Pick a driver:
npm i kafkajs # pure JS, zero native deps
# or:
npm i @confluentinc/kafka-javascript # librdkafka, fine-grained tuningWhat you get:
-
KafkaPublisher— thePublisherinterface implementation. Idempotent + transactional EOS, error classification, DLQ enrichment, OTel tracing hook, lifecycle hooks. - Two drivers via the same surface — switch with
driver: "kafkajs" | "confluent". Capability gaps (e.g.lingerMson kafkajs) log a one-time warning. -
publisher.admin(),publisher.ensureTopics(specs, { growPartitions }),validateTopicsOnConnect. -
publisher.healthCheck({ timeoutMs })— cheap reachability probe for/healthz//readyz. -
@eventferry/kafka/consumesubpath —decode(message, { decoder })+extractTraceContext(headers)for the consumer side. Producer-free import — your consumer doesn't pull in kafkajs/confluent twice. - Power-user escape hatches:
rawProducerConfig(librdkafka),rawKafkaJsProducerConfig(kafkajs),customPartitioner,compressionLevel. - Producer-fenced restart with optional
autoRecoverFromFence. - librdkafka stats hook (
onStats).
When to use:
- Any time you want eventferry's outbox rows to land on a Kafka or Redpanda topic. Same package handles both — Redpanda speaks the Kafka protocol.
Driver choice rule of thumb:
-
kafkajs— default. Pure JS, no native build step, fine in Docker / Lambda / Alpine. Limited tuning surface (nolingerMs,batchSizeetc.). -
confluent— pick when you need real librdkafka throughput (queue buffer tuning, compression level, stats), or when AWS MSK IAM with a high message rate matters. Adds a C++ build dep.
Confluent Schema Registry serializer — Avro / Protobuf / JSON Schema in the Confluent wire format.
npm i @eventferry/schema-registry @kafkajs/confluent-schema-registryWhat you get:
-
SchemaRegistrySerializer— drop-in replacement for the default JSON serializer. Plug intonew Relay({ serializer }). -
subjectStrategy:TopicNameStrategy(default) |RecordNameStrategy|TopicRecordNameStrategy. -
schemas/keySchemas— per-topic schema registration with idempotent caching. -
serializeKey(record)— Avro-encoded message keys (call from your publish glue, not the relay). -
autoRegister: false— production registries where schemas are managed out-of-band (matches Confluent client'sauto.register.schemas=false). - Auth:
{ type: "basic", username, password }or{ type: "bearer", token }(callable for rotation).
When to use:
- You're consuming with the Confluent JVM stack, kafka-streams, kafka-connect, or any client that expects the magic-byte + schema-id wire format.
- You want schema evolution checks (backwards / forwards compatibility) enforced by the registry.
When NOT:
- Pure JSON consumers in the same monorepo as your producer.
defineOutbox(registry).decode()gives you typed payloads without a registry — see Type-Safe Events. - A non-Confluent registry (Apicurio, AWS Glue). Those need separate adapters; the wire format differs.
AWS MSK IAM (SASL/OAUTHBEARER over SigV4) helper. Separate package so the core Kafka surface stays AWS-free.
npm i @eventferry/kafka-iam aws-msk-iam-sasl-signer-jsWhat you get:
-
createMskIamSasl({ region })→ returns a ready-to-usesaslblock forKafkaPublisher. - Process-local token cache with a configurable refresh-ahead window (default 60s on the 15-min MSK lifetime).
- Concurrent-refresh dedup — thundering-herd cannot hit the SigV4 signer more than once per refresh.
- Transient signer failure recovery (the in-flight slot is cleared, next call retries cleanly).
- Custom signer DI for tests.
- Named profile + assumed-role support (
awsProfile,awsRoleArn,awsRoleSessionName).
When to use:
- AWS MSK with IAM auth enabled.
When NOT:
- Any non-MSK Kafka — MSK IAM is AWS-specific. For Confluent Cloud's OAuth, wire
SaslOauthbearerConfigdirectly with your IdP.
Meta-package. Installs and re-exports core + postgres + mysql + kafka + schema-registry in one shot.
npm i @eventferry/all@eventferry/kafka-iam is deliberately excluded from this — it's AWS-specific and would pull aws-msk-iam-sasl-signer-js into every install that wants Postgres + Kafka.
When to use:
- Getting started, prototyping, or a single-service app where one big
import { ... } from "@eventferry/all"keeps things tidy.
When NOT:
- Production services that only need one database. Installing
@eventferry/allto use Postgres-only pulls in the MySQLmysql2optional surface too (silent, but noise on your dependency tree). Prefer the individual packages.
Every package follows independent SemVer — a minor on @eventferry/kafka doesn't force a bump on @eventferry/postgres. Cross-package compatibility is encoded via peerDependencies:
- Adapters declare a
peerDependenciesfloor on@eventferry/core(currently^3). -
@eventferry/kafka-iamdeclares a floor on@eventferry/kafka(currently^3). - The optional native peers (
pg,mysql2,kafkajs, etc.) are declared withpeerDependenciesMeta: { optional: true }so you only install what you actually use.
Mixing a @eventferry/core@3.x with @eventferry/postgres@2.x is flagged by npm / pnpm as a peer mismatch — fix the alignment, don't ignore the warning.
- Configure your store → Postgres Adapter or MySQL Adapter
- Configure the publisher → Kafka Publisher
- Stitch them together with type safety → Type-Safe Events
Repository · Issues · npm: @eventferry/all · MIT
Get going
Adapters
Type & schema
Security
Operational
- Transactions and EOS
- Admin Operations
- Observability
- Consuming Events
- Dead-Letter Queue
- Reliability and Error Handling
Operations
Reference