Kafka Message Bus 🚌✨

Introduction

Kafka Message Bus is a wrapper for KafkaJS, enhancing its functionality with deadletter topic support and handlers. It efficiently manages failed message processing in Kafka topics by redirecting them to deadletter topics for resolution, improving reliability in Kafka-based systems.

Getting Started

1. Clone the repository:

   git clone https://github.com/tomer555/kafka-message-bus.git
   

2. Install dependencies:

   npm install
   

Running Tests 🧪

Execute tests using:

npm run test

Kafka Provider Usage 📡

Configure, initialize, and use the Kafka provider in your application.

In-Memory Provider Usage 🧠

Set up and use the In-Memory provider for testing or lightweight handling.

Usage Examples

Setting Up the Consumer

This example demonstrates how to define a consumer with deadletter handling using only the kafka-message-bus package. The consumer listens to a specified topic, processes each message, and handles errors by utilizing a deadletter queue.

import { KafkaConsumerMessage } from "kafka-message-bus";

// Function to simulate processing of a generic event
async function processEvent(event: any) {
  // Replace with your actual event processing logic
  console.log(`Processing event: ${JSON.stringify(event)}`);
}

// Define the Kafka consumer
const exampleConsumer = {
  topic: "your-topic-name",
  deadletter: true,
  handler: async (kafkaMsg: KafkaConsumerMessage) => {
    console.log(
      `Received message in ${this.topic}: ${JSON.stringify(kafkaMsg)}`
    );
    try {
      // Assuming 'kafkaMsg.message' contains the event object
      await processEvent(kafkaMsg.message);
    } catch (error) {
      console.error(
        `Failed to process message in ${this.topic}: ${JSON.stringify(
          kafkaMsg
        )}`,
        error
      );
      throw error; // This will route the message to the deadletter topic
    }
  },
};

// Initialize and start the consumer (configure according to your Kafka setup)
// ...

Star History

Contributing

We welcome contributions! Here's how you can contribute:

1. Fork the Repository: Create your own fork of the project.
2. Create a Feature Branch: Work on new features or bug fixes in your own branch.
3. Commit Your Changes: Make sure your changes are well-documented and tested.
4. Submit a Pull Request: Submit your changes for review.

For a detailed guide on contributing to projects on GitHub, please refer to the GitHub contribution guide.

Support

License

MIT License. See LICENSE.

---

For more details and advanced usage, visit the GitHub repository.