This is a patched version of `kafkajs` to improve CPU utilization on idle consumers

This PR has been applied on top of version 2.2.4 of kafkajs: tulios#1667

Repo state from which this package was published: https://github.com/sergeyevstifeev/kafkajs.git

The rest of the library has been kept intact.

KafkaJS

A modern Apache Kafka® client for Node.js
Get Started »

Read the Docs · Report Bug · Request Feature

About the Project

KafkaJS is a modern Apache Kafka client for Node.js. It is compatible with Kafka 0.10+ and offers native support for 0.11 features.

KAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use by KafkaJS. KafkaJS has no affiliation with and is not endorsed by The Apache Software Foundation.

Sponsors ❤️

Upstash: Serverless Kafka

True Serverless Kafka with per-request-pricing
Managed Apache Kafka, works with all Kafka clients
Built-in REST API designed for serverless and edge functions
Start for free in 30 seconds!

Get help directly from a KafkaJS developer

Become a Github Sponsor to have a video call with a KafkaJS developer
Receive personalized support, validate ideas or accelerate your learning
Save time and get productive sooner, while supporting KafkaJS!
See support options!

To become a sponsor, reach out in our Slack community to get in touch with one of the maintainers. Also consider becoming a Github Sponsor by following any of the links under "Sponsor this project" in the sidebar.

Features

Producer
Consumer groups with pause, resume, and seek
Transactional support for producers and consumers
Message headers
GZIP compression
- Snappy, LZ4 and ZSTD compression through pluggable codecs
Plain, SSL and SASL_SSL implementations
Support for SCRAM-SHA-256 and SCRAM-SHA-512
Support for AWS IAM authentication
Admin client

Getting Started

npm install kafkajs
# yarn add kafkajs

Usage

const { Kafka } = require('kafkajs')

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['kafka1:9092', 'kafka2:9092']
})

const producer = kafka.producer()
const consumer = kafka.consumer({ groupId: 'test-group' })

const run = async () => {
  // Producing
  await producer.connect()
  await producer.send({
    topic: 'test-topic',
    messages: [
      { value: 'Hello KafkaJS user!' },
    ],
  })

  // Consuming
  await consumer.connect()
  await consumer.subscribe({ topic: 'test-topic', fromBeginning: true })

  await consumer.run({
    eachMessage: async ({ topic, partition, message }) => {
      console.log({
        partition,
        offset: message.offset,
        value: message.value.toString(),
      })
    },
  })
}

run().catch(console.error)

Learn more about using KafkaJS on the official site!

Read something on the website that didn't work with the latest stable version?
Check the pre-release versions - the website is updated on every merge to master.

Contributing

KafkaJS is an open-source project where development takes place in the open on GitHub. Although the project is maintained by a small group of dedicated volunteers, we are grateful to the community for bug fixes, feature development and other contributions.

See Developing KafkaJS for information on how to run and develop KafkaJS.

Help wanted 🤝

We welcome contributions to KafkaJS, but we also want to see a thriving third-party ecosystem. If you would like to create an open-source project that builds on top of KafkaJS, please get in touch and we'd be happy to provide feedback and support.

Here are some projects that we would like to build, but haven't yet been able to prioritize:

Dead Letter Queue - Automatically reprocess messages
✅ Schema Registry - Now available! thanks to @erikengervall
Metrics - Integrate with the instrumentation events to expose commonly used metrics

Contact 💬

Join our Slack community

License

See LICENSE for more details.

Acknowledgements

Thanks to Sebastian Norde for the V1 logo ❤️
Thanks to Tracy (Tan Yun) for the V2 logo ❤️

Apache Kafka and Kafka are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries. KafkaJS has no affiliation with the Apache Software Foundation.

License

sergeyevstifeev/kafkajs

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Languages

Error
Looks like something went wrong!

Packages