.NET Stream Processing Library for Apache Kafka ^TM · ·

Quality Statistics

Project Statistics

---

Streamiz Kafka .NET is .NET stream processing library for Apache Kafka.

KAFKA is a registered trademark of The Apache Software Foundation and
has been licensed for use by Streamiz. Streamiz has no
affiliation with and is not endorsed by The Apache Software Foundation.

It's allowed to develop .NET applications that transform input Kafka topics into output Kafka topics. It's supported .NET Standard 2.1.

It's a rewriting inspired by Kafka Streams. Finally it will provide the same functionality as Kafka Streams.

This project is being written. Thanks for you contribution !

ROADMAP

• 1.2.0 - Persistent state store (eg: RocksDB Store), Repartition and Changelog topics
• 1.3.0 - Processor API, Metrics
• 1.4.0 - Standby Replica
• 1.5.0 - Interactive Queries

Documentation

Read the full documentation on https://lgouellec.github.io/kafka-streams-dotnet/

Installation

Nuget packages are listed to nuget.org

Install the last version with :

dotnet add package Streamiz.Kafka.Net

Usage

There, a sample streamiz application :

static async System.Threading.Tasks.Task Main(string[] args)
{ 
    var config = new StreamConfig<StringSerDes, StringSerDes>();
    config.ApplicationId = "test-app";
    config.BootstrapServers = "192.168.56.1:9092";
    
    StreamBuilder builder = new StreamBuilder();

    var kstream = builder.Stream<string, string>("stream");
    var ktable = builder.Table("table", InMemory<string, string>.As("table-store"));

    kstream.Join(ktable, (v, v1) => $"{v}-{v1}")
           .To("join-topic");

    Topology t = builder.Build();
    KafkaStream stream = new KafkaStream(t, config);

    Console.CancelKeyPress += (o, e) => {
        stream.Dispose();
    };

    await stream.StartAsync();
}

TODO implementation

• Transform and Through Processor
• Consumer Incremental Rebalance Protocol #KIP-429
• Supress Processor (.suppress(Suppressed.untilWindowCloses(Suppressed.BufferConfig.unbounded())))
• Repartition topic
• Repartition Processor KAFKA-8611 | PR #7170
• Processor API
• Sample projects (Micro-services, console sample, topology implementation, etc ..) which use Streamiz package (see)
• Json SerDes which interact Confluent Schema Registry
• Optimizing Kafka Streams Topologies
• Standby Replica
• Interactive Queries
• Metrics

Contributing

Owners:

• lgouellec

Maintainers:

• lgouellec

Streamiz Kafka .Net is a community project. We invite your participation through issues and pull requests! You can peruse the contributing guidelines.

When adding or changing a service please add tests and documentations.

Support

You can found support here