Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


A PoC that defines single-message topic schemas for Apache Kafka using Protocol Buffers (Protobuf) and Scala.

The goal for this PoC is to evaluate the usage of a single envelope protobuf definition per kafka topic. The main reason behind that is having a clear topic schema and clear expectations for consumers of these messages.

I've written a blog post based on this PoC which explains this concept in a bit more details. The blog post can be found here

P.S. Feel also free to use this PoC as a quick start project using Scala, Kafka and Protobuf.


The idea is to have an envelope per topic defined in protobuf. We can add any header fields we need (e.g. correlation id) but the payload must use protobuf's oneof to list all the message types are sent on that topic.


In the users.proto file we have got a basic protobuf definitions for the users topic:

// protobuf file headers ..

message UsersEnvelope {
    string correlation_id = 1;
    // some other metadata
    oneof payload {
        UserCreated user_created = 11;
        UserUpdated user_updated = 12;
        UserActivated user_activated = 13;

// the rest of the messages ..

Each protobuf compiler will generate these classes in its own way along with an enum or some class hierarchy for the oneof cases (depending on the language). Usually it will also generate a special Empty case (in Scala, Java and C# at least, the name varies per language) to handle invalid or missing messages (cases).

Tech stack


This already assumes you have clonned this repository and installed sbt and docker.

  • First, we need to start Kafka. This repository has a docker-compose file provided. All we need to do is to run:
docker-compose -f docker/docker-compose.yml up -d 
  • To start our app:
sbt "runMain com.tudorzgureanu.Main"  
  • Now we are ready to send some messages to kafka. For that we would use our KafkaMessageSender utility class. Feel free to change this class locally to send any messages you prefer. Finally, simply run:
sbt "runMain com.tudorzgureanu.KafkaMessageSender"


PoC using scala that defines single-message protobuf schema per Kafka topic




No releases published


No packages published