Message Schema

This repository contains the protobuf schema definitions for platform wide messages. This is useful for platforms that communicate using a defined message schema over the wire and need a harmonized common way of handling message communication, in this case Protobuf. This could also be used to send messages to message brokers and queues that know how to serialize & deserialize protocol buffer message types, for example Kafka.

This has been built to generate code that can be used in projects of different languages, making it language agnostic. Currently, the code generation is done using buf and the languages that are generated are:

1. Python
2. Kotlin
3. Golang
4. JavaScript

The schemas themselves are published to a buf registry making it easer to use in languages that have no code generates set here yet.

Additionally, packages for the respective languages are generated and published to respective repositories or package registries.

For Python, this is PyPI and this can be found here. Kotlin can be found here, JavaScript can be found here & here.

Note that the same packages are also published to Gitlab Package Registry which follow the same versioning format.

Pre-Requisites

In order to setup the project, you will need a couple of things first setup locally:

• Buf to handle code generation
• make for running convenience scripts in Makefiles.
• Java
• Gradle Build tool to generate builds for JVM based projects, in this case Kotlin in kotlin
• Python to handle Python generated code in python
• NPM to handle package management and installation of dependencies for the JS based project in js
• Go for the generated go code in go

Suggested Tools/Plugins

• (If using IntelliJ) Install the buf plugin.

Quick Start

• Install all Pre-Requisites.
• Run pre-commit install to install the pre-commit hooks into your locally checked out repository.
• Run make help to see a list of commands and what they do.

Recommended Reading

• We follow The Official Buf Style Guide.
• We make use of the protoc-gen-validate plugin for validation rules.
• More documentation can be found here

Releases

Releases are created by a Gitlab pipeline when version tags are added to the repo. This is all automated and no manual intervention is needed other than to trigger a build manually. The release versions are created based of the commit messages using tool commitlint, cz-cli & semantic-release.

Note that there are 2 set of pipelines that run:

1. GitLab Pipelines
2. Github Actions

GitLab Pipelines runs lint, build and generates the buf schemas and publishes them to the buf registry also publishes the language packages to GitLab Package Registry which is private for now. Github Actions handles the same processes but is triggered by Gitlab Pipelines & will publish the package to Github Package Registry for Maven & NPM & to NPM & PyPI also triggering a Github release. These are public.

Github & Bitbucket act as mirror repositories with the former for public use while GitLab is for private use.