Generate avro schemas from python dataclasses, Pydantic models and Faust Records. Code generation from avro schemas. Serialize/Deserialize python instances with avro schemas.

Apache Avro <-> pandas DataFrame

A pure python avro schema validator

Samples related to data engineering, e.g. spark, embulk, airflow, etc.

Simulate alert streams for the Fink broker

Construct a streaming event pipeline around Apache Kafka and its ecosystem. Using public data from the Chicago Transit Authority, we will construct an event pipeline around Kafka that allows us to simulate and display the status of train lines in real time.

Make avro-tools available via a Docker container.