Converts avro dml to abinitio specific dml
-
Updated
Jan 15, 2019 - Python
Converts avro dml to abinitio specific dml
The project showcases a data pipeline integrating Kafka messaging and MongoDB for efficient logistics data ingestion. It uses Python for Kafka producer/consumer operations and MongoDB for data storage.
Avro Schema v1.8.2 (https://avro.apache.org/docs/1.8.2/spec.html) formatting with marshmallow - WIP
The project showcases a robust real-time data pipeline integrating Confluent Kafka, MySQL, and Avro serialization to facilitate seamless and immediate processing of e-commerce updates. The pipeline enables efficient streaming, transformation, and storage of incremental data updates for downstream analytics and business intelligence.
BigQuery ingestion POC using PubSub messages with different AVRO schemas and Apache Beam
Generate Apache Avro schemas for Pydantic data models.
CLI tool to generate HTML documentation for an Apache Avro schema
Benchmarking avro libraries in Python 3
Visualize your raw data from .avro files for the EmbracePlus device from Empatica
Package defining schema relevant to astronomy and astrophysics data, providing useful interfaces for interaction with those schema.
An automated and extensible space-efficiency benchmark of JSON-compatible serialization specifications
Async Python Rest Client to interact against Schema Registry confluent server
A Python ORM for Avro Schemas
☀️ Avro, Protobuf, Thrift on Swagger
A pure python avro schema validator
Avrotize is a command-line tool for converting data structure definitions between different schema formats, using Apache Avro Schema as the integration schema model.
Add a description, image, and links to the avro-schema topic page so that developers can more easily learn about it.
To associate your repository with the avro-schema topic, visit your repo's landing page and select "manage topics."