Kiba is a lightweight Ruby ETL framework (Extract - Transform - Load) framework. It makes it easy to write reliable, concise, well-tested & maintainable data-processing code in Ruby.
Kiba development & maintenance is funded by LoGeek, a French data-oriented consultancy.
Read for more info:
- Core Concepts
- How to define ETL jobs with Kiba
- How to run ETL jobs with Kiba
- Implementing ETL sources
- Implementing ETL transforms
- Implementing ETL destinations
- Implementing pre and post-processors
- Implementation Guidelines
- How to extend the Kiba DSL
- Can Kiba handle multiple sources and destinations?
- Ruby 2.7+ keyword arguments compatibility note
Kiba Common is an OSS companion gem to Kiba, bringing commonly used features to Kiba:
- Enumerable Source (turn any enumerable into a stream of rows)
- Enumerable Exploder (explodes an enumerable row into a sequence of output rows)
- Source Transform Adapter (dynamical sources using input rows as instantiation parameters)
- CSV Source
- CSV Destination
- Lambda Destination
- How to add logging to Kiba?
- How to pretty-color-print rows in Kiba?
Kiba Pro adds powerful vendor-backed extensions for Kiba ETL. Purchasing a Kiba Pro subscription (via https://www.logeek.fr) also supports the continued maintenance & development of Kiba Open-Source.
Purchasing, installation, misc
Roadmap (not in a specific order):
- SQL Bulk Lookup
- Generic Paginating HTTP source
- Parallel Transform
- Data profiling tools
- S3 components
- CPU & memory profiler