BREAKING - the
kiba CLI is deprecated
The programmatic API allows everything the "command" mode supported, plus much more, and actually encourage better coding practices. For instance:
- API mode allows to pass live variables (rather than just ENV configuration from command line or JSON configs from files)
- Doing so permits to wrap resources open/close around running a job
- API mode makes it easier to run testing on an ETL process (via minitest/rspec) directly in-process (which allows stubbing/webmock etc), rather than via a command call
- API mode enforces use of clean modules with explicit loading, rather than polluting the top-level namespace with global methods
- API mode allows to run jobs from Sidekiq or background job systems, from an HTTP call (if the job is fast), without necessarily waiting for a command line binary to run - this supports more dynamic interactions (e.g. a job is created in reaction to an external event received via HTTP or a websocket)
kiba-legacy-cli gem is available (https://github.com/thbar/kiba-legacy-cli) to ease migration, but the recommendation is really to migrate over and use
Kiba.parse directly, as described in the current documentation.
Kiba now defaults to
Introduced in v2.0.0 [#44] to ensure a transform could yield N rows for 1 input row, and improved in v2.5.0 [#57] to help implement "buffering transforms", the
StreamingRunner is now made the default to process the jobs [#83].
This change is expected to be backward compatible and will help with reusability & features of ETL components.
Ruby compatibility notice
- Kiba now officially supports MRI Ruby 2.4+ (although 2.3 will still work for now), JRuby 9.2+ or TruffleRuby.
- You may get warnings with Ruby 2.7 and errors with Ruby 2.8+. See [#85] for status on Ruby 3 keyword arguments support.