Roadmap

We would like to ease the adoption of the Versatile Data Kit project in various use cases and to that end we are going to implement the following features/enhancements:

• Ease of use improvements - since the project is in very early stage we would like to address any community feedback on the ease of use and operation of the project. This includes various small usability improvements as well as extending the existing documentation and examples.
• Workflow engine integration - in order to provide the ability to execute multiple data jobs sequentially and to provide better retry and notification capabilities we want to provide an integration with an existing workflow engine/management system like Apache Airflow
• Data Lineage tracking capabilities - we want to provide the ability to collect data lineage metainformation during the execution of data jobs. This capability will facilitate the debugging and troubleshooting of Data Jobs as well as provide better data governance and change management.

• Support for more Data Base engines - we would like to extend the range or supported DB engines by providing connectors for the more popular DBs like Postgres, ????