Quickstart VDK

Versatile Data Kit SDK (VDK) is the go to tool for developing and running Data Jobs locally. It's lego-like extensibility allows for creating different distributions with different plugins installed providing different functionalities.

But if you are just starting, Quickstart VDK is a good distribution to use for getting started with VDK.

Prerequisites

Versatile Data Kit CLI requires Python 3.7+. If you're new to Python, we recommend Anaconda.

It's recommended to have the latest version of pip :

pip install -U pip setuptools wheel

Installation

Main README.md guide

pip install quickstart-vdk

This will install VDK with support for some common databases and job lifecycle management operations.

To upgrade an already existing installation of quickstart-vdk, use:

pip install --upgrade --upgrade-strategy eager quickstart-vdk

See help to see what you can do:

vdk --help

Check out Getting Started to create your first Data Job and the Examples for the various things you can do with Versatile Data Kit.

Install VDK Plugins

VDK comes with a number of plugins that can be installed to change or extend its behavior.

pip install <vdk-plugin-name>

Once the plugin is installed, vdk automatically finds it and activates it. Check the plugin documentation for options. Run vdk config-help for a list of all configuration settings of all plugins.

You can find a list of plugins that we have already developed in plugins directory.

See the plugins README for more info.