Installation

• Install SDK
  • Prerequisites
  • Installation
  • Optional features
• Install control service server
  • Prerequisites
  • Install Versatile Data Kit Control Service
  • Use

Install SDK

Versatile Data Kit SDK (vdk) is the go to tool for developing and running data jobs locally.

Prerequisites

Versatile Data Kit CLI (VDK) requires Python 3.7+. If you're new to Python, we recommend Anaconda.

Installation

pip install quickstart-vdk

This will install VDK with support for sqlite and trino databases and job lifecycle management operations.

See help to see what you can do:

vdk --help

Optional features

VDK comes with number plugins that can be installed to change or extend behaviour of vdk.

pip install vdk-plugin-name

See plugins page for more info.

Install control service server

Versatile Data Kit includes Control Service server enabling deploying, managing and monitoring data jobs. It is installed using helm.

Prerequisites

• Install helm
• Install kubernetes distribution. If you're new to kubernetes, we recommend minikube
• Install docker

Install Versatile Data Kit Control Service

helm repo add vdk-gitlab https://gitlab.com/api/v4/projects/28814611/packages/helm/stable
helm install my-release vdk-gitlab/pipelines-control-service 

Read more about the installation in the Versatile Data Kit Control Service Chart Github repository

Use

# see Help to see what you can do and start playing around
vdk --help 
# for example create a new job:
vdk create -u <URI-YOU-GOT-FROM-helm-install-OUTPUT>