Examples

Examples use cases solved with Versatile Data Kit:

Data Engineering

• Hello World Data Job

• Ingesting data from REST API into Database

• Ingesting data from DB into Database

• Ingesting data from S3 into Database

• Ingesting local CSV file into Database

• Processing data using SQL and local database

• Scheduling a Data Job for automatic execution

• Processing data using VDK templates and a Big Data database (Presto/Trino)

• Incremental ingestion using Job Properties

• Troubleshoot Data Job bugs with data and code lineage (not ready - todo)

• Integration with Airflow (not ready - todo)

• Ingest, process, and publish using VDK & Streamlit Vincent Van Gogh's paintings

• Ingest, process, and publish using VDK & Streamlit American states life expectancy

• Ingest, process, and publish using VDK & Streamlit analysis of relationship between Natural Gas prices and the temperature

• Debug Data Job in IDE

The source code for the examples can found in git at https://github.com/vmware/versatile-data-kit/tree/main/examples