Ingestion

A typical Ingestion job:

• Extracts data from various sources** (HTTP APIs, Databases, CSV, etc.).
• Does NOT do any transformations on the data (besides formating the payload to be accepted by target (e.g json serialization)).
• Loads the data to your preferred Ingestion target (database, cloud storage)

Ingestion examples:

➡ Ingesting data from REST API into Database
➡ Ingesting data from DB into Database
➡ Ingesting local CSV file into Database
➡ Incremental ingestion using Job Properties
➡ Ingesting data from an authenticated REST API using Secrets

Jupyter Ingestion Tutorial

➡ VDK Ingestion Tutorial with Jupyter Notebooks

Ingestion Videos

▶️ Data Ingestion Intro
▶️ Incremental Ingestion

All VDK Examples

All VDK examples can be found here

➡️ Next section: Transformation