Lineage bundle

This package is distributed under the "DataSentics SW packages Terms of Use." See license

Lineage bundle allows you to generate and publish lineage of notebooks and notebook functions of your Daipe project.

Getting started

Add lineage-bundle to your project

poetry add lineage-bundle --dev

Generate and publish lineage to a static HTML

console lineage:publish:html

Optional:

To use a databaase: add sqlalchemybundle.yaml file to [PROJECT_NAME]/_config/bundles/

parameters:
  sqlalchemybundle:
    connections:
      default:
        engine: mssql
        server: '%env(DB_HOST)%'
        database: '%env(DB_NAME)%'
        username: '%env(DB_USER)%'
        password: '%env(DB_PASS)%'
        driver: '{ODBC Driver 17 for SQL Server}'

In .env in your Daipe project

APP_ENV=dev

# Databricks
DBX_TOKEN=
# Lineage
DB_HOST=address.of.mssql.server.com
DB_NAME=db_name
DB_USER=username
DB_PASS=password

Initialize the database

console lineage:database:init

Generate and publish lineage to database

console lineage:publish:database

Preview

Pipelines lineage

Functions lineage