Data Mesh Manager Asset Bundle Template For Databricks

This repository essentially is a copy of the default Databricks Asset Bundle template with some additions to automatically synchronize your Databricks asset bundles as dataproducts to Data Mesh Manager.

For more insight on Databricks Asset Bundles read the documentation at https://docs.databricks.com/dev-tools/bundles/index.html.

Particularities

In addition to the files that are generated by the default template, the following files are added to your project:

• datacontract.yml: The interface definition for your data product. It is pushed to the Data Mesh Manager.
• dataproduct.yml: The internal definition of your data product. It is pushed to the Data Mesh Manager.
• src/datamesh_manager/main.py + src/datamesh_manager/main.py: The Python code that pushes your data product and data contract definitions to Data Mesh Manager.
• resources/datamesh_manager_job.yml: A job definition to execute the Data Mesh Manager related Python code.

The setup.py is modified to add an entry point for the Data Mesh Manager related Python code to the generated Python wheel.

Usage

To use this template, simply run

databricks bundle init https://github.com/datamesh-manager/databricks-bundle-python-template

Licenses

This project is distributed under the DB (Databricks) License. It includes various open-source dependencies, each governed by its respective license.

For more details, please refer to the LICENSE file.