---

dbt-π 🧬

Python wrapper for dbt-core to extend dbt with custom Python.

Shimmy shimmy shim 🕺🕺🕺

This package is a shim for dbt-core, inspired by (cough stolen from cough) my old boss, @darkdreamingdan:

• https://gist.github.com/darkdreamingdan/c5ded709a90fc3c5b420cee5f644f499

Before using this package, it's recommended to get up to speed with the Python modules that are already available in dbt:

• https://docs.getdbt.com/reference/dbt-jinja-functions/modules

The existing Python modules are available in the dbt Jinja context under the modules object, for example:

{{ modules.datetime.datetime.now() }}

Installation ⬇️

While in preview, this package is only available from GitHub:

pip install git+https://github.com/billwallis/dbt-py@v0.0.4

This will be made available on PyPI once it's ready for general use.

Usage 📖

This package adds a new executable, dbt-py, which injects your custom Python into dbt and then runs dbt. Either a custom module or a custom package can be injected. A custom module is the simplest to get started with.

The default module/package name is custom which would make custom Python available in the dbt Jinja context under the modules.custom object. This can be configured (see the Configuration section below).

Important

If you create the Python files in your dbt repo, you must install your own project as a package for your files to be found and imported by dbt.

This is typically achieved with pip install -e .; please see the pip docs for more details:

• https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs

If you use a package manager such as Poetry or uv, they should automatically install your project provided you configure the project metadata correctly.

See the following repo for a minimal example that uses requirements.txt and setuptools:

• https://github.com/billwallis/dbt-py-test

Custom Module 🐍

Create a module called custom.py in the root of your dbt project. This module can contain any Python code you like, for example:

def salutation(name: str) -> str:
    return f"Hello, {name}!"

Reference this module and function in the dbt Jinja context of a dbt model:

{{ modules.custom.salutation("World") }}

Rather than run dbt with the dbt command, instead run it with dbt-py:

dbt-py clean
dbt-py build

Note that dbt-py is a wrapper around dbt so all the usual dbt commands are available -- all the arguments passed to dbt-py are passed through to dbt, too.

dbt-py --help
dbt-py run --select my_model
dbt-py test --select tag:unit-test

Custom Package 📦

Using a custom package is similar to using a custom module: create a package called custom in the root of your dbt project.

The submodules of this package will be available in the dbt Jinja context too. For example, suppose you have a package called custom with a submodule called greetings:

custom/
    __init__.py
    greetings.py

If the greetings.py submodule contains the same salutation function as above, then it can be referenced in the dbt Jinja context as follows:

{{ modules.custom.greetings.salutation("World") }}

Alternatively, you can expose the salutation function via the __init__.py file and then reference it directly via custom:

{{ modules.custom.salutation("World") }}

Configuration 🛠️

The default module/package and Jinja context name is custom but both can be configured with the following environment variables:

• DBT_PY_PACKAGE_ROOT: The Python-style ref to the custom module/package, e.g. package.module.submodule
• DBT_PY_PACKAGE_NAME: The name to give the custom module/package in the dbt Jinja context, e.g. custom_py. Defaults to the value of DBT_PY_PACKAGE_ROOT

In particular, you can use the DBT_PY_PACKAGE_ROOT environment variable to reference a custom module/package that is not at the root of your dbt project.

Warning

If you set the DBT_PY_PACKAGE_ROOT environment variable to a name that already exists, this package will use the existing module/package rather than your custom one. Make sure that your custom module/package name does not clash with any existing modules/packages.

This is likely to change in a future release, but for now you may choose to exploit this behaviour to use an existing module/package in your dbt Jinja context. For example, you could set DBT_PY_PACKAGE_ROOT to math and then reference the math standard library in your dbt Jinja context:

{{ modules.math.pi }}

Future Work 🚧

This is still in preview, and there are a few things to be added before it's ready for general use:

• Support for importing any number of packages (currently only one package is supported)
• Configuration via config files and CLI arguments (currently only environment variables are supported)
• More robust testing

Contributing 🤝

Raise an issue, or fork the repo and open a pull request.

This project uses uv and pre-commit. After cloning the repo, install the dependencies and enable pre-commit:

uv sync --all-groups
pre-commit install --install-hooks