Skip to content

Latest commit

 

History

History
74 lines (63 loc) · 3.87 KB

README.md

File metadata and controls

74 lines (63 loc) · 3.87 KB
Raw

dbt & VS Code Dev Containers

A template for dbt-Core projects inside of docker, using poetry to manage dependencies and VS Code Dev Containers for local development.

Table of Contents

Overview

VS Code Dev Containers provide an immediate development environment for teams to manage their dbt-Core project.

A dockerized dbt environment allows for a consistent runtime across contributors and orchestration tools: the versions, packages, scripts and behavior of a dbt-Core project are identical whether the project is run locally or deployed.

Examples of possible changes to the environment are:

  • package versions controlled with poetry
  • VS Code extensions (versions, included extensions)
  • other scripts (eg. CI scripts that can be run locally)
  • CLI tools like aws, git, etc

These changes can be made through changes to the relevant files. Simply rebuilding the container will allow every team member and tool to use the new environment.

Getting Started

Complete the following steps to launch the Dev Container in VS Code:

  1. Download Docker Desktop.
  2. Download VS Code.
  3. Install the VS Code Dev Containers Extension.
  4. Open the project in VS Code and click "Repoen in Container" when prompted.

Configuring the Template

The following features of this template repository can be configured to specific use cases:

  • The dbt project can be replaced with an existing project.
    • Note that dbt/profiles.yml should be retained within the container.
      • Typically dbt uses profiles from ~/.dbt/profiles.yml.
      • However this dbt project is designed to be run both locally and by pipelines/orchestration tools.
      • Thus, the profiles need to be included in the container.
      • The connection secrets in the profile should be provided to the container via the environment.
  • Environment variables should be passed to the development container for local development via an .env file.
    • Any local .env files are ignored by git.
    • See .example.env for an example of passing connection details.
  • Use poetry to install and manage new dependencies in the container.
  • The scripts/ folder can be used to house build and pipeline scripts.
  • The devcontainer.json contains recommended VS Code extensions for development.
    • These are cached by default in a docker volume for fast rebuilds of the Dev Container.
    • The SQLTools extension is setup with a Snowflake connection by default.
      • The connection uses params in .example.env which are injected by the postAttachCommand in the devcontainer.json.
      • SQLTools connections allow viewing and querying data in the warehouse directly from the VS Code Dev Container.
      • dbt models can be created in the warehouse and the data viewed from within VS Code to accelerate development.

Contributions

Please reach out and start a discussion if you are interested in enhancing or altering this template.

You can find the author in the dbt Slack or simply open an issue here. Cheers!