fabric-utils (demo)

Demo repository for a dbt notebook pattern in Microsoft Fabric plus Azure DevOps CI/CD templates for validating dbt changes and deploying Fabric workspace items.

Note: This repo contains example values in code (workspace names, URLs, etc.). The documentation below stays public-safe and uses placeholders.

What’s included

• Fabric workspace item sources under fabric-workspaces/
  • Demo notebook: fabric-workspaces/data-and-analytics/dbt.Notebook/notebook-content.py
• Azure DevOps pipelines and deployment script under .devops/
  • PR validation: .devops/dbt-ci-validation.yml
  • Releases: .devops/release-staging.yml, .devops/release-production.yml
  • Fabric deploy helper: .devops/deploy-fabric-resources.py
• Environment-specific replacement config (for Fabric item publishing): parameter.yml

How the notebook works

The notebook in notebook-content.py demonstrates a common “orchestrate dbt from Fabric” approach:

1. Installs dbt adapter (dbt-fabric) (see the pip install cell in notebook-content.py).
2. Retrieves runtime secrets using your preferred mechanism (the demo uses notebookutils.credentials.getSecret(...)).
3. Clones a git repo at a chosen ref (git_tag) (see clone logic in notebook-content.py).
4. Runs dbt steps via the Python CLI runner (deps, debug, build) (see _run_dbt_step()).
5. Generates static dbt docs and copies artifacts (docs, manifest.json, run_results.json) to the lakehouse file system (see docs + copy in notebook-content.py).

CI/CD templates

Pull request validation (dbt CI)

.devops/dbt-ci-validation.yml is a PR-oriented pipeline that:

• Detects changed dbt model SQL files.
• Installs Python dependencies using uv.
• Ensures dbt connectivity/syntax (dbt debug, dbt deps, dbt parse).
• Runs sqlfluff lint against the changed files.

Fabric releases (staging + production)

• .devops/release-staging.yml deploys on updates to main (path-filtered to fabric-workspaces/*).
• .devops/release-production.yml deploys on tags matching v*.

Both call .devops/deploy-fabric-resources.py, which uses the fabric-cicd library to publish workspace items (see publish_all_items()).

Environment replacement (parameter.yml)

parameter.yml defines find/replace rules applied during publishing to make the same Fabric item sources deployable across environments.

In this demo it includes:

• Regex replacements for Fabric notebook metadata IDs (default lakehouse + workspace IDs) (see rules in parameter.yml).
• An environment-specific switch for is_prod (see parameter.yml).
• A regex replacement for git_tag so STG/PROD can deploy from an environment-provided ref (see parameter.yml).

Quickstart (template-level)

1. Fabric notebook

   • Import/open fabric-workspaces/data-and-analytics/dbt.Notebook/notebook-content.py in Fabric.
   • Replace the demo secret retrieval + repo clone pieces with your own approach (Managed Identity, Key Vault, or workspace secrets).

2. Azure DevOps

   • Create a pipeline from .devops/dbt-ci-validation.yml for PR validation.
   • Create release pipelines from .devops/release-staging.yml and .devops/release-production.yml.
   • Configure an Azure service connection suitable for Fabric deployments (used by the AzureCLI@2 steps).
   • Update workspace names and repository directories in the YAML parameters to match your repo.

Repo layout

.
├── .devops/                         # Azure DevOps pipeline templates + deploy script
├── fabric-workspaces/               # Fabric workspace item sources (Notebook, etc.)
├── parameter.yml                    # Environment replacement rules for publishing
└── LICENSE

License

MIT — see LICENSE.