Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
romyli committed Nov 20, 2023
0 parents commit 90d9eee
Show file tree
Hide file tree
Showing 56 changed files with 157,229 additions and 0 deletions.
23 changes: 23 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "Python 3",
"image": "mcr.microsoft.com/devcontainers/python:0-3.9-bullseye",
"features": {
"ghcr.io/devcontainers-contrib/features/black:2.0.9": {},
"ghcr.io/meaningful-ooo/devcontainer-features/homebrew:2": {},
"ghcr.io/devcontainers/features/node:1": {},
"ghcr.io/eitsupi/devcontainer-features/go-task:1": {}
},
"postCreateCommand": "task install",
"customizations": {
"vscode": {
"settings": {
"terminal.integrated.defaultProfile.linux": "zsh"
},
"extensions": [
"dorzey.vscode-sqlfluff",
"esbenp.prettier-vscode",
"GitHub.codespaces"
]
}
}
}
Binary file added .github/static/codespaces-setup-screen.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/static/open-codespace.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/static/use-template.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 15 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
.venv
venv
.env
env

target/
dbt_packages/
logs/

.DS_Store

*.duckdb
*.duckdb-wal
*.db
*.db-wal
26 changes: 26 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- id: requirements-txt-fixer
- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.0.287
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- repo: https://github.com/sqlfluff/sqlfluff
rev: "2.3.1"
hooks:
- id: sqlfluff-lint
additional_dependencies:
["dbt-metricflow[snowflake]~=0.2.0", "sqlfluff-templater-dbt~=2.3.1"]
- id: sqlfluff-fix
additional_dependencies:
["dbt-metricflow[snowflake]~=0.2.0", "sqlfluff-templater-dbt~=2.3.1"]
- repo: https://github.com/psf/black
rev: "23.7.0"
hooks:
- id: black
37 changes: 37 additions & 0 deletions .sqlfluff
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
[sqlfluff]
dialect = snowflake
templater = dbt
runaway_limit = 10
max_line_length = 80
indent_unit = space

[sqlfluff:indentation]
tab_space_size = 4

[sqlfluff:layout:type:comma]
spacing_before = touch
line_position = trailing

[sqlfluff:rules:capitalisation.keywords]
capitalisation_policy = lower

[sqlfluff:rules:aliasing.table]
aliasing = explicit

[sqlfluff:rules:aliasing.column]
aliasing = explicit

[sqlfluff:rules:aliasing.expression]
allow_scalar = False

[sqlfluff:rules:capitalisation.identifiers]
extended_capitalisation_policy = lower

[sqlfluff:rules:capitalisation.functions]
capitalisation_policy = lower

[sqlfluff:rules:capitalisation.literals]
capitalisation_policy = lower

[sqlfluff:rules:ambiguous.column_references] # Number in group by
group_by_and_order_by_style = implicit
4 changes: 4 additions & 0 deletions .sqlfluffignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
target
dbt_packages
macros
.venv
1 change: 1 addition & 0 deletions .user.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
id: eb151271-6f32-46f6-9900-c28e734cd8ac
18 changes: 18 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"recommendations": [
"GitHub.vscode-pull-request-github",
"cschleiden.vscode-github-actions",
"ms-python.python",
"ms-python.vscode-pylance",
"ms-python.black-formatter",
"ms-python.isort",
"charliermarsh.ruff",
"redhat.vscode-yaml",
"task.vscode-task",
"samuelcolvin.jinjahtml",
"bungcip.better-toml",
"tamasfe.even-better-toml",
"mechatroner.rainbow-csv",
"bastienboutonnet.vscode-dbt"
]
}
9 changes: 9 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"python.analysis.typeCheckingMode": "basic",
"sqlfluff.dialect": "snowflake",
"sqlfluff.experimental.format.executeInTerminal": true,
"[jinja-sql]": {
"editor.defaultFormatter": "dorzey.vscode-sqlfluff",
"editor.formatOnSave": false
}
}
74 changes: 74 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# 🥪 The Jaffle Shop 🦘

This is a sandbox project for exploring the basic functionality and latest features of dbt. It's based on a fictional restaurant called the Jaffle Shop that serves [jaffles](https://en.wikipedia.org/wiki/Pie_iron). Enjoy!

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/dbt-labs/jaffle-shop?quickstart=1)
[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/dbt-labs/jaffle-shop)

## Create new repo from template

1. <details>
<summary>Click the green "Use this template" button at the top of the page to create a new repository from this template.</summary>

![Click 'Use this template'](/.github/static/use-template.gif)
</details>
3. Follow the steps to create a new repository.

## Platform setup

### dbt Cloud IDE (most beginner friendly)
1. Set up a dbt Cloud account and follow Step 4 in the [Quickstart instructions for your data platform](https://docs.getdbt.com/quickstarts), to connect your platform to dbt Cloud.
2. Choose the repo you created in Step 1 as the repository for your dbt Project code.
3. Click `Develop` in the top nav, you should be prompted to run a `dbt deps`, which you should do.

### dbt Cloud in GitHub Codespaces (more customizable)

1. <details>
<summary>In the new repository, click the green "Code" button and select "Open with Codespaces" from the dropdown. If possible, open in VSCode locally rather than the web version, performance is significantly better.</summary>

![Create codespace on main](.github/static/open-codespace.gif)
</details>
2. Install the recommend extensions when prompted unless you have set preferences here.
3. Run `task install-cloud`[^1] in the integrated terminal.

### dbt Core with DuckDB in GitHub Codespaces (think local, act global)

1. <details>
<summary>In the new repository, click the green "Code" button and select "Open with Codespaces" from the dropdown. If possible, open in VSCode locally rather than the web version, performance is significantly better.</summary>

![Create codespace on main](.github/static/open-codespace.gif)
</details>
2. Install the recommend extensions when prompted unless you have set preferences here.
3. Run `task install-core`[^2] in the integrated terminal.

### dbt Core with other platforms (choose your own adventure)

If you know what you're doing, you can use this repo with any local or cloud database with a dbt adapter. We can't offer support for this setup, but the general steps should be as follows:

1. Clone the new repository to your local machine or open it in a GitHub Codespace.
2. Run `task venv`.[^3]
3. Run `source .venv/bin/activate && exec $SHELL`
4. Run `task install-core`.[^2]
5. [Live your life](https://www.youtube.com/watch?v=koVHN6eO4Xg&t=72s)!

## Project setup

Once your project is set up, use the following steps to get the project ready for whatever you'd like to do with it.

### dbt Cloud IDE

1. Run `dbt seed` to load the sample data into your raw schema.
2. Delete the `jaffle-data` directory now that the raw data is loaded into the warehouse.

### All other paths

1. Run `task setup`.[^4]
2. Run a `dbt build` to build your project.
3. [Ready to run](https://www.youtube.com/watch?v=Wu4_zVxmufY&t=234s) whatever you want!

---

[^1]: This will install the dbt Cloud CLI [currently in beta] as well as the python packages necessary for running MetricFlow queries, linting your code, and other tasks.
[^2]: This will install dbt Core and the DuckDB adapter as well as the python packages necessary for running MetricFlow queries, linting your code, and other tasks.
[^3]: This will create a virtual environment called `.venv`.
[^4]: This will run a `dbt seed` then `rm -rf jaffle-data`, deleting the sample data now that it's loaded into your raw schema.
27 changes: 27 additions & 0 deletions Taskfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
version: "3"

tasks:
venv:
cmds:
- python -m venv .venv

install-cloud:
cmds:
- brew tap dbt-labs/dbt-cli
- brew install dbt-cloud-cli
- python -m pip install --upgrade pip
- python -m pip install --progress-bar off -r requirements-cloud.txt
# until SL is in the dbt Cloud CLI mf needs a profile to run
# - if [ -f profiles.yml ]; then rm -f profiles.yml; fi

install-core:
cmds:
- python -m pip install --upgrade pip
- python -m pip install --progress-bar off -r requirements-core.txt
# - if [ -f dbt_cloud.yml ]; then rm -f dbt_cloud.yml; fi

setup:
cmds:
- dbt deps
- dbt seed
# - rm -rf jaffle-data
Empty file added analyses/.gitkeep
Empty file.
1 change: 1 addition & 0 deletions dbt_cloud.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
project-id: 282964 # Put your project id here
33 changes: 33 additions & 0 deletions dbt_project.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: "jaffle_shop"
version: "2.1.0"
config-version: 2

profile: "snowflake"
require-dbt-version: ">=1.6.0"

model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["jaffle-data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"
clean-targets:
- "target"
- "dbt_packages"

vars:
truncate_timespan_to: "{{ current_timestamp() }}"
"dbt_date:time_zone": "America/Los_Angeles"

seeds:
jaffle_shop:
+schema: jaffle_shop_raw

models:
jaffle_shop:
staging:
+materialized: view
marts:
+materialized: table

0 comments on commit 90d9eee

Please sign in to comment.