Skip to content

Releases: Ddscully/dlt-dbt-duckdb-evidence

Release list

Warehouse snapshot 2026-10-30

Warehouse snapshot 2026-10-30 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 30 Oct 09:40
data-2026-10-30
121f764
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-10-30, from commit
121f764.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2026;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-10-30/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb everything below, plus history 164.6 MB
lakehouse.tar.gz the landing zone dlt wrote (DuckLake) 296,676 6.3 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,891 1960–2024 1.2 MB
analytics__pipeline_runs.parquet analytics.pipeline_runs 3,378 — 89 kB
analytics__pipeline_sources.parquet analytics.pipeline_sources 8 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 36 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 469 — 18 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 184 kB
marts__dim_country.parquet marts.dim_country 228 — 9 kB
marts__dim_country_income_history.parquet marts.dim_country_income_history 7,901 1987–2025 7 kB
marts__dim_country_year.parquet marts.dim_country_year 63,156 1750–2026 11 kB
marts__dim_currency.parquet marts.dim_currency 45 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,587 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 253 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 169 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,665 — 272 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,665 1990–2024 113 kB
marts__fct_country_weather_year.parquet marts.fct_country_weather_year 820 2007–2026 30 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,286 1750–2026 1.7 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,413 2007–2026 40 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 379,262 1999–2026 3.6 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,614 1999–2026 1.0 MB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 263,541 — 3.7 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,044,848 2009–2011 30.2 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 17,974 — 539 kB
staging__stg_co2.parquet staging.stg_co2 42,648 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,291 1900–2025 304 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 740 2007–2026 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,413 2007–2026 9 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 263,541 — 3.3 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,044,848 — 16.1 MB
staging__stg_wdi.parquet staging.stg_wdi 17,226 1960–2025 858 kB
staging__stg_weather_daily.parquet staging.stg_weather_daily 296,676 — 3.2 MB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

manifest.json also says which columns may be summed. Its additivity map
labels every numeric column of every marts and analytics table published
here — 254 of them: additive (sum it in any direction),
semi_additive (summable some ways and not others — the column's own
description says which), non_additive (a ratio, rate, price, average or
extremum: recompute it from its components rather than aggregating it) and
not_a_measure (a key, a calendar part, or a parameter carried on the row).
Roughly half the numeric columns here are non-additive, which a Parquet file has
no way of telling you: sum(renewables_share_pct) and avg(co2_per_capita)
across countries are both meaningless and both come back a number.

And what each column means. Its relations map gives every published table's
description and each column's type and description — 422 of the
569 columns carry one; null means none has been written yet, not
that the column needs none. The same text is on the tables in
warehouse.duckdb (select comment from duckdb_columns()), and nowhere in the
Parquet, which has no place for it.

raw is not in warehouse.duckdb any more. dlt lands the source tables
in a DuckLake catalog rather than in the database, so
the database holds what dbt built — staging, marts, analytics, history —
and the landing zone ships separately, as the tarball above. That asset carries
only what a rebuild cannot fetch again: the capital-city weather archive, which
costs more than a day of Open-Meteo's free-tier allowance. Unpack it and open it
where it lands — the catalog records a relative data path, so no
OVERRIDE_DATA_PATH is needed:

-- tar xzf lakehouse.tar.gz && cd lakehouse
install ducklake; load ducklake;
attach 'ducklake:duckdb:catalog.duckdb' as lakehouse;
select count(*) from lakehouse.raw.om_weather_daily;
  • Grain: one row per (country_iso3, year), with these exceptions —
    marts.dim_country is one row per country (names, region, income group,
    capital coordinates) and is what everything else's country_iso3 joins to, the
    two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.5% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two. All four fact and
    dimension tables carry country_iso3 beside the source's own country label,
    so retail can be joined to the country tables above — the source spells nine
    of its 43 countries in its own way (EIRE, RSA, USA, ...) and joining
    those two halves on a country name silently drops them.
  • customer_id is pseudonymised in this release and is stable across
    releases.     It is a salted digest of the publisher's own id, applied to every
    copy of the column in the file (staging, marts, analytics), so it
    joins the retail tables to each other and to the previous release, and it does
    not join them back to the source workbook. The columns beside it are not
    masked and are not anonymous: 98.6% of the 5,881 customers are unique on
    (first_order_gbp, net_revenue_gbp, n_orders) alone. Treat these tables as
    personal data, because that is what they are — the reasoning is in
    docs/DATA_PROTECTION.md.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate forward across weekends and holidays and marks it (is_carried_forward,
    rate_source_date), and the *per...
Read more
Assets 42

Warehouse snapshot 2026-10-14

Warehouse snapshot 2026-10-14

Choose a tag to compare

View all tags
@github-actions github-actions released this 14 Oct 05:57
data-2026-10-14
af9eede
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-10-14, from commit
af9eede.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2026;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-10-14/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb everything below, plus history 165.7 MB
lakehouse.tar.gz the landing zone dlt wrote (DuckLake) 296,020 6.3 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,891 1960–2024 1.2 MB
analytics__pipeline_runs.parquet analytics.pipeline_runs 2,816 — 92 kB
analytics__pipeline_sources.parquet analytics.pipeline_sources 8 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 36 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 469 — 18 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 184 kB
marts__dim_country.parquet marts.dim_country 228 — 9 kB
marts__dim_country_income_history.parquet marts.dim_country_income_history 7,901 1987–2025 7 kB
marts__dim_country_year.parquet marts.dim_country_year 63,156 1750–2026 11 kB
marts__dim_currency.parquet marts.dim_currency 45 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,587 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 252 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 170 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,665 — 272 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,665 1990–2024 113 kB
marts__fct_country_weather_year.parquet marts.fct_country_weather_year 820 2007–2026 30 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,283 1750–2026 1.7 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,409 2007–2026 40 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 378,798 1999–2026 3.6 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,614 1999–2026 1.0 MB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 263,193 — 3.7 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,067,371 2009–2011 30.7 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 18,286 — 545 kB
staging__stg_co2.parquet staging.stg_co2 42,654 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,291 1900–2025 304 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 737 2007–2026 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,409 2007–2026 9 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 263,193 — 3.3 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,067,371 — 16.5 MB
staging__stg_wdi.parquet staging.stg_wdi 17,226 1960–2025 859 kB
staging__stg_weather_daily.parquet staging.stg_weather_daily 296,020 — 3.2 MB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

manifest.json also says which columns may be summed. Its additivity map
labels every numeric column of every marts and analytics table published
here — 254 of them: additive (sum it in any direction),
semi_additive (summable some ways and not others — the column's own
description says which), non_additive (a ratio, rate, price, average or
extremum: recompute it from its components rather than aggregating it) and
not_a_measure (a key, a calendar part, or a parameter carried on the row).
Roughly half the numeric columns here are non-additive, which a Parquet file has
no way of telling you: sum(renewables_share_pct) and avg(co2_per_capita)
across countries are both meaningless and both come back a number.

And what each column means. Its relations map gives every published table's
description and each column's type and description — 422 of the
569 columns carry one; null means none has been written yet, not
that the column needs none. The same text is on the tables in
warehouse.duckdb (select comment from duckdb_columns()), and nowhere in the
Parquet, which has no place for it.

raw is not in warehouse.duckdb any more. dlt lands the source tables
in a DuckLake catalog rather than in the database, so
the database holds what dbt built — staging, marts, analytics, history —
and the landing zone ships separately, as the tarball above. That asset carries
only what a rebuild cannot fetch again: the capital-city weather archive, which
costs more than a day of Open-Meteo's free-tier allowance. Unpack it and open it
where it lands — the catalog records a relative data path, so no
OVERRIDE_DATA_PATH is needed:

-- tar xzf lakehouse.tar.gz && cd lakehouse
install ducklake; load ducklake;
attach 'ducklake:duckdb:catalog.duckdb' as lakehouse;
select count(*) from lakehouse.raw.om_weather_daily;
  • Grain: one row per (country_iso3, year), with these exceptions —
    marts.dim_country is one row per country (names, region, income group,
    capital coordinates) and is what everything else's country_iso3 joins to, the
    two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.8% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two. All four fact and
    dimension tables carry country_iso3 beside the source's own country label,
    so retail can be joined to the country tables above — the source spells nine
    of its 43 countries in its own way (EIRE, RSA, USA, ...) and joining
    those two halves on a country name silently drops them.
  • customer_id is pseudonymised in this release and is stable across
    releases.     It is a salted digest of the publisher's own id, applied to every
    copy of the column in the file (staging, marts, analytics), so it
    joins the retail tables to each other and to the previous release, and it does
    not join them back to the source workbook. The columns beside it are not
    masked and are not anonymous: 98.6% of the 5,881 customers are unique on
    (first_order_gbp, net_revenue_gbp, n_orders) alone. Treat these tables as
    personal data, because that is what they are — the reasoning is in
    docs/DATA_PROTECTION.md.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate forward across weekends and holidays and marks it (is_carried_forward,
    rate_source_date), and the *per...
Read more
Loading

Warehouse snapshot 2026-10-13

Warehouse snapshot 2026-10-13

Choose a tag to compare

View all tags
@github-actions github-actions released this 13 Oct 18:42
data-2026-10-13
af9eede
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-10-13, from commit
af9eede.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2026;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-10-13/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb everything below, plus history 165.2 MB
lakehouse.tar.gz the landing zone dlt wrote (DuckLake) 236,078 5.0 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,891 1960–2024 1.2 MB
analytics__pipeline_runs.parquet analytics.pipeline_runs 1,696 — 64 kB
analytics__pipeline_sources.parquet analytics.pipeline_sources 8 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 36 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 469 — 18 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 184 kB
marts__dim_country.parquet marts.dim_country 228 — 9 kB
marts__dim_country_income_history.parquet marts.dim_country_income_history 7,901 1987–2025 7 kB
marts__dim_country_year.parquet marts.dim_country_year 63,156 1750–2026 11 kB
marts__dim_currency.parquet marts.dim_currency 45 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,587 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 252 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 170 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,665 — 272 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,665 1990–2024 113 kB
marts__fct_country_weather_year.parquet marts.fct_country_weather_year 656 2007–2026 25 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,279 1750–2026 1.7 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,405 2007–2026 40 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 378,798 1999–2026 3.6 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,614 1999–2026 1.0 MB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 263,193 — 3.7 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,067,371 2009–2011 30.7 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 18,286 — 544 kB
staging__stg_co2.parquet staging.stg_co2 42,654 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,291 1900–2025 304 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 733 2007–2026 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,405 2007–2026 9 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 263,193 — 3.3 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,067,371 — 16.4 MB
staging__stg_wdi.parquet staging.stg_wdi 17,226 1960–2025 858 kB
staging__stg_weather_daily.parquet staging.stg_weather_daily 236,078 — 2.6 MB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

manifest.json also says which columns may be summed. Its additivity map
labels every numeric column of every marts and analytics table published
here — 254 of them: additive (sum it in any direction),
semi_additive (summable some ways and not others — the column's own
description says which), non_additive (a ratio, rate, price, average or
extremum: recompute it from its components rather than aggregating it) and
not_a_measure (a key, a calendar part, or a parameter carried on the row).
Roughly half the numeric columns here are non-additive, which a Parquet file has
no way of telling you: sum(renewables_share_pct) and avg(co2_per_capita)
across countries are both meaningless and both come back a number.

And what each column means. Its relations map gives every published table's
description and each column's type and description — 422 of the
569 columns carry one; null means none has been written yet, not
that the column needs none. The same text is on the tables in
warehouse.duckdb (select comment from duckdb_columns()), and nowhere in the
Parquet, which has no place for it.

raw is not in warehouse.duckdb any more. dlt lands the source tables
in a DuckLake catalog rather than in the database, so
the database holds what dbt built — staging, marts, analytics, history —
and the landing zone ships separately, as the tarball above. That asset carries
only what a rebuild cannot fetch again: the capital-city weather archive, which
costs more than a day of Open-Meteo's free-tier allowance. Unpack it and open it
where it lands — the catalog records a relative data path, so no
OVERRIDE_DATA_PATH is needed:

-- tar xzf lakehouse.tar.gz && cd lakehouse
install ducklake; load ducklake;
attach 'ducklake:duckdb:catalog.duckdb' as lakehouse;
select count(*) from lakehouse.raw.om_weather_daily;
  • Grain: one row per (country_iso3, year), with these exceptions —
    marts.dim_country is one row per country (names, region, income group,
    capital coordinates) and is what everything else's country_iso3 joins to, the
    two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.8% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two. All four fact and
    dimension tables carry country_iso3 beside the source's own country label,
    so retail can be joined to the country tables above — the source spells nine
    of its 43 countries in its own way (EIRE, RSA, USA, ...) and joining
    those two halves on a country name silently drops them.
  • customer_id is pseudonymised in this release and is stable across
    releases.     It is a salted digest of the publisher's own id, applied to every
    copy of the column in the file (staging, marts, analytics), so it
    joins the retail tables to each other and to the previous release, and it does
    not join them back to the source workbook. The columns beside it are not
    masked and are not anonymous: 98.6% of the 5,881 customers are unique on
    (first_order_gbp, net_revenue_gbp, n_orders) alone. Treat these tables as
    personal data, because that is what they are — the reasoning is in
    docs/DATA_PROTECTION.md.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate forward across weekends and holidays and marks it (is_carried_forward,
    rate_source_date), and the *per...
Read more
Loading

Warehouse snapshot 2026-10-01

Warehouse snapshot 2026-10-01

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Oct 09:35
data-2026-10-01
e8c1239
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-10-02, from commit
e8c1239.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2025;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-10-01/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb everything below, plus history 167.0 MB
lakehouse.tar.gz the landing zone dlt wrote (DuckLake) 235,791 5.1 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 1.2 MB
analytics__pipeline_runs.parquet analytics.pipeline_runs 576 — 33 kB
analytics__pipeline_sources.parquet analytics.pipeline_sources 8 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 37 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 486 — 18 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 184 kB
marts__dim_country.parquet marts.dim_country 228 — 9 kB
marts__dim_country_income_history.parquet marts.dim_country_income_history 7,901 1987–2025 7 kB
marts__dim_country_year.parquet marts.dim_country_year 62,928 1750–2025 8 kB
marts__dim_currency.parquet marts.dim_currency 45 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,561 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 252 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 169 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,665 — 271 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,630 1990–2024 112 kB
marts__fct_country_weather_year.parquet marts.fct_country_weather_year 656 2007–2026 24 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,138 1750–2025 1.7 MB
marts__fct_emissions_energy_v1.parquet marts.fct_emissions_energy_v1 43,138 1750–2025 1.7 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,373 2007–2025 39 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 378,450 1999–2026 3.5 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,614 1999–2026 1.0 MB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 262,961 — 3.7 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,067,371 2009–2011 30.7 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 18,286 — 544 kB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 304 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,373 2007–2025 9 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 262,961 — 3.3 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,067,371 — 16.4 MB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 851 kB
staging__stg_weather_daily.parquet staging.stg_weather_daily 235,791 — 2.6 MB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

manifest.json also says which columns may be summed. Its additivity map
labels every numeric column of every marts and analytics table published
here — 292 of them: additive (sum it in any direction),
semi_additive (summable some ways and not others — the column's own
description says which), non_additive (a ratio, rate, price, average or
extremum: recompute it from its components rather than aggregating it) and
not_a_measure (a key, a calendar part, or a parameter carried on the row).
Roughly half the numeric columns here are non-additive, which a Parquet file has
no way of telling you: sum(renewables_share_pct) and avg(co2_per_capita)
across countries are both meaningless and both come back a number.

And what each column means. Its relations map gives every published table's
description and each column's type and description — 465 of the
612 columns carry one; null means none has been written yet, not
that the column needs none. The same text is on the tables in
warehouse.duckdb (select comment from duckdb_columns()), and nowhere in the
Parquet, which has no place for it.

raw is not in warehouse.duckdb any more. dlt lands the source tables
in a DuckLake catalog rather than in the database, so
the database holds what dbt built — staging, marts, analytics, history —
and the landing zone ships separately, as the tarball above. That asset carries
only what a rebuild cannot fetch again: the capital-city weather archive, which
costs more than a day of Open-Meteo's free-tier allowance. Unpack it and open it
where it lands — the catalog records a relative data path, so no
OVERRIDE_DATA_PATH is needed:

-- tar xzf lakehouse.tar.gz && cd lakehouse
install ducklake; load ducklake;
attach 'ducklake:duckdb:catalog.duckdb' as lakehouse;
select count(*) from lakehouse.raw.om_weather_daily;
  • Grain: one row per (country_iso3, year), with these exceptions —
    marts.dim_country is one row per country (names, region, income group,
    capital coordinates) and is what everything else's country_iso3 joins to, the
    two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.8% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two. All four fact and
    dimension tables carry country_iso3 beside the source's own country label,
    so retail can be joined to the country tables above — the source spells nine
    of its 43 countries in its own way (EIRE, RSA, USA, ...) and joining
    those two halves on a country name silently drops them.
  • customer_id is pseudonymised in this release and is stable across
    releases.     It is a salted digest of the publisher's own id, applied to every
    copy of the column in the file (staging, marts, analytics), so it
    joins the retail tables to each other and to the previous release, and it does
    not join them back to the source workbook. The columns beside it are not
    masked and are not anonymous: 98.6% of the 5,881 customers are unique on
    (first_order_gbp, net_revenue_gbp, n_orders) alone. Treat these tables as
    personal data, because that is what they are — the reasoning is in
    docs/DATA_PROTECTION.md.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate fo...
Read more
Loading

Warehouse snapshot 2026-09-01

Warehouse snapshot 2026-09-01

Choose a tag to compare

View all tags
@github-actions github-actions released this 01 Sep 12:35
data-2026-09-01
b006e1e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-09-01, from commit
b006e1e.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2025;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-09-01/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb everything below, plus history 162.5 MB
lakehouse.tar.gz the landing zone dlt wrote (DuckLake) 234,520 5.1 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 1.2 MB
analytics__pipeline_sources.parquet analytics.pipeline_sources 8 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 35 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 440 — 17 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 183 kB
marts__dim_country_year.parquet marts.dim_country_year 62,928 1750–2025 8 kB
marts__dim_currency.parquet marts.dim_currency 47 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,561 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 251 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 169 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,665 — 275 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,630 1990–2024 112 kB
marts__fct_country_weather_year.parquet marts.fct_country_weather_year 656 2007–2026 24 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,138 1750–2025 1.6 MB
marts__fct_emissions_energy_v1.parquet marts.fct_emissions_energy_v1 43,138 1750–2025 1.6 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,373 2007–2025 38 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 382,110 1999–2026 3.5 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,765 1999–2026 1.0 MB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 265,499 — 3.7 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,067,371 2009–2011 30.5 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 18,286 — 516 kB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 641 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 284 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,373 2007–2025 8 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 265,499 — 3.3 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,067,371 — 16.2 MB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 850 kB
staging__stg_weather_daily.parquet staging.stg_weather_daily 234,520 — 2.6 MB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

raw is not in warehouse.duckdb any more. dlt lands the source tables
in a DuckLake catalog rather than in the database, so
the database holds what dbt built — staging, marts, analytics, history —
and the landing zone ships separately, as the tarball above. That asset carries
only what a rebuild cannot fetch again: the capital-city weather archive, which
costs more than a day of Open-Meteo's free-tier allowance. Unpack it and open it
where it lands — the catalog records a relative data path, so no
OVERRIDE_DATA_PATH is needed:

-- tar xzf lakehouse.tar.gz && cd lakehouse
install ducklake; load ducklake;
attach 'ducklake:duckdb:catalog.duckdb' as lakehouse;
select count(*) from lakehouse.raw.om_weather_daily;
  • Grain: one row per (country_iso3, year), with these exceptions —
    staging.stg_country is the country dimension (region, income group), the two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.8% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two.
  • customer_id is pseudonymised in this release and is stable across
    releases.     It is a salted digest of the publisher's own id, applied to every
    copy of the column in the file (raw, staging, marts, analytics), so it
    joins the retail tables to each other and to the previous release, and it does
    not join them back to the source workbook. The columns beside it are not
    masked and are not anonymous: 98.6% of the 5,881 customers are unique on
    (first_order_gbp, net_revenue_gbp, n_orders) alone. Treat these tables as
    personal data, because that is what they are — the reasoning is in
    docs/DATA_PROTECTION.md.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate forward across weekends and holidays and marks it (is_carried_forward,
    rate_source_date), and the periods table ships both a period average and a
    period-end rate because converting a flow and converting a balance are not the
    same operation. Rates are quoted per euro, as the ECB quotes them; the
    reciprocal ships beside every one of them.
  • marts.fct_cbam_exposure is a screening tool, not a filing. It prices Annex
    I of Implementing Regulation (EU) 2025/2621 — the CBAM default values an
    importer uses when they have no verified supplier data — at a carbon price that
    is a parameter, not a market quote. The price this build used is stated on
    every row in ets_price_eur_per_t, and the tonnage columns ship beside the euro
    columns so you can re-price without rebuilding.
    The seeds it is built from (cbam_default_values, cbam_goods,
    cbam_markup_schedule) are in the DuckDB file's main schema, not the Parquet
    set.
  • The CBAM annex was corrected, and this table changed shape with it.
    Implementing Regulation (EU) 2026/1740 replaced Annexes I and IV in full on
    3 August 2026, retroactive to 1 January. Two columns are gone:
    markup_is_inferred and markup_schedule_is_irregular both described
    artefacts of the marked-up columns the annex used to publish and no longer
    does, so neither has a subject any more. There is no compatibility view for
    them — they cannot be reconstructed from the corrected annex at any price.
    The mark-up itself is unchanged in law (10 / 20 / 30%, fertilisers a flat 1%)
    and is now applied from the cbam_markup_schedule seed rather than read off
    the annex; markup_2026_pct still states it per row. The values moved very
    little — 66 of 10,503 comparable rows — but the good and count...
Read more
Loading

Warehouse snapshot 2026-08-28

Warehouse snapshot 2026-08-28

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Aug 09:43
data-2026-08-28
b929232
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-08-28, from commit
b929232.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2025;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-08-28/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb everything below, plus history 162.3 MB
lakehouse.tar.gz the landing zone dlt wrote (DuckLake) 234,356 5.1 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 1.2 MB
analytics__pipeline_sources.parquet analytics.pipeline_sources 8 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 32 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 424 — 16 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 183 kB
marts__dim_country_year.parquet marts.dim_country_year 62,928 1750–2025 8 kB
marts__dim_currency.parquet marts.dim_currency 47 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,561 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 252 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 169 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,665 — 271 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,630 1990–2024 112 kB
marts__fct_country_weather_year.parquet marts.fct_country_weather_year 656 2007–2026 24 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,138 1750–2025 1.6 MB
marts__fct_emissions_energy_v1.parquet marts.fct_emissions_energy_v1 43,138 1750–2025 1.6 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,373 2007–2025 38 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 381,994 1999–2026 3.6 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,765 1999–2026 1.0 MB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 265,441 — 3.7 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,067,371 2009–2011 30.5 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 18,286 — 516 kB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 643 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 284 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,373 2007–2025 8 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 265,441 — 3.3 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,067,371 — 16.3 MB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 851 kB
staging__stg_weather_daily.parquet staging.stg_weather_daily 234,356 — 2.6 MB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

raw is not in warehouse.duckdb any more. dlt lands the source tables
in a DuckLake catalog rather than in the database, so
the database holds what dbt built — staging, marts, analytics, history —
and the landing zone ships separately, as the tarball above. That asset carries
only what a rebuild cannot fetch again: the capital-city weather archive, which
costs more than a day of Open-Meteo's free-tier allowance. Unpack it and open it
where it lands — the catalog records a relative data path, so no
OVERRIDE_DATA_PATH is needed:

-- tar xzf lakehouse.tar.gz && cd lakehouse
install ducklake; load ducklake;
attach 'ducklake:duckdb:catalog.duckdb' as lakehouse;
select count(*) from lakehouse.raw.om_weather_daily;
  • Grain: one row per (country_iso3, year), with these exceptions —
    staging.stg_country is the country dimension (region, income group), the two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.8% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two.
  • customer_id is pseudonymised in this release and is stable across
    releases.     It is a salted digest of the publisher's own id, applied to every
    copy of the column in the file (raw, staging, marts, analytics), so it
    joins the retail tables to each other and to the previous release, and it does
    not join them back to the source workbook. The columns beside it are not
    masked and are not anonymous: 98.6% of the 5,881 customers are unique on
    (first_order_gbp, net_revenue_gbp, n_orders) alone. Treat these tables as
    personal data, because that is what they are — the reasoning is in
    docs/DATA_PROTECTION.md.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate forward across weekends and holidays and marks it (is_carried_forward,
    rate_source_date), and the periods table ships both a period average and a
    period-end rate because converting a flow and converting a balance are not the
    same operation. Rates are quoted per euro, as the ECB quotes them; the
    reciprocal ships beside every one of them.
  • marts.fct_cbam_exposure is a screening tool, not a filing. It prices Annex
    I of Implementing Regulation (EU) 2025/2621 — the CBAM default values an
    importer uses when they have no verified supplier data — at a carbon price that
    is a parameter, not a market quote. The price this build used is stated on
    every row in ets_price_eur_per_t, and the tonnage columns ship beside the euro
    columns so you can re-price without rebuilding.
    The seeds it is built from (cbam_default_values, cbam_goods,
    cbam_markup_schedule) are in the DuckDB file's main schema, not the Parquet
    set.
  • The CBAM annex was corrected, and this table changed shape with it.
    Implementing Regulation (EU) 2026/1740 replaced Annexes I and IV in full on
    3 August 2026, retroactive to 1 January. Two columns are gone:
    markup_is_inferred and markup_schedule_is_irregular both described
    artefacts of the marked-up columns the annex used to publish and no longer
    does, so neither has a subject any more. There is no compatibility view for
    them — they cannot be reconstructed from the corrected annex at any price.
    The mark-up itself is unchanged in law (10 / 20 / 30%, fertilisers a flat 1%)
    and is now applied from the cbam_markup_schedule seed rather than read off
    the annex; markup_2026_pct still states it per row. The values moved very
    little — 66 of 10,503 comparable rows — but the good and count...
Read more
Loading

Warehouse snapshot 2026-08-10

Warehouse snapshot 2026-08-10

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Aug 06:44
data-2026-08-10
652e9ea

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-08-10, from commit
652e9ea.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2025;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-08-10/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb the whole warehouse (raw + all of the below) 196.4 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 1.2 MB
analytics__pipeline_sources.parquet analytics.pipeline_sources 7 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 30 — 2 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 354 — 14 kB
analytics__retail_rfm.parquet analytics.retail_rfm 5,881 — 142 kB
marts__dim_country_year.parquet marts.dim_country_year 62,928 1750–2025 8 kB
marts__dim_currency.parquet marts.dim_currency 47 — 3 kB
marts__dim_date.parquet marts.dim_date 10,227 1999–2026 91 kB
marts__dim_grid_emission_factors.parquet marts.dim_grid_emission_factors 5,561 1990–2025 94 kB
marts__dim_retail_customer.parquet marts.dim_retail_customer 5,881 — 190 kB
marts__dim_retail_product.parquet marts.dim_retail_product 5,131 — 169 kB
marts__fct_cbam_exposure.parquet marts.fct_cbam_exposure 11,657 — 280 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,630 1990–2024 112 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,138 1750–2025 1.6 MB
marts__fct_emissions_energy_v1.parquet marts.fct_emissions_energy_v1 43,138 1750–2025 1.6 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,373 2007–2025 38 kB
marts__fct_example_scope2_emissions.parquet marts.fct_example_scope2_emissions 12 — 3 kB
marts__fct_fx_rates_daily.parquet marts.fct_fx_rates_daily 381,414 1999–2026 3.6 MB
marts__fct_fx_rates_periods.parquet marts.fct_fx_rates_periods 19,765 1999–2026 992 kB
marts__fct_fx_rates_published.parquet marts.fct_fx_rates_published 265,035 — 3.5 MB
marts__fct_retail_customer_cohorts.parquet marts.fct_retail_customer_cohorts 325 — 9 kB
marts__fct_retail_order_line.parquet marts.fct_retail_order_line 1,067,371 2009–2011 30.2 MB
marts__fct_retail_returns.parquet marts.fct_retail_returns 18,286 — 500 kB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 283 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,373 2007–2025 8 kB
staging__stg_fx_rates.parquet staging.stg_fx_rates 265,035 — 3.1 MB
staging__stg_retail_lines.parquet staging.stg_retail_lines 1,067,371 — 15.9 MB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 851 kB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

  • Grain: one row per (country_iso3, year), with these exceptions —
    staging.stg_country is the country dimension (region, income group), the two
    *_semiannual tables keep Eurostat's published (country_iso3, year, half),
    marts.fct_example_scope2_emissions is one row per site,
    marts.fct_cbam_exposure is one row per (sourcing country, good) with no year
    at all — it is a regulatory schedule, not a time series — the five FX
    tables below have no country in them, and the retail tables are per invoice
    line, product, customer or cohort month.
  • The retail tables are the one sub-country grain here. marts carries
    fct_retail_order_line (one row per invoice line, priced in GBP, EUR and USD
    at the transaction date's ECB fixing), dim_retail_product,
    dim_retail_customer, fct_retail_returns and fct_retail_customer_cohorts;
    analytics.retail_rfm scores each customer. Three things to read first: a
    negative quantity on a sale invoice is a stock write-off and not a return
    (is_stock_write_off), 22.8% of lines carry no customer id so every
    per-customer table covers a subset of the business, and returns are matched to
    the sale they reverse by inference — match_status says how confidently, per
    row, because the source has no key linking the two.
  • The FX tables are the one daily grain here. marts.dim_date is a calendar,
    marts.dim_currency is the currency dimension, and
    marts.fct_fx_rates_published / _daily / _periods are the ECB's euro
    reference rates as published, gap-filled, and aggregated to month / quarter /
    half / year. Two things to read before using them: the daily table carries a
    rate forward across weekends and holidays and marks it (is_carried_forward,
    rate_source_date), and the periods table ships both a period average and a
    period-end rate because converting a flow and converting a balance are not the
    same operation. Rates are quoted per euro, as the ECB quotes them; the
    reciprocal ships beside every one of them.
  • marts.fct_cbam_exposure is a screening tool, not a filing. It prices Annex
    I of Implementing Regulation (EU) 2025/2621 — the CBAM default values an
    importer uses when they have no verified supplier data — at a carbon price that
    is a parameter, not a market quote. The price this build used is stated on
    every row in ets_price_eur_per_t, and the tonnage columns ship beside the euro
    columns so you can re-price without rebuilding.
    The two seeds it is built from (cbam_default_values, cbam_goods) are in the
    DuckDB file's main schema, not the Parquet set.
  • One table in here is invented and says so:
    marts.fct_example_scope2_emissions prices twelve hypothetical sites against
    real grid emission factors, as a worked example of
    marts.dim_grid_emission_factors. Nothing else in this artifact is fabricated.
  • Written by DuckDB 1.5.5. Older clients may not read
    the storage format; the Parquet files have no such constraint.
  • Data last landed: 2026-08-10T06:42:55.140693+00:00.
  • Revision history (CO₂ estimates): 7,630 country-years, all on version 1 — nothing restated yet. OWID restates published
    years; history.snap_co2_estimates (in the DuckDB file, not the Parquet) keeps
    every version it has served and marts.fct_co2_estimate_versions summarises
    them. history.snap_grid_emission_factors does the same for the Scope 2
    emission factors from 2015 on, summarised into the first_published_* and
    is_restated columns of marts.dim_grid_emission_factors. Each release
    carries the previous one's history forward, so both accumulate.
  • co2_kg_per_gdp_ppp_2011 vs. co2_per_gdp_const_usd are different bases
    (OWID's 2011 international-$ PPP vs. constant 2015 US$ derived here) and their
    levels are not comparable. Divide by gdp_constant_usd, never gdp_usd, for
    anything measured over time.
  • One column was renamed, and the old shape still ships.
    marts.fct_emissions_energy.co2_per_gdp is co2_kg_per_gdp_ppp_2011 from this
    release on — same numbers, a name that states the unit and the basis, because
    the column beside it in analytics.co2_intensity is a different basis and the
    old name said neither. marts.fct_emissions_energy_v1 ships alongside with the
    old column name, and is removed on 2026-11-01. If you read the mart, move to
    the new name before then; if you read nothing else in this artifact, nothing
    else moved.

Licence

This artifact is a derived dataset: the pipeline that produced it fetches
public data at run time, cleans it, and joins it. The underlying data belongs to
its publishers and is redistributed here under their licences.

Source Publisher Licence
CO₂ and greenhouse-gas emissions Our World in Data CC BY 4.0
Energy production and consumption Our World in Data CC BY 4.0
World Development Indicators, country dimension World Bank CC BY 4.0 ...
Read more
Loading

Warehouse snapshot 2026-08-01

Warehouse snapshot 2026-08-01

Choose a tag to compare

View all tags
@github-actions github-actions released this 01 Aug 09:21
data-2026-08-01
c6223fd

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-08-01, from commit
c6223fd.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2025;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-08-01/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb the whole warehouse (raw + all of the below) 45.9 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 1.2 MB
analytics__pipeline_sources.parquet analytics.pipeline_sources 5 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 12 — 1 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 113 — 6 kB
marts__dim_country_year.parquet marts.dim_country_year 62,928 1750–2025 8 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,630 1990–2024 112 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,138 1750–2025 1.6 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,373 2007–2025 25 kB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 283 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,373 2007–2025 8 kB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 850 kB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

  • Grain: one row per (country_iso3, year) in everything but
    staging.stg_country, which is the country dimension (region, income group).
  • Written by DuckDB 1.5.5. Older clients may not read
    the storage format; the Parquet files have no such constraint.
  • Data last landed: 2026-08-01T09:20:59.400003+00:00.
  • Revision history: 7,630 country-years, all on version 1 — nothing restated yet. OWID restates published years;
    history.snap_co2_estimates (in the DuckDB file, not the Parquet) keeps every
    version it has served and marts.fct_co2_estimate_versions summarises them.
    Each release carries the previous one's history forward, so this accumulates.
  • co2_per_gdp vs. co2_per_gdp_const_usd are different bases (OWID's 2011
    international-$ PPP vs. constant 2015 US$ derived here) and their levels are
    not comparable. Divide by gdp_constant_usd, never gdp_usd, for anything
    measured over time.

Licence

This artifact is a derived dataset: the pipeline that produced it fetches
public data at run time, cleans it, and joins it. The underlying data belongs to
its publishers and is redistributed here under their licences.

Source Publisher Licence
CO₂ and greenhouse-gas emissions Our World in Data CC BY 4.0
Energy production and consumption Our World in Data CC BY 4.0
World Development Indicators, country dimension World Bank CC BY 4.0
Household electricity prices Eurostat Eurostat reuse policy

Attribute the publishers, not this repository, when you use the numbers. Neither
the publishers nor this project warrant the data; the transformations are the
pipeline's and any error in them is ours.

The pipeline code is MIT licensed.

Loading

Warehouse snapshot 2026-07-30

Warehouse snapshot 2026-07-30

Choose a tag to compare

View all tags
@github-actions github-actions released this 30 Jul 13:19
data-2026-07-30
7e6f090

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-07-30, from commit
7e6f090.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2025;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-07-30/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb the whole warehouse (raw + all of the below) 46.1 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 1.2 MB
analytics__pipeline_sources.parquet analytics.pipeline_sources 5 — 1 kB
analytics__pipeline_tables.parquet analytics.pipeline_tables 12 — 1 kB
analytics__pipeline_tests.parquet analytics.pipeline_tests 113 — 6 kB
marts__dim_country_year.parquet marts.dim_country_year 62,928 1750–2025 8 kB
marts__fct_co2_estimate_versions.parquet marts.fct_co2_estimate_versions 7,630 1990–2024 112 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 43,138 1750–2025 1.6 MB
marts__fct_eu_electricity_prices_semiannual.parquet marts.fct_eu_electricity_prices_semiannual 1,373 2007–2025 25 kB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 642 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 283 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 6 kB
staging__stg_eu_electricity_prices_semiannual.parquet staging.stg_eu_electricity_prices_semiannual 1,373 2007–2025 8 kB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 850 kB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

  • Grain: one row per (country_iso3, year) in everything but
    staging.stg_country, which is the country dimension (region, income group).
  • Written by DuckDB 1.5.4. Older clients may not read
    the storage format; the Parquet files have no such constraint.
  • Data last landed: 2026-07-30T13:19:00.296853+00:00.
  • co2_per_gdp vs. co2_per_gdp_const_usd are different bases (OWID's 2011
    international-$ PPP vs. constant 2015 US$ derived here) and their levels are
    not comparable. Divide by gdp_constant_usd, never gdp_usd, for anything
    measured over time.

Licence

This artifact is a derived dataset: the pipeline that produced it fetches
public data at run time, cleans it, and joins it. The underlying data belongs to
its publishers and is redistributed here under their licences.

Source Publisher Licence
CO₂ and greenhouse-gas emissions Our World in Data CC BY 4.0
Energy production and consumption Our World in Data CC BY 4.0
World Development Indicators, country dimension World Bank CC BY 4.0
Household electricity prices Eurostat Eurostat reuse policy

Attribute the publishers, not this repository, when you use the numbers. Neither
the publishers nor this project warrant the data; the transformations are the
pipeline's and any error in them is ours.

The pipeline code is MIT licensed.

Loading

Warehouse snapshot 2026-07-29

Warehouse snapshot 2026-07-29

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Jul 22:54
data-2026-07-29
3af6b75

Emissions, energy and development for ~200 countries at (country_iso3, year)
grain — the output of this repo's pipeline, so you can use the data without
running dlt, dbt, Polars or DuckDB yourself.

Built from the live sources on 2026-07-29, from commit
3af6b75.

Query it without downloading it

INSTALL httpfs; LOAD httpfs;
-- Alias it `warehouse`, not something shorter: the `staging` views store
-- fully-qualified SQL and resolve against that catalog name. Tables (`marts`,
-- `analytics`) read fine under any alias; the views raise
-- `Catalog "warehouse" does not exist`.
ATTACH 'https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb' AS warehouse (READ_ONLY);
SELECT * FROM warehouse.marts.fct_emissions_energy LIMIT 10;

Or a single table, no DuckDB file at all:

SELECT country_name, year, co2_mt, renewables_share_pct
FROM read_parquet('https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/marts__fct_emissions_energy.parquet')
WHERE year = 2024;

Or just grab it:

curl -LO https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/latest/download/warehouse.duckdb
duckdb warehouse.duckdb -c "select * from marts.fct_emissions_energy limit 5"

latest/download/… always points at the newest snapshot. Pin a run by swapping
it for https://github.com/Ddscully/dlt-dbt-duckdb-evidence/releases/download/data-2026-07-29/….

What's in it

Asset Table Rows Years Size
warehouse.duckdb the whole warehouse (raw + all of the below) 29.1 MB
analytics__co2_intensity.parquet analytics.co2_intensity 10,874 1960–2024 845 kB
marts__fct_emissions_energy.parquet marts.fct_emissions_energy 42,480 1750–2024 1.0 MB
staging__stg_co2.parquet staging.stg_co2 42,480 1750–2024 407 kB
staging__stg_country.parquet staging.stg_country 228 — 9 kB
staging__stg_energy.parquet staging.stg_energy 17,265 1900–2025 150 kB
staging__stg_eu_electricity_prices.parquet staging.stg_eu_electricity_prices 701 2007–2025 5 kB
staging__stg_wdi.parquet staging.stg_wdi 17,160 1960–2025 801 kB

manifest.json carries the row counts, year coverage and SHA-256 of every asset;
SHA256SUMS is sha256sum -c-compatible.

  • Grain: one row per (country_iso3, year) in everything but
    staging.stg_country, which is the country dimension (region, income group).
  • Written by DuckDB 1.5.4. Older clients may not read
    the storage format; the Parquet files have no such constraint.
  • Data last landed: 2026-07-29T22:54:23.386727+00:00.
  • co2_per_gdp vs. co2_per_gdp_const_usd are different bases (OWID's 2011
    international-$ PPP vs. constant 2015 US$ derived here) and their levels are
    not comparable. Divide by gdp_constant_usd, never gdp_usd, for anything
    measured over time.

Licence

This artifact is a derived dataset: the pipeline that produced it fetches
public data at run time, cleans it, and joins it. The underlying data belongs to
its publishers and is redistributed here under their licences.

Source Publisher Licence
CO₂ and greenhouse-gas emissions Our World in Data CC BY 4.0
Energy production and consumption Our World in Data CC BY 4.0
World Development Indicators, country dimension World Bank CC BY 4.0
Household electricity prices Eurostat Eurostat reuse policy

Attribute the publishers, not this repository, when you use the numbers. Neither
the publishers nor this project warrant the data; the transformations are the
pipeline's and any error in them is ours.

The pipeline code is MIT licensed.

Loading