Weekly diesel crack spreads and retail fuel prices, as a static site on GitHub Pages. Two questions: what does a refiner earn turning crude into diesel, and how does Sweden's pump price compare with the rest of the EU and the USA.
No server, no database at runtime. A DuckDB pipeline writes JSON; a Scala.js app draws it.
- Site: https://oluies.github.io/crack/
- Spec:
specs/001-crack-and-retail-fuel-site/— built with GitHub Spec Kit; the constitution is the standing brief.
| Chart | Shows |
|---|---|
| Diesel crack spreads | NYH ULSD against Brent and WTI since 2022, weekly or daily. Weekly has a threshold ("mountain") view that fills green above and red below a breakeven you set; daily has an optional 7-day ruler. NW Europe needs ICE gasoil — see below. |
| Retail diesel and petrol | Pump prices per litre for all 27 EU members plus the USA, Sweden emphasised. Toggles for fuel, tax treatment, and EUR/USD/SEK. |
| US regional spread | Nine states for petrol and five PADD regions for diesel, against the bold national average. California against Texas is roughly a 50% difference on the same fuel, and most of it is tax. |
| Rockets and feathers | Crude benchmarks against the US pump on a dual axis — retail chases a crude spike up in weeks and drifts back down over months. |
| Source | Series | Licence |
|---|---|---|
| EIA Open Data v2 | NYH ULSD spot EER_EPD2DXL0_PF4_Y35NY_DPG, Brent RBRTE, WTI RWTC (daily, published with roughly a week's lag), US retail petrol EMM_EPMR_PTE_NUS_DPG, US retail diesel EMD_EPD2D_PTE_NUS_DPG, nine state petrol series EMM_EPMR_PTE_S**_DPG, five PADD diesel series EMD_EPD2D_PTE_R**_DPG |
US Government work — public domain. A free API key is required. |
| EU Weekly Oil Bulletin | Euro-super 95 and gas oil automobile, with and without taxes, all member states | European Commission open data, CC BY 4.0. Attribution required. |
| ECB SDMX | EXR.D.USD.EUR.SP00.A, EXR.D.SEK.EUR.SP00.A |
ECB open data. Attribution required. |
| ICE Low Sulphur Gasoil | Futures settlement | Not redistributable. See below. |
The North-West European crack is gasoil against Brent. ICE licenses gasoil futures settlements and publishes no free API, and the data cannot be redistributed here. So this repository ships that leg empty, and the chart says so rather than quietly drawing nothing.
If you have licensed access, append rows to
data/manual/ice_gasoil.csv:
obs_date,usd_per_tonne
2026-08-14,712.25Daily or weekly rows both work; the pipeline reduces either to an ISO-week mean.
Rerun pipeline/run.sh and the NW European region populates.
This is deliberate. A stub that admits the gap is honest; a series interpolated from a proxy would look identical on the chart and be wrong.
crack spread = product price in USD/bbl − crude price in USD/bbl
NYH ULSD – Brent = ULSD USD/gal × 42 − Brent USD/bbl
NYH ULSD – WTI = ULSD USD/gal × 42 − WTI USD/bbl
ICE gasoil – Brent = gasoil USD/t ÷ 7.45 − Brent USD/bbl
42 US gallons to the barrel is exact by definition. 7.45 barrels of gasoil per tonne is a conventional density factor — an approximation, which is why the chart states it.
Other conventions:
- Cross-source comparison is weekly, bucketed to ISO weeks and keyed to that week's Monday. Sources publish on different days; joining on the week key rather than the publication date makes the comparison exact.
- The axis ends at the last week a weekly survey published, capped at the current week and never earlier than the last complete week. The Oil Bulletin and EIA's retail series are point-in-time surveys — one observation is the week — so excluding the current week threw away a published price for up to seven days. Daily-sampled spot cannot extend the axis; an unfinished week there is a partial mean, which is what the three-day rule below is for.
- A weekly point needs at least three trading days. Below that the week
publishes as null and the chart draws a gap. Without this rule the week of
2026-08-10 was published as 86.28 USD/bbl — the mean of Monday and Tuesday
alone,
(84.16 + 88.39) / 2— during a steep climb, and looked exactly like a complete week. Three rather than five, so a holiday-shortened week survives. - The crack is also published daily, in
cracks_daily.json, on its own observation-date axis. That axis is not interchangeable with the weekly one and nothing may index between them. The optional 7-day ruler is the unweighted mean of the trailing seven calendar days — one trading week in steady state, shortening honestly over a holiday, where a seven-observation window would quietly reach nine calendar days back and still call itself "7-day". - EIA spot is a weekly release, and the lag depends on when you sample it.
The daily series drops on Wednesdays carrying through the prior Tuesday, so it
is about a day behind just after a release and eight days behind just before
the next one. An earlier note here called the week-long lag a property of the
source that no schedule could touch; that was measured hours before a release,
and the swing is real.
refresh.ymlnow runs Wednesday evenings to sample near the fresh end, and the freshness invariants stay wide because they fail the whole build — they are there for a source that has stopped delivering, not for a run that landed at the stale end of the cycle. What no schedule fixes is that a lag exists at all, someta.generated— when the pipeline ran — still says nothing about how old the data is, and the daily view prints the last observation date and the lag beside it rather than letting the two be confused. - Oil Bulletin prices are already in EUR per 1000 litres for every country,
and are stored as EUR/L. The workbook's
{CC}_exchange_ratecolumns are not applied — doing so would divide Swedish prices by eleven. An invariant check guards against that regression. - Currency conversion happens in the browser, against the ECB reference rate for the same week. Series are published in their native currency so there is one source of truth rather than three separately rounded pipelines.
- Missing is null. Gaps are drawn as gaps, never zero and never carried forward. The one exception is exchange rates over bank holidays, where a rate genuinely persists until it is restated.
# macOS: keep the key in the login keychain rather than a file on disk.
# NOT the Passwords app — it stores items in the data-protection keychain,
# which security(1) cannot read at all (iCloud on or off), so a shell script
# will not find it there however plainly Passwords shows it.
security add-generic-password -a "$USER" -s EIA_API_KEY -w
# or, anywhere: export EIA_API_KEY=... / put it in .env (gitignored)
pipeline/run.sh # fetch, build, verify, export to site/public/data/
mill site.fullLinkJS # or site.bundleFull to link straight into site/public/
cd site && python3 -m http.server 8000A free key comes from https://www.eia.gov/opendata/register.php. run.sh looks
for it in the environment, then the macOS Keychain, then .env — the Keychain
first of the two stored forms, because a key in cleartext on disk is a key that
eventually lands in a commit, a backup or a synced folder.
Needs DuckDB 1.5+, Mill 1.1+, JDK 21 and curl. More detail, including the
--offline / --fixtures / --verify-only modes, in
quickstart.md.
pipeline/ DuckDB SQL, in execution order, plus run.sh and verify.sql
data/manual/ ICE gasoil CSV you maintain yourself
data/fixtures/ trimmed samples so CI runs without an API key
site/ index.html, the Scala.js app, and the published JSON
specs/ the spec-driven-development artefacts
The two kinds of "incomplete" are handled separately and deliberately: the axis
decides which weeks exist (a survey either published or it did not), and
min_week_obs decides whether a daily-sampled week has enough observations to
average. Collapsing them into one rule is what discarded a published retail week
and, before that, published a two-day crack average as if it were five.
pipeline/verify.sql and pipeline/60_verify_export.sql are the real test suite.
They assert the invariants that catch failures producing a plausible but wrong
chart — a dropped ×42 on the crack, an exchange rate applied twice — because those
do not announce themselves.
pipeline/test/negative.sh tests the tests: it corrupts each input in turn and
asserts the run dies naming the right check. A check that cannot fail is worse
than no check, because it reports green and is believed.
.github/workflows/ci.yml— on push and pull request: compile the frontend, run the pipeline against fixtures, run the negative tests, and run the headless frontend smoke test..github/workflows/refresh.yml— Wednesdays 22:00 UTC, after both EIA release calendars; the EU bulletin has no published release time and usually, but not always, lands by then. Runs the pipeline for real, commits changed JSON, builds, and deploys to Pages. A run that changes nothing commits nothing — which is also what a badly timed schedule looks like, so a scheduled run that moves nothing now raises a warning annotation instead of passing in silence. Check the release calendars in the cron comment before suspecting the fetch.
The Oil Bulletin download path is a UUID the Commission reissues when it
republishes. It lives in pipeline/sources.env; when the download starts failing,
lift the current one from the
bulletin page.
The pipeline checks that what it downloaded is actually a spreadsheet, so a
reissued UUID fails with that message rather than as a parse error.
The site counts page views with GoatCounter
(oluies.goatcounter.com). No cookies, no cross-site tracking, no personal data —
which is why there is no consent banner: there is nothing to consent to under
GDPR. The counter script skips localhost, so development does not pollute the
figures.
It is loaded async and is strictly optional. Block it, and the site behaves
identically; the smoke test never executes it, so every run proves the page works
without it. The ECharts CDN is the opposite — it is the charting library, so if it
fails there are no charts. Both are permitted third-party origins under
the constitution, which spells out that
distinction rather than lumping them together.
Code © 2026 Örjan Lundberg, MIT. Data belongs to the sources above under their own terms — EIA public domain, Oil Bulletin CC BY 4.0, ECB with attribution.
Sibling project: elmix, same house style.