Skip to content

audit(data): vehicles + customs duty provenance#39

Merged
SoapyRED merged 2 commits into
mainfrom
audit/vehicles-customs-provenance
May 16, 2026
Merged

audit(data): vehicles + customs duty provenance#39
SoapyRED merged 2 commits into
mainfrom
audit/vehicles-customs-provenance

Conversation

@SoapyRED
Copy link
Copy Markdown
Owner

Summary

Data-integrity Sprint E — vehicles reference dataset + customs duty methodology.

Part A — Vehicles

  • All 18 records in lib/data/vehicles-ref.json carry per-record provenance (≥2 canonical sources each — EU Directive 96/53/EC, UK gov.uk vehicle weights & licence categories, 49 CFR Part 393 / FMCSA 393.5 for US, plus manufacturer specs from Schmitz Cargobull / Krone / Faymonville / DAF / Mercedes-Benz / VW / Ford / Wabash National) + auditedAt: "2026-05-16", verified: false, decisionRationale.
  • lib/calculations/vehicle-ref.ts extends VehicleSpec with the mandatory provenance: VehicleProvenance field (mirrors the container-capacity.ts pattern from PR #36).
  • app/api/vehicles/route.ts surfaces provenance snake-cased — provenance.{sources, audited_at, verified, decision_rationale}. Additive; no breaking change.

Part B — Customs duty

  • lib/calculations/duty.ts gains a top-of-file methodology docstring + a structured DUTY_METHODOLOGY constant — formula (CIF → duty → VAT → totals), Trade Tariff measure-type resolution (103 / 142 / 305 / 695 / 277), 8 HMRC source URLs, verified: false, decision rationale.
  • app/api/duty/route.ts gains a GET handler: GET /api/duty?methodology=trueDUTY_METHODOLOGY. Any other GET shape returns 400 with the api-docs link. POST is unchanged.
  • lib/data/duty-sample-fixture.json10-code regression fixture (0901 coffee, 2204 wine, 6110 jerseys, 8471 computers, 8703 motor cars, 3004 medicaments, 7113 jewellery, 4011 tyres, 9504 game consoles, 0207 poultry). Anchors expected description substrings + common origins + canonical Trade Tariff URLs. Data-only this sprint; smoke assertions are a follow-up chore.

Discipline

Sandbox HTTPS allowlist did not include eur-lex.europa.eu, gov.uk, trade-tariff.service.gov.uk, manufacturer domains, ecfr.gov, fmcsa.dot.gov, unece.org, or en.wikipedia.org this audit pass. Per the rule "no field changes without primary-source backing", NO field values were mutated. Every record + the methodology carry verified: false until a future audit pass with HTTPS access can re-fetch and diff per the runbook in the verification doc.

Audit docs

FAULT 5 checklist applied

  • CHANGELOG.md entry added (2026-05-16, Data Update tag)
  • lib/changelog-data.ts entry added so /changelog renders it
  • withAuditRest wrap intact on /api/vehicles and /api/duty (lint:audit PASS — 20 in-scope routes wrapped)
  • lint:api-casing PASS — 38 route files clean, snake_case throughout
  • lint:seo-titles PASS — 4 detail templates + 4 index pages clean, builders pass 21 fixtures
  • tsc clean (npx tsc --noEmit exit 0)
  • No new public page (so no sitemap entry, no nav, no homepage tool grid change)
  • No MCP tool change (so no freightutils-mcp version bump, no README update)
  • No openapi.json / Postman update needed (additive response fields only; no new endpoint contract)

Test plan

  • Vercel preview READY (auto)
  • curl -s {preview}/api/vehicles?slug=standard-curtainsider | jq '.result.provenance.sources | length' → expect ≥ 2
  • curl -s {preview}/api/vehicles?slug=us-53ft-dry-van | jq '.result.provenance.verified' → expect false
  • curl -s {preview}/api/vehicles?slug=luton-van | jq '.result.provenance.audited_at' → expect "2026-05-16"
  • curl -s {preview}/api/duty?methodology=true | jq '.sources | length' → expect 8
  • curl -s {preview}/api/duty?methodology=true | jq '.verified' → expect false
  • curl -X POST {preview}/api/duty -H "content-type: application/json" -d '{"commodity_code":"0901","origin_country":"BR","customs_value":5000}' | jq '.commodity_description' → expect substring match "Coffee"
  • Post-merge: same checks against https://www.freightutils.com
  • Post-merge: /changelog renders the new May 16 vehicles + duty entry
  • Post-merge: 10-path Sentry-quiet sweep — zero 5xx

🤖 Generated with Claude Code

PART A — Vehicles

All 18 records in lib/data/vehicles-ref.json carry per-record
`provenance` (≥2 sources each: EU Directive 96/53/EC, UK gov.uk
weights & licence categories, 49 CFR Part 393 / FMCSA 393.5 for US
trailers, plus manufacturer published specs from Schmitz Cargobull,
Krone, Faymonville, DAF, Mercedes-Benz, VW, Ford, Wabash National)
plus `auditedAt: "2026-05-16"`, `verified: false`, and a record-
specific `decisionRationale` citing the binding regulation and the
manufacturer match.

lib/calculations/vehicle-ref.ts gained a `VehicleProvenance`
interface (snake_case `accessed_at` on source items, camelCase
`auditedAt` / `decisionRationale` matching the container-capacity.ts
pattern from PR #36).

app/api/vehicles/route.ts now surfaces the provenance block snake-
cased: `provenance.{sources, audited_at, verified, decision_rationale}`.
Additive — no breaking change for existing callers.

PART B — Customs duty

lib/calculations/duty.ts gained a top-of-file methodology docstring +
a structured `DUTY_METHODOLOGY` constant: the four-step formula
(CIF → duty → VAT → totals), Trade Tariff measure-type resolution
(103 third-country, 142 preferential, 305 VAT, 695 anti-dumping,
277 restrictions), 8 HMRC source URLs, `audited_at: "2026-05-16"`,
`verified: false`, and decision rationale.

app/api/duty/route.ts gained a GET handler:
  GET /api/duty?methodology=true → DUTY_METHODOLOGY
Any other GET shape returns 400 with the api-docs link. POST is
unchanged.

lib/data/duty-sample-fixture.json anchors 10 high-volume HS headings
(0901 coffee, 2204 wine, 6110 jerseys, 8471 computers, 8703 motor
cars, 3004 medicaments, 7113 jewellery, 4011 tyres, 9504 game
consoles, 0207 poultry) to expected description substrings + common
origins + canonical Trade Tariff URLs. Data only this sprint — smoke
assertions are a follow-up chore.

DISCIPLINE

Sandbox HTTPS allowlist did not include eur-lex.europa.eu, gov.uk,
trade-tariff.service.gov.uk, the manufacturer domains, ecfr.gov,
fmcsa.dot.gov, unece.org, or en.wikipedia.org on this audit pass.
Per the rule "no field changes without primary-source backing",
NO field values were mutated. Every record + the methodology carry
`verified: false` until a future audit pass with HTTPS access can
re-fetch and diff.

DOCS

- docs/audit/vehicles-customs-completeness-2026-05-16.md — per-field
  gap report (18 vehicles, methodology, fixture).
- docs/audit/vehicles-customs-verification-2026-05-16.md — what was
  verified, what was deferred, runbook for flipping verified: false
  → verified: true once HTTPS access is restored.

FAULT 5 — CHANGELOG.md + lib/changelog-data.ts entries added for
2026-05-16. No new public page, no sitemap entry needed, no MCP
surface affected. Existing /api/vehicles + /api/duty routes still
audit-wrapped (lint:audit confirms). lint:api-casing confirms 38
route files clean — snake_case throughout.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 16, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
freighttools Ready Ready Preview, Comment May 16, 2026 7:39am

Request Review

Miscounted the source dataset; the live API meta.total flagged it
(returned 17). Updates four files:

- docs/audit/vehicles-customs-completeness-2026-05-16.md —
  total count, per-category breakdown (9 artic / 5 rigid / 3 van),
  EU/US split (15 / 2), eur-lex citation count (10 records, not 8).
- docs/audit/vehicles-customs-verification-2026-05-16.md —
  "all 17 records" + the blocked-domains table count.
- CHANGELOG.md and lib/changelog-data.ts entries — "17" not "18".

No code change; no field-value change; provenance count unchanged
(17/17 = 100% as already documented).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@SoapyRED SoapyRED merged commit 84fce3c into main May 16, 2026
2 checks passed
@SoapyRED SoapyRED deleted the audit/vehicles-customs-provenance branch May 16, 2026 07:40
SoapyRED added a commit that referenced this pull request May 16, 2026
The May 16 vehicles + duty changelog entry's title still said
"18 vehicles" after PR #39 merged. The desc body was correct (17)
since the count-correction follow-up commit on the audit branch
only updated `desc`, not `title`. Fixes the visible /changelog
title to match the actual dataset count.

Co-authored-by: SoapyRED <soapyred@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SoapyRED added a commit that referenced this pull request May 16, 2026
Bumps Last-updated 9 May → 16 May. Captures the 17 PRs landed across
2026-05-13..2026-05-16 (PR #25 through PR #41) plus the 14 May infra
changes that didn't have their own PR (Cloudflare disconnect, Upstash
PAYG, IndexNow live).

Sections refreshed:
- Sprint cadence 13–16 May (new): full PR list with one-liner per PR.
- Platform: MCP v2.1.0 → v2.1.1; route count 36 → 38.
- Infrastructure changes (new): CF Workers disconnected 14 May, CF DNS-
  only / Vercel firewall is sole edge security, Upstash PAYG $20 cap,
  CLAUDE.md at root encodes FAULT 5 + FAULT 14, IndexNow workflow live.
- Data integrity status (new): table for ULD / Airlines / ADR / Containers
  / UN-LOCODE / HS / Vehicles / Customs-duty. ULD + Airlines + ADR
  verified: true; the other 5 verified: false pending allowlist
  extension (specific domains enumerated).
- Scraper defence status (new): PR #31 / #32 / #33 / #38 live, Phases
  3+4 deferred to runbook, Phase 2 skipped.
- Edge firewall: scoped to Vercel-only (CF inert now).
- Distribution surfaces: table with current download counts, Smithery
  score, MCP Registry STALE flag, Glama description STALE flag.
- Weekly digest CLI (new): six FAULT 14 invariants summarised; points
  at scripts/weekly-digest/README.md for the full spec.
- Vercel Analytics: 30-day baseline updated (3,311 visitors / 6,070
  PV / 69% bounce / SG 73%).
- First validated user signals: Tom (CEVA) preserved + Simon's team
  organic adoption added per 16 May report.
- What's blocked / What's next / Red flags: updated to reflect today's
  reality — vehicles+customs SHIPPED (#39 #40), weekly digest SHIPPED
  (#41), Make.com Town Hall 21 May 4PM BST queued, CEVA→WFS transition
  complete with week 2 of induction pending.
- Canonical references: added pointers to scripts/weekly-digest/ and
  the IndexNow workflow.

No CHANGELOG entry — internal doc, not user-visible. Per the prompt.

Co-authored-by: SoapyRED <soapyred@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant