DCAT US v3

DCAT-US v3

Tier: Intermediate Command covered: profile (default profile: dcat-us-v3)

DCAT-US v3 is the metadata schema data.gov harvests: a data.json Catalog holds Datasets, and each Dataset describes its Distributions. qsv profile writes that record from the data itself and checks it against GSA's own schema before you publish it.

This page covers what qsv does specifically for v3: how it follows GSA's guidance, what it computes so you don't have to, and how its validation lines up with data.gov's validator. For the command as a whole see Metadata Profiling; for a step-by-step walk-through see Recipe: DCAT-US v3 for data.gov.

qsv profile data.csv --catalog --validate --strict --initial-context seed.json
jq '.projection' data.csv.metadata.json > data.json

GSA guidance

These are the sources qsv follows. When GSA's prose and its schema disagree, qsv follows the schema.

Source	What it is
DCAT-US v3.0 overview	GSA's guidance page: what stays the same, changes from v1.1, fields not in the v3.0 core schema, validation
Technical Migration Guide	the 16 steps from v1.1 to v3.0
Priority Agency Data Documentation Guidance	extra keywords and fields for AI research and COVID-19 datasets (see Priority datasets)
GSA/dcat-us	the JSON Schema 2020-12 definitions and their good/bad examples
harvest.data.gov/validate	data.gov's online validator

v3 is plain JSON, not JSON-LD

GSA retired the JSON-LD context and SHACL shapes, moving them aside in the v3 repository. v3 is plain JSON validated by JSON Schema 2020-12:

• keys are unprefixed (title, not dct:title);
• @type values are unprefixed (Dataset, Distribution, Catalog, Location, …);
• there is no @context (Step 15).

qsv has emitted this canonical form since 24.0.0 (#4632). Earlier releases emitted the prefixed JSON-LD form, which data.gov's validator rejects.

Fields kept on purpose

GSA's fields not in the v3.0 core schema says "the v3.0 schema will not reject records that include them." Agencies still need three of them for Project Open Data, so qsv keeps them, prefixed so that they cannot be mistaken for v3 properties:

Field	qsv emits
bureauCode	dcat-us:bureauCode, checked against the NNN:NN pattern by qsv's schema overlay
programCode	dcat-us:programCode, checked against NNN:NNN
accessLevel	dcat-us:accessLevel (default public), with accessRights alongside it when you supply one (Step 5)
dataQuality	not emitted. qsv computes hasQualityMeasurement instead, as GSA directs (see below)

Priority datasets

GSA's priority guidance asks AI research datasets for the usg-artificial-intelligence keyword (plus usg-ai-training-data for training data), a contact who can discuss restrictions with AI researchers, isReferencedBy and page (v3's replacements for references), and a quality measurement saying whether the dataset meets the agency's Information Quality Guidelines. COVID-19 datasets get COVID-19 and coronavirus, though that guidance is no longer enforced. profile can't tell that a dataset is one of these, and only the agency can make the quality assertion, so none of it is inferred. You supply it in the seed; Recipe: A DCAT-US v3 Catalog Entry for data.gov → Priority datasets shows how, and how to add your measurement without losing the computed ones.

The migration steps, applied

GSA's Technical Migration Guide lists 16 steps for turning a v1.1 record into a v3.0 record. profile applies them when it builds the record, so a CKAN package or a seed file written in v1.1 terms still produces valid v3. Each rule lives in dcat-us-v3.yaml. Some of its comments cite step numbers from an earlier edition of the guide, which GSA has since renumbered; the table below uses the current numbers.

Step	What profile does
1: modified	dates are normalized to UTC with a Z; an offset-less CKAN timestamp gains one. An interval or repeating value (R/P1Y, 2024-01-01/2024-06-30) or an unparseable date is left out rather than emitted invalid
2: temporal	a PeriodOfTime object, computed from the Date and DateTime columns
3: spatial	a Location object with a WKT bounding box computed from the coordinates; --locate adds place names
4: language	ISO 639-1 two-letter codes, whether supplied or detected
5: accessLevel / accessRights	both kept (see above)
6: license on Distribution	license goes on the Distribution, with CKAN license ids such as cc-by mapped to their URLs. --dcat-legacy-license also keeps a Dataset-level copy
7: rights array	a single string is wrapped in an array
8: describedBy	a data-dictionary URL becomes a Distribution object, and describedByType becomes its mediaType
10: conformsTo	a Standard object on the Dataset, and on the Distribution when the resource supplies one (a bare identifier is wrapped as a Standard)
11: landingPage	a Document object, titled with the dataset title when no page title is given (GSA requires one)
12: issued	normalized like modified
14: Catalog conformsTo	--catalog writes a Standard keyed identifier: "https://resources.data.gov/dcat-us/3.0.0". That IRI is an identifier and is not meant to resolve
15: no @context	never emitted, including when DCAT discovery merges a publisher's JSON-LD markup that carries one

Steps 9 (publisher.subOrganizationOf), 13 (isPartOf → datasetSeries) and 16 (Catalog-level modified) concern fields that profile never generates. If you add them through a dataset_info override, they are written exactly as given and then checked by --validate.

Publisher markup found by DCAT discovery is usually still prefixed (dct:title). profile strips the prefixes before merging, so dct:title fills title instead of landing next to it.

Computed properties

Many v3 properties are facts about the file. They are tedious and error-prone to write by hand, and qsv can measure them:

• the time span, its resolution and update schedule;
• the spatial extent and coordinate resolution;
• the language;
• quality measurements, the column schema, the checksum and the byte size.

The Computed DCAT-US v3 properties table on the Metadata Profiling page gives the source of each one. For v3 specifically:

• spatial comes from latitude/longitude columns or, without them, a WKT POINT column (28.0.0, #4974).
• temporal, spatial and language are three of the four fields GSA singles out as most likely to fail validation. qsv builds them in the required object shapes from the data, so they don't have to be hand-converted.
• hasQualityMeasurement replaces v1.1's dataQuality flag, as GSA's fields not in the v3.0 core schema directs. Its measurements are DQV with W3C ldqd dimensions: completeness always, plus duplicate rows and CSV/UTF-8 validity with --validate.
• accrualPeriodicity is emitted only when the dates step by one exact calendar interval at least 90% of the time. Event timestamps, such as 311 requests, say nothing about how often a dataset is updated, so they get none.
• theme: each CKAN group becomes a Concept in the portal's own group scheme. With --tag-vocab, LLM tags that belong to your vocabulary become Concepts as well.
• description: --describegpt / --infer supply one from an LLM, after checking it. When there is none, a factual summary is written from the statistics. Either way it carries a disclosure that it was generated and keeps a Recommended review warning, so a generated description is never passed off as human-authored.

Note

28.0.0 (#4967, #4968, #4969, #4972, #4973, #4974, #4975): the computed properties, --locate (including county columns), the WKT POINT fallback for spatial, the LLM inputs, the CKAN-group themes and the lighter duplicate-row count. 27.0.1 emits the field shapes above, but computes fewer of them.

contactPoint is never inferred. When no identifier is supplied, the dataset's identifier falls back to the resource URL, with a review warning. That URL is the input URL for URL inputs, or a seeded resource.url (for example, a CKAN resource's) for local files. It must be an absolute IRI.

Who wins

The values qsv measures from the file (byte size, checksum, bbox, temporal coverage, column schema, quality) always come from the file. Values it generates, detects or estimates form an inferred tier. These include the temporal and spatial resolution, and a Distribution modified taken from Last-Modified. Anything you supply (--initial-context, a spec) or that the publisher's discovered DCAT provides replaces them. projection_provenance records which values were inferred. It also lists the LLM dictionary's column annotations (28.0.0, #4981); discovered DCAT has no per-column metadata, so only a dataset_info override replaces those. See Supplied, discovered, inferred: who wins.

Kept cheap

• One statistics pass. profile computes its own statistics in-process and reuses them for every derived property.
• A capped --locate sample. Place lookups use at most 40 of the densest coordinate cells, each weighted by its row count, not every row.
• Costly checks only with --validate. The duplicate-row count is one more pass over the file, and the CSV structure/UTF-8 check runs qsv validate. Both run only with --validate, so a default run stays light. On the 1M-row, 255 MB LA crime file (release build, Apple M4 Max), profile --catalog peaks at about 360 MB in 2.4 s; adding --validate raises that to about 450 MB in 3.1 s. 28.0.0 (#4975): the duplicate-row count keeps a 128-bit hash per distinct row instead of every full row. Before that, --validate peaked at about 990 MB on this file.
• Cached describegpt answers. --infer honors describegpt's cache, so re-profiling the same file doesn't call the LLM again.
• No network for local files by default. --locate and --infer are opt-in. With them off, a local file's output depends only on the file and your inputs, so it is reproducible. A URL input always goes to the network: qsv downloads the file, probes it for the publisher's DCAT markup (turn this off with --no-dcat-discovery) and reads its Last-Modified header.

How --validate matches GSA's validator

data.gov's validator checks submissions against GSA's JSON Schema bundle. qsv ships the same bundle and runs it locally. This lets you catch a rejection before you register a harvest source, and in CI where there is no browser.

The vendored bundle

• resources/dcat-us-v3/ holds all 26 GSA definitions, copied unmodified.
• MANIFEST.json pins them to GSA commit 1ca074b6 and records each file's SHA-256. A test re-hashes every file on every CI run, so any edit to a vendored file fails CI.
• The bundle's README documents how to refresh it. Schemas and conformance examples must be refreshed from the same commit.
• qsv adds two overlays that wrap GSA's Dataset and Catalog entry points:
  • the Dataset overlay checks the shape of dcat-us:bureauCode and dcat-us:programCode;
  • the Catalog overlay passes the record through unchanged.

What qsv checks, and why

These checks shipped in 24.0.0 (#4632, #4633).

Check	Why it matters
Format assertion on	JSON Schema 2020-12 treats format (e.g. date-time) as an annotation by default. A naive validator therefore passes 2024-12-15T08:30:00, which data.gov rejects. qsv asserts formats, as data.gov does.
Every GSA definition registered	Cross-references resolve inside the bundle, with no network access.
Severity from GSA's requirementLevel	The severity of each finding comes from the GSA schema's own requirementLevel for the class that owns it. Class matters: modified is Recommended on a Dataset but Mandatory on a CatalogRecord. Mandatory findings are Required, which is what --strict aborts on.
Unknown-key check	No GSA schema sets additionalProperties, so the schema alone accepts any misspelled or removed key. qsv reports unknown keys on the Catalog, each Dataset and each Distribution. Its own extensions are allowed: dcat-us:bureauCode, dcat-us:programCode, dcat-us:accessLevel, qsv:sourcePath and csvw:tableSchema.
Catalog or Dataset entry point	The Catalog schema is used when the record is a Catalog (--catalog), and the Dataset schema otherwise.

Findings go to projection_warnings. --strict turns Required findings into a non-zero exit with no output file. Recommended findings, such as a bad date format or an unknown key, are reported but don't abort the run.

Tested against GSA's own examples

qsv's test suite runs the validator over GSA's conformance examples, taken from the same commit as the schemas:

Examples	Good (must pass)	Bad (must fail)
Catalog	3	5
Dataset	8	1
Distribution	4	2

A further test checks every field in dcat-us-v3.yaml against the bundle, so a property GSA renames or removes is caught even when its template rarely emits it.

Checking with data.gov's validator

The local check and data.gov's validator use the same schema bundle. It is still worth one external check before registering a harvest source:

1. Profile with --catalog: data.gov's validator offers dcatus3.0 catalog as its only v3 option, so a Dataset-only record has nothing to check it against there.
2. Extract the Catalog: jq '.projection' data.csv.metadata.json > data.json.
3. At https://harvest.data.gov/validate/, choose dcatus3.0 catalog and upload data.json. Uploading checks the exact bytes. Pasted text can be silently altered (an em-dash, for example), so a pass would not prove anything about the file you publish.
4. Also upload a copy you have broken on purpose, for example with contactPoint deleted. It must fail. Otherwise you can't tell a real pass from a page that didn't re-run.

The Catalog emitted by 24.0.0 passed data.gov's validator when it was released.

Worked example: a popular data.gov dataset, v1.1 to v3

Crime Data from 2020 to Present (City of Los Angeles) is one of data.gov's most-viewed datasets. data.gov harvests it as a DCAT-US v1.1 record, so it makes a realistic migration test: 1,004,894 rows (255 MB), two date columns in US 04/11/2021 12:00:00 AM format, latitude/longitude with placeholder (0, 0) rows, and English free text.

# The data, and the publisher's own v1.1 record as data.gov holds it
curl -L -o lacrime.csv 'https://data.lacity.org/api/views/2nrs-mtv8/rows.csv?accessType=DOWNLOAD'
curl -s 'https://catalog.data.gov/api/dataset/crime-data-from-2020-to-present' | jq '.results[0].dcat' > la-v11.json

# Map the v1.1 fields to the CKAN package terms profile reads
jq '(first(.distribution[] | select(.describedBy))) as $d | {package: {
  title, notes: .description, name: .identifier, publisher: .publisher.name,
  contact_point: {fn: .contactPoint.fn, hasEmail: .contactPoint.hasEmail},
  license_id: .license, metadata_created: .issued, metadata_modified: .modified,
  landing_page: .landingPage, tags: .keyword, groups: [.theme[] | {title: .}],
  access_level: .accessLevel, data_dictionary: $d.describedBy, describedByType: $d.describedByType
}, resource: {url: "https://data.lacity.org/api/views/2nrs-mtv8/rows.csv?accessType=DOWNLOAD"}}' la-v11.json > la-seed.json

qsv profile lacrime.csv --catalog --validate --locate both --initial-context la-seed.json
jq '.projection' lacrime.csv.metadata.json > data.json

The v1.1 record's plain-string landingPage and describedBy (with describedByType) became v3 objects, the license moved to the Distribution, and the Public Safety theme became a Concept. From the data, qsv added:

• temporal: one PeriodOfTime per date column, 2020-01-01 to 2025-03-28. The publisher's title says "2020 to 2024"; the data runs further.
• spatial: a bounding box over Los Angeles that leaves out 2,240 rows at (0, 0) or out of range (noted in projection_warnings), plus Census Locations for California (06) and Los Angeles County (06037).
• spatialResolutionInMeters 11.1, from the 4-decimal coordinates; language en.
• hasQualityMeasurement: 80.13% of cells non-empty, 13 columns with gaps, no duplicate rows, valid UTF-8 CSV.
• csvw:tableSchema with DR_NO as the primary key.

On 2026-11-01, this Catalog, made by unreleased master, passed https://harvest.data.gov/validate/ (dcatus3.0 catalog, uploaded) with no validation errors. The same file with contactPoint deleted was rejected with 'contactPoint' is a required property. data.gov's most popular dataset, Electric Vehicle Population Data (Washington State), passed too. It has no latitude/longitude columns; with (28.0.0) #4973, --locate census names Washington and its King, Snohomish, Pierce and Clark counties from the file's State and County columns.

See also

• Metadata Profiling: output blocks, profiles, computed properties, seeding and overrides
• Recipe: DCAT-US v3 for data.gov: from a bare CSV to a record that passes --strict
• Recipe: CKAN Integration
• US Census Integration: the --locate census lookups
• docs/help/profile.md: flag reference