harvest.data.gov

Jump to bottom

Susan Valente edited this page Apr 17, 2026 · 1 revision

harvest.data.gov is Data.gov's automated metadata ingestion service, built on the datagov-harvester application.

What it does

Pulls metadata from agency harvest sources on a scheduled basis
Supports DCAT-US JSON and WAF source types
Manages harvest sources, jobs, and records for 120+ publishing organizations

Access

Production: harvest.data.gov
Source code: GSA/datagov-harvester

Technologies

Python / Flask
Postgres
Redis

Related pages

Harvesting 2.0 -- system overview and terminology
Harvest O&M and Debugging -- operational procedures