-
Notifications
You must be signed in to change notification settings - Fork 5
Harvest Source Review and Validation
This page explains how agencies can review current harvest sources and validate proposed or updated harvest sources before submitting a request for a new DCAT-US 3 harvest source or a change to an existing harvest source.
Agencies should use these tools before submitting a harvest source change request to help confirm existing harvest source information, test whether the proposed source is expected to pass validation, and reduce back-and-forth during the review process.
| Topic | Resource | Purpose |
|---|---|---|
| Review current harvest sources | https://harvest.data.gov | View current agency harvest sources |
| Validate proposed or updated harvest source | https://harvest.data.gov/validate/ | Test whether a proposed or updated harvest source is expected to pass validation |
| Submit harvest source change request | https://touchpoints.app.cloud.gov/touchpoints/ffb7e9d0 | Request a new DCAT-US 3 harvest source or a change to an existing harvest source |
| DCAT-US 3 implementation guidance | https://resources.data.gov/resources/dcat-us-3-implementation/ | Review DCAT-US 3 requirements and implementation guidance |
Before submitting a harvest source change request, agencies should review their current harvest sources and validate the proposed or updated harvest source.
This helps agencies:
- Confirm existing harvest source information
- Determine whether a new harvest source is needed
- Determine whether an existing harvest source should be updated, replaced, or retired
- Confirm that the proposed source URL is correct
- Identify validation errors or warnings early
- Resolve metadata issues before submitting a request
- Reduce follow-up questions during Data.gov review
- Support a smoother DCAT-US 3 transition process
Resource:
https://harvest.data.gov
Agencies can view their current harvest sources at harvest.data.gov.
This site allows agencies to review the harvest sources currently associated with their agency. Agencies should use this information to understand their existing harvest source configuration before requesting a new DCAT-US 3 harvest source or submitting a change to a current harvest source.
Agencies can use harvest.data.gov to review current harvest source information, including whether an agency already has existing harvest sources that may need to be updated, replaced, or retired.
Reviewing current harvest sources can help agencies determine whether they need to:
- Request a new DCAT-US 3 harvest source
- Update an existing harvest source
- Correct harvest source information
- Confirm the status or configuration of existing harvest sources
- Identify whether a current source should be replaced, updated, or retired
- Determine whether they are transitioning from DCAT-US 1.1 to DCAT-US 3
Agencies may want to consider the following questions:
- What harvest sources are currently associated with the agency?
- Is the existing source URL still correct?
- Is the agency planning to replace an existing harvest source?
- Is the agency planning to add a new DCAT-US 3 harvest source while temporarily maintaining an existing source?
- Is the agency planning to update an existing source in place?
- Should the current source be retired after transition?
- Does the agency need to transition datasets or metadata records from DCAT-US 1.1 to DCAT-US 3?
- Who is the agency point of contact for the harvest source request?
Resource:
https://harvest.data.gov/validate/
Agencies can use the Harvest Source Validation Tool to check whether their proposed or updated harvest source is expected to pass validation.
This tool can be used before submitting a request for a new DCAT-US 3 harvest source or before requesting a change to an existing harvest source.
Using the validation tool can help agencies identify and resolve metadata issues earlier in the process, before a request is submitted through the Harvest Source Change Request Form.
Agencies may use the validation tool to:
- Test whether a proposed harvest source passes validation
- Identify metadata errors or warnings
- Confirm that a DCAT-US 3 harvest source is structured correctly
- Resolve validation issues before requesting that Data.gov add or update the harvest source
- Reduce back-and-forth during the harvest source review process
If the harvest source does not pass validation, agencies should review and correct the identified issues before submitting a new or updated harvest source request, where possible.
When validating a proposed or updated harvest source, agencies should confirm that:
- The source URL is correct.
- The source URL is accessible.
- The metadata file is available at the submitted URL.
- The metadata file follows the expected DCAT-US 3 structure.
- Required fields are present.
- Metadata values use expected formats.
- Controlled values, where applicable, are used correctly.
- Dataset and distribution information is current.
- Validation errors or warnings have been reviewed.
- Any known issues are documented for follow-up.
If the proposed or updated harvest source passes validation, agencies may proceed with submitting a harvest source change request, if needed.
Harvest Source Change Request Form:
https://touchpoints.app.cloud.gov/touchpoints/ffb7e9d0
Before submitting the form, agencies should also confirm:
- The source URL is final and correct.
- The agency point of contact is identified.
- The request type is clear.
- The agency knows whether it is requesting a new harvest source or a change to an existing harvest source.
- Any transition considerations have been identified.
If the proposed or updated harvest source does not pass validation, agencies should review the validation results and correct issues where possible before submitting a harvest source change request.
Agencies may need to:
- Correct missing required fields
- Correct invalid values or formats
- Review field structure
- Review dataset or distribution metadata
- Update the metadata file at the source URL
- Re-run validation after making corrections
- Document any known issues that may require follow-up
If the agency needs help interpreting validation results, the agency may bring the question to DCAT Interagency Q&A Office Hours.
Meeting schedule updates are posted at:
https://resources.data.gov/resources/dcat-us-3-updates/
Agencies should use the following workflow before submitting a harvest source change request.
Review the DCAT-US 3 Implementation Guide:
https://resources.data.gov/resources/dcat-us-3-implementation/
Agencies may also review:
-
DCAT-US 3 to M-25-05 Crosswalk:
https://resources.data.gov/resources/dcat-us-3-crosswalk/ -
DCAT-US 3 Technical Implementation FAQs:
https://github.com/GSA/dcat-us/wiki/DCAT-US-3-Technical-Implementation-FAQ
Create or update the metadata file using the agency’s selected process or tool.
This may include:
- Inventory.data.gov
- An agency-managed metadata system
- An agency data inventory platform
- A catalog or publishing system
- Another internal metadata process
If access to Inventory.data.gov is needed, agencies can request access here:
https://touchpoints.app.cloud.gov/touchpoints/9145dd7e
Once an Inventory.data.gov account is established, agencies must re-submit the same form once per quarter to maintain access.
Review current agency harvest sources at:
Use this step to confirm existing harvest source information and determine whether the agency needs to request:
- A new DCAT-US 3 harvest source
- A change to an existing harvest source
- Replacement or retirement of a current source
- Transition from DCAT-US 1.1 to DCAT-US 3
Validate the proposed or updated harvest source at:
https://harvest.data.gov/validate/
Review the validation results and resolve errors or warnings where possible.
If a new harvest source or change to an existing harvest source is needed, submit the Harvest Source Change Request Form:
https://touchpoints.app.cloud.gov/touchpoints/ffb7e9d0
Before submitting a harvest source change request, agencies should complete the following checklist.
- Reviewed the DCAT-US 3 Implementation Guide.
- Reviewed the DCAT-US 3 to M-25-05 Crosswalk, if applicable.
- Checked the DCAT-US 3 Technical Implementation FAQs.
- Created or updated the DCAT-US 3 compliant metadata file.
- Reviewed current harvest sources at https://harvest.data.gov.
- Confirmed whether the request is for a new harvest source or a change to an existing harvest source.
- Confirmed whether an existing source should be updated, replaced, or retired.
- Validated the proposed or updated source at https://harvest.data.gov/validate/.
- Reviewed validation results.
- Addressed validation errors or warnings where possible.
- Confirmed the source URL.
- Identified the agency point of contact for follow-up.
- Submitted the Harvest Source Change Request Form, if needed.
Agencies can view current harvest sources at:
Agencies can validate a proposed or updated harvest source at:
https://harvest.data.gov/validate/
Yes. Agencies should validate the proposed or updated harvest source before submitting a request. This helps identify issues earlier and can reduce back-and-forth during the review process.
Agencies should review the validation results, correct issues where possible, update the metadata file, and re-run validation.
If the agency needs help interpreting validation results, the agency may bring the question to DCAT Interagency Q&A Office Hours.
Agencies should resolve validation issues before submitting a harvest source change request where possible. If issues remain, agencies should document the known issues and be prepared to discuss them with the Data.gov team.
Agencies should submit harvest source change requests using the Harvest Source Change Request Form:
https://touchpoints.app.cloud.gov/touchpoints/ffb7e9d0
https://resources.data.gov/resources/dcat-us-3-implementation/
Primary reference for agencies implementing the DCAT-US 3 metadata standard.
https://resources.data.gov/resources/dcat-us-3-crosswalk/
Reference for understanding how DCAT-US 3 metadata elements relate to M-25-05-related considerations.
https://github.com/GSA/dcat-us/wiki/DCAT-US-3-Technical-Implementation-FAQ
Frequently asked questions related to technical implementation of the DCAT-US 3 metadata standard are posted here as they are developed.
Use this site to view current agency harvest sources.
https://harvest.data.gov/validate/
Use this tool to test whether a proposed or updated harvest source is expected to pass validation.
https://touchpoints.app.cloud.gov/touchpoints/ffb7e9d0
Use this form to request a new DCAT-US 3 harvest source or a change to an existing harvest source.
https://touchpoints.app.cloud.gov/touchpoints/9145dd7e
Use this form to request or renew access to Inventory.data.gov. Once an account is established, this same form must be re-submitted once per quarter to maintain access.
https://resources.data.gov/resources/dcat-us-3-updates/
Use this page for meeting schedule updates and related DCAT-US 3 engagement information.
- DCAT-US 3 Implementation Resources
- Agency Workflow for DCAT-US 3 Harvest Source Requests
- Inventory.data.gov Access
- Harvest Source Review and Validation
- Harvest Source Change Requests
- FAQ Overview
- DCAT-US 3 Technical Implementation FAQ
- Harvest Source Request FAQ
- Inventory.data.gov Access FAQ
- Meeting FAQ
- Tracking and Issue Routing FAQ
- DCAT Interagency Maintenance Team
- DCAT IMT CHARTER
- Maintenance Process Overview
- Roles and Responsibilities
- Issue Triage and GitHub Workflow
- Semi-Annual Release Cycles
- Agency Feedback and Participation
- Release Communications and Outreach
- Annual Retrospective
- Maintenance Meeting Schedule
- Maintenance Templates