Skip to content

Annual Retrospective

Jump to bottom
David Aguiar edited this page Aug 31, 2026 · 2 revisions

Annual Retrospective

Draft for discussion and revision by the DCAT Interagency Maintenance Team and the Data.gov team.

This page describes the proposed annual retrospective process for the DCAT-US v3.0 maintenance cycle.

The annual retrospective is conducted as part of Cycle 2: Summer Release and Retrospective and is intended to review implementation experience, validation trends, documentation gaps, agency blockers, and priorities for the next annual maintenance cycle.

The DCAT Interagency Maintenance Team governs schema maintenance decisions. The Data.gov team at GSA maintains the DCAT-US repository and coordinates the maintenance cycle and retrospective process.

Purpose

The purpose of the annual retrospective is to review how DCAT-US v3.0 implementation and maintenance worked over the prior year and identify improvements for the next annual cycle.

The retrospective supports:

  • Better implementation guidance
  • Improved documentation
  • Better validation and harvesting outcomes
  • Identification of recurring agency blockers
  • Identification of systemic field-level error patterns
  • Prioritization of future schema clarifications or changes
  • Targeted agency implementation support
  • Improved release communications
  • Improved maintenance process operations

Timing

The annual retrospective occurs in July as part of Cycle 2: Summer Release and Retrospective.

The July timing aligns with the M-25-05 annual compliance reference date and allows the DCAT Interagency Maintenance Team and Data.gov team to review annual implementation trends, release outcomes, and agency support needs.

Participants

The annual retrospective may include:

  • DCAT Interagency Maintenance Team members
  • Data.gov team members
  • Schema team representatives
  • Documentation contributors
  • Selected agency representatives
  • Agency metadata or data inventory leads
  • Agency technical implementation leads
  • Agency policy or data governance representatives
  • Data.gov analytics or monitoring representatives familiar with New Relic data
  • Harvesting or validation subject matter experts

Agency participation may be targeted based on implementation experience, validation trends, metadata complexity, or active harvest source transition work.

Inputs

The retrospective should be informed by both quantitative and qualitative information.

Potential inputs include:

  • Harvest validation error trends for the year
  • New Relic data
  • GitHub issue trends
  • Common questions from DCAT Interagency Q&A Office Hours
  • Topics raised during DCAT Interagency Info Sharing Sessions
  • Agency feedback meeting notes
  • Harvest source request trends
  • Documentation issues
  • FAQ topics
  • Release feedback
  • Migration guidance questions
  • Data.gov Help questions
  • Known implementation blockers
  • Issues deferred from Cycle 1 or Cycle 2
  • Agency examples or edge cases

Retrospective Activities

Annual retrospective activities include:

  • Review harvest validation error trends for the year using New Relic data
  • Identify systemic field-level error patterns that may warrant schema clarification or guidance updates
  • Review recurring questions from agencies
  • Review common validation failures or warnings
  • Review documentation gaps identified during the year
  • Review issues filed in the DCAT-US GitHub repository
  • Review issues deferred from maintenance cycles
  • Review agency feedback from release outreach or maintenance meetings
  • Identify agency cohorts that may benefit from targeted implementation assistance
  • File issues as needed for schema clarification, documentation updates, FAQ updates, or implementation guidance updates
  • Document lessons learned
  • Identify priorities for the next annual maintenance cycle

Validation Trend Review

A core part of the retrospective is reviewing harvest validation error trends for the year.

The purpose of this review is to identify recurring validation issues that may indicate:

  • Field-level ambiguity
  • Documentation gaps
  • Common metadata formatting problems
  • Misalignment between schema behavior and implementation guidance
  • Common agency implementation challenges
  • Potential need for examples or FAQs
  • Potential need for schema clarification
  • Potential need for targeted implementation support

New Relic Data Review

New Relic data may be used to review harvest validation error trends and identify recurring patterns.

When reviewing New Relic data, the team may consider:

  • Which fields generate the most validation errors
  • Whether errors are concentrated among specific agency cohorts
  • Whether errors are linked to specific metadata patterns
  • Whether errors increased or decreased after a release
  • Whether new errors appeared after a schema or documentation change
  • Whether errors indicate a need for schema clarification
  • Whether errors indicate a need for documentation or examples
  • Whether targeted outreach may help agencies resolve common issues

Field-Level Error Pattern Analysis

The retrospective should identify systemic field-level error patterns that may warrant schema clarification or guidance updates.

Examples of field-level issues may include:

  • Required fields frequently missing
  • Values not matching expected formats
  • Confusion about controlled values or enums
  • Incorrect use of access or use restriction fields
  • Confusion between similar fields
  • Repeated structure or nesting errors
  • Distribution-level metadata issues
  • Dataset-level metadata issues
  • Errors related to accessRestriction, useRestriction, or cuiRestriction
  • Errors caused by unclear examples or documentation

When systemic error patterns are identified, the team should determine whether the response should be:

  • Documentation update
  • FAQ update
  • Implementation guide update
  • Example update
  • Schema clarification
  • GitHub issue
  • Agency outreach
  • Office hours topic
  • Future maintenance cycle issue

Documentation and Guidance Review

The retrospective should review whether documentation and guidance supported agency implementation effectively.

The team may consider:

  • Were implementation guide sections clear?
  • Were field definitions and usage notes sufficient?
  • Were examples accurate and validated against the current schema?
  • Were migration guidance materials sufficient?
  • Were recurring questions addressed in FAQs?
  • Were resources.data.gov pages aligned with the live schema?
  • Were release notes clear and actionable?
  • Were deprecation notices clear?
  • Did agencies know where to ask questions?
  • Did documentation updates occur in a timely way?

Documentation gaps should be filed as issues or assigned as follow-up tasks.

GitHub Issue Review

The retrospective should review GitHub issue activity from the year.

The team may consider:

  • Number of issues submitted
  • Types of issues submitted:
    • bug
    • enhancement
    • question
  • Number of issues closed
  • Number of issues deferred
  • Number of issues approved for PR
  • Number of issues that became documentation updates
  • Common themes across issues
  • Issues that took longer than expected to resolve
  • Issues that require future maintenance cycle consideration
  • Whether triage within two weeks was feasible
  • Whether labels and milestones were useful

Release Review

The retrospective should review the spring and summer release processes.

The team may consider:

  • Were release timelines realistic?
  • Were issues assigned to milestones appropriately?
  • Were pull requests reviewed on time?
  • Were documentation updates completed?
  • Were release notes published within two weeks of the release tag?
  • Were changelogs complete?
  • Were deprecation notices included when needed?
  • Was the 90-day grace period communicated clearly?
  • Were agencies able to understand whether action was required?
  • Were any release-related questions or blockers raised by agencies?

Agency Feedback Review

The retrospective should include review of agency feedback received during the year.

Sources may include:

  • Agency feedback meetings
  • Q&A Office Hours
  • Info Sharing Sessions
  • CDO Council touchpoints
  • Data.gov Help questions
  • Harvest source request communications
  • GitHub issues submitted by agencies
  • Direct agency implementation examples

The team may consider:

  • What worked well for agencies?
  • What was difficult or unclear?
  • Which fields caused confusion?
  • Which validation errors were hardest to resolve?
  • Whether release communications were clear
  • Whether timelines were reasonable
  • Whether agencies had enough time to implement changes
  • Whether additional examples or guidance would help
  • Whether targeted assistance is needed for specific agency cohorts

Annual Compliance Summary

The retrospective should result in an annual compliance summary identifying agency cohorts that may benefit from targeted implementation assistance.

The compliance summary is an implementation support tool.

It is intended to help identify:

  • Common implementation challenges
  • Agencies or agency cohorts that may benefit from targeted assistance
  • Field-level error patterns
  • Documentation gaps
  • Areas where additional examples, FAQs, or guidance may be needed
  • Opportunities for future office hours or information sharing topics

The compliance summary is not an enforcement document.

Compliance Summary Content

The annual compliance summary may include:

  • Overview of the annual review period
  • Summary of validation trends
  • Common field-level error patterns
  • Common agency implementation blockers
  • Documentation or guidance gaps
  • Agency cohorts that may benefit from targeted support
  • Recommended support activities
  • Recommended documentation updates
  • Recommended FAQ updates
  • Recommended GitHub issues or maintenance priorities
  • Lessons learned
  • Priorities for the next annual cycle

Annual Compliance Summary Template

Use the following template for the annual compliance summary.

# Annual Compliance Support Summary

## Review Period

Start date:  
End date:

## Purpose

This summary identifies implementation support needs related to DCAT-US v3.0. It is intended as an implementation support tool and is not an enforcement document.

## Data Sources Reviewed

- Harvest validation trends
- New Relic data
- GitHub issues
- Office hours questions
- Info sharing session topics
- Agency feedback
- Documentation issues
- Data.gov Help questions

## Summary of Key Findings

Summarize the most important findings from the review period.

## Validation Trends

Describe recurring validation errors or patterns.

## Field-Level Error Patterns

List fields or schema areas with recurring issues.

## Agency Implementation Blockers

Summarize common blockers identified by agencies.

## Documentation and Guidance Gaps

Identify needed documentation, FAQ, migration guidance, or example updates.

## Agency Cohorts for Targeted Support

Identify agency groups that may benefit from targeted assistance.

## Recommended Support Activities

Examples:
- Office hours topic
- Info sharing session topic
- Direct outreach
- Documentation update
- FAQ update
- Validation guidance
- Migration guidance

## Recommended GitHub Issues or Maintenance Actions

List issues to file or prioritize.

## Lessons Learned

Document process or implementation lessons learned.

## Priorities for Next Annual Cycle

List recommended priorities for the next annual maintenance cycle.

Lessons Learned

The retrospective should document lessons learned about both DCAT-US v3.0 implementation and the maintenance process itself.

Lessons learned may address:

  • Schema clarity
  • Documentation quality
  • Validation behavior
  • Release timing
  • Agency outreach
  • Grace period communication
  • Issue triage
  • GitHub workflow
  • Agency feedback meetings
  • Office hours effectiveness
  • Info sharing session effectiveness
  • Data.gov team coordination
  • Maintenance team decision-making

Next-Cycle Priorities

The retrospective should identify priorities for the next annual maintenance cycle.

Priorities may include:

  • Fields needing clarification
  • Documentation sections needing updates
  • Examples needing validation or revision
  • FAQs to develop
  • Schema issues to review
  • Deprecation notices to consider
  • Agency cohorts for targeted support
  • Release process improvements
  • Meeting cadence adjustments
  • Updates to maintenance templates
  • Improvements to issue triage workflow

Retrospective Meeting Agenda

A typical annual retrospective meeting may use the following agenda.

1. Welcome and Objectives

  • Confirm purpose of the retrospective
  • Review desired outputs
  • Confirm agenda

2. Review of Maintenance Year

  • Review Cycle 1 activities
  • Review Cycle 2 activities
  • Review release outcomes
  • Review major schema or documentation updates

3. Validation Trend Review

  • Review harvest validation error trends
  • Review New Relic data
  • Identify recurring patterns

4. Agency Feedback Review

  • Summarize agency feedback from meetings, office hours, GitHub issues, and support channels
  • Identify recurring blockers or questions

5. Documentation and Guidance Review

  • Identify documentation gaps
  • Identify needed examples, FAQs, or migration guidance

6. Issue and Release Process Review

  • Review GitHub issue workflow
  • Review triage timing
  • Review milestone use
  • Review release communications

7. Compliance Support Summary

  • Discuss agency cohorts that may benefit from targeted implementation assistance
  • Confirm summary findings

8. Lessons Learned

  • Discuss what worked well
  • Discuss what should be improved

9. Next-Cycle Priorities

  • Identify priorities for the next annual maintenance cycle
  • Assign follow-up actions

10. Wrap-Up

  • Confirm action items
  • Confirm owners
  • Confirm timeline for publishing or sharing retrospective outputs

Retrospective Outputs

Expected outputs from the annual retrospective include:

  • Annual retrospective notes
  • Annual compliance support summary
  • List of systemic validation or implementation issues
  • GitHub issues filed for schema clarification or guidance updates
  • Documentation update tasks
  • FAQ update tasks
  • Recommended agency support activities
  • Lessons learned
  • Priorities for the next annual maintenance cycle
  • Updates to maintenance process documentation, if needed

Follow-Up Actions

After the retrospective, the Data.gov team and DCAT Interagency Maintenance Team should confirm follow-up actions.

Follow-up actions may include:

  • File new GitHub issues
  • Update existing GitHub issues
  • Add issues to future milestones
  • Update documentation
  • Update FAQs
  • Plan targeted office hours topics
  • Plan info sharing session topics
  • Conduct targeted agency outreach
  • Update migration guidance
  • Revise maintenance templates
  • Revise meeting cadence or participant list
  • Share retrospective findings with relevant stakeholders

Retrospective Checklist

Use the following checklist to support annual retrospective planning.

- [ ] Retrospective meeting scheduled.
- [ ] Participants identified.
- [ ] Harvest validation trend data collected.
- [ ] New Relic data reviewed or prepared.
- [ ] GitHub issue summary prepared.
- [ ] Documentation issue summary prepared.
- [ ] Office hours question themes summarized.
- [ ] Info sharing session themes summarized.
- [ ] Agency feedback summarized.
- [ ] Release notes and outreach timing reviewed.
- [ ] Deferred issues reviewed.
- [ ] Draft compliance support summary prepared.
- [ ] Lessons learned discussion prepared.
- [ ] Next-cycle priority discussion prepared.
- [ ] Follow-up action tracker prepared.

Relationship to Other Maintenance Pages

The annual retrospective connects to several other maintenance activities.

Related Page Relationship
Maintenance Process Overview Describes where the retrospective fits within the annual maintenance cycle
Semi-Annual Release Cycles Describes Cycle 2 and the July retrospective timing
Issue Triage and GitHub Workflow Retrospective findings may result in new GitHub issues or triage improvements
Agency Feedback and Participation Agency feedback is a key input to the retrospective
Release Communications and Outreach Retrospective reviews whether communications were clear and timely
Maintenance Meeting Schedule Includes the annual retrospective meeting
Maintenance Templates Includes templates for retrospective summaries and compliance support summaries

Related Resources

DCAT-US GitHub Issues

https://github.com/GSA/dcat-us/issues

Use this issue tracker to request or discuss changes to the DCAT-US metadata standard itself.

DCAT-US GitHub Repository

https://github.com/GSA/dcat-us

Repository for the DCAT-US metadata standard.

DCAT-US 3 Implementation Guide

https://resources.data.gov/resources/dcat-us-3-implementation/

Primary reference for agencies implementing the DCAT-US 3 metadata standard.

DCAT-US 3 Technical Implementation FAQs

https://github.com/GSA/dcat-us/wiki/DCAT-US-3-Technical-Implementation-FAQ

Frequently asked questions related to technical implementation of the DCAT-US 3 metadata standard are posted here as they are developed.

DCAT-US 3 Updates

https://resources.data.gov/resources/dcat-us-3-updates/

Use this page for meeting schedule updates and related DCAT-US 3 engagement information.

Current Harvest Sources

https://harvest.data.gov

Use this site to view current agency harvest sources.

Harvest Source Validation Tool

https://harvest.data.gov/validate/

Use this tool to test whether a proposed or updated harvest source is expected to pass validation.

Data.gov Help

Questions about Data.gov harvesting or resources.data.gov documentation can be directed to:

datagovhelp@gsa.gov

Related Wiki Pages

DCAT-US 3 Implementation Support

Meetings

Implementation Resources

Data.gov Team Process

FAQs

Reference

DCAT Interagency Maintenance Team

Clone this wiki locally