Skip to content

Feature: GitHub metadata import for OpenCite - #31

Merged
beanbean9339 merged 123 commits into
mainfrom
feature/github-metadata-import
Sep 3, 2026
Merged

Feature: GitHub metadata import for OpenCite#31
beanbean9339 merged 123 commits into
mainfrom
feature/github-metadata-import

Conversation

@beanbean9339

@beanbean9339 beanbean9339 commented Jun 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR introduces GitHub repository metadata import and improves OpenCite's metadata processing and validation workflow.

Users can now provide a GitHub repository URL and automatically import available repository metadata into OpenCite. Imported metadata is normalized into OpenCite's internal metadata model, allowing users to review and edit information before validating and generating standardized citation metadata outputs.

This PR improves the end-to-end repository-to-citation workflow through GitHub API integration, metadata normalization, citation validation, author handling, export consistency, and improved documentation.

Repository provenance files and licensing metadata (CITATION.cff, .zenodo.json, LICENSE, and CI workflow changes) are maintained separately in the provenance metadata PR.


Scope of Changes

GitHub Metadata Import

  • Added GitHub URL-based metadata importing.
  • Added GitHub API integration to retrieve repository metadata.
  • Added metadata normalization to map GitHub data into OpenCite's internal metadata model.
  • Supports importing:
    • repository name/project title
    • description
    • repository URL and homepage
    • license information
    • topics/keywords
    • owner and contributor information
    • release/version metadata
  • Added support for reading existing repository citation metadata when available.
  • Documented that repository citation metadata import currently uses the repository default branch.

Metadata Pipeline Improvements

  • Unified metadata handling across:
    • manual metadata entry
    • GitHub repository imports
    • citation metadata generation
  • Refactored metadata processing into clearer stages:
    • GitHub API fetching
    • metadata transformation and normalization
    • validation
    • export formatting
  • Improved handling of incomplete or missing metadata.

Author and Citation Improvements

  • Added structured author metadata support.
  • Added ORCID validation and normalization.
  • Improved author editing workflows, including:
    • author reordering
    • author suggestions
    • contributor fallback handling
  • Added contributor fallback handling for repositories without complete citation author information.
  • Improved duplicate author detection and author consistency checks.

Validation and Citation Health Improvements

  • Improved required field guidance and validation messaging.
  • Added citation health checks for:
    • version consistency
    • repository URL consistency
    • license consistency
    • release information
    • ORCID metadata
    • DOI expectations
  • Reduced false-positive warnings by comparing normalized metadata values directly.
  • Improved handling of missing versus invalid metadata.

Export Improvements

  • Improved consistency between generated citation metadata outputs.
  • Added support for:
    • references
    • publication metadata
    • release identifiers
    • software-specific metadata fields
  • Improved export/download workflows.
  • Fixed .zenodo.json downloads to preserve the required leading period in the filename.

Documentation and Project Improvements

  • Updated contributor documentation and project guidance.
  • Added GitHub metadata import documentation and usage examples.
  • Added deployment information for the GitHub Pages application.
  • Documented known issues and current metadata validation limitations.

How It Works

  1. User provides a GitHub repository URL.
  2. OpenCite retrieves available repository metadata through the GitHub API.
  3. Imported information is normalized into OpenCite's metadata model.
  4. Users review, edit, and validate the imported metadata.
  5. OpenCite generates standardized citation metadata outputs:
    • CITATION.cff
    • .zenodo.json

Testing

Tested using repositories with:

  • complete citation metadata
  • missing descriptions
  • missing licenses
  • incomplete topics/keywords
  • different contributor and author configurations

Verified:

  • GitHub metadata import
  • metadata normalization
  • validation behavior
  • citation health checks
  • ORCID handling
  • citation metadata generation
  • export workflows

Automated Testing

  • npm run test
    • 50 tests passing
  • npm run build
    • Production build successful

Known Limitations

  • GitHub API rate limits may affect imports requiring additional metadata lookups.
  • Some repositories may not contain complete citation metadata, requiring fallback handling.
  • GitHub contributors and citation authors represent different concepts and may not always map one-to-one.
  • Citation health checks rely on available repository/archive metadata and cannot infer unavailable information.
  • DOI checks depend on available Zenodo/archive signals.
  • Field-level metadata provenance is not currently exposed in the UI.
  • CITATION.cff parsing supports common structures but does not fully model every advanced CFF feature.

Reviewer Notes

This PR focuses on improving the workflow:

GitHub repository → OpenCite metadata model → validation → citation metadata generation

The main areas for review are:

  • GitHub import behavior and metadata precedence
  • metadata normalization logic
  • validation and citation health accuracy
  • author and contributor handling
  • export consistency
  • documentation updates

Provenance metadata files and repository configuration changes have been separated into the dedicated provenance metadata PR.

beanbean9339 and others added 30 commits May 29, 2026 17:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 38 out of 39 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

package.json:13

  • zenodraft@0.14.1 (added below) requires Node >= 20. Adding an explicit engines.node entry helps avoid confusing install/runtime failures for contributors/CI on older Node versions when running npm run validate:metadata.
  },
  "dependencies": {

src/services/citation.js:180

  • citationMessage is keyed off metadata.references.length, but the actual emitted references are based on the sanitized references array. If metadata.references contains only blank/invalid entries, the message can mention “associated references” even though no references: section is emitted.
  const hasReferences = Array.isArray(metadata.references) && metadata.references.length > 0;

src/components/MetadataForm.jsx:267

  • The UI doesn’t mark Publication date as required, but validateMetadata treats publicationDate as required. This mismatch can confuse users about why export is blocked.
            <span>Publication date</span>

Comment thread src/services/githubApi.js Outdated

@mandylan75 mandylan75 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! It seems like majority of the functionality is working.

I tested a couple repos:

For repo-exporter, I got a few import warnings, but I think those are fine since they're addressed in the Citation health scan, which is a helpful feature that I love! For the authors, are they supposed to be in the order that they are in `CITATION.cff`` ? If so, there might be an ordering issue since it goes from Elizabeth, Amanda, and Balaji.

Image Image

I also received a lot of console errors, but I think you've already implemented a solution for them in the refactoring PR since this is an older PR.

Image

For sum-buddy:

Almost everything seems to be running good, with a few minor observations that I got confused on. There are duplicates for Matt, although there is one with "J." while the other one doesn't and I'm not sure where this duplicate is coming from. There is just "Matthew J. Thompson" in CITATION.cff, but there's no 2 Matts listed as contributors separately in the repo. OpenCite was able to catch Nipun as a contributor and added that as s preview at the very bottom when they weren't listed in CITATION.cff, which is nice. It also added Hilmar in the preview when he wasn't present as one of the contributors.

Image Image Also got console errors.. Image

catalog:

There's the same issue with how Emerson has duplicated like Matt from above, and I'm not sure where that's coming from either. I'm assuming that Matt's username was different to how his name was listed in CITATION.cff but Emersons name is the same in CITATION.cff and his username. There are also console errors.

Image Image

if (!repoUrl && !metadataRepoUrl) {
return buildCheck(
'warning',
'Repository URL is current',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted offline, these warning headings could use an update for clarity. You already fixed the ORCID one in #31, but this, authors, version, dates, etc. should be revised for clarity.

@beanbean9339
beanbean9339 merged commit c4b7253 into main Sep 3, 2026
@beanbean9339
beanbean9339 deleted the feature/github-metadata-import branch September 3, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants