feature/git2gen3

[bwalsh]

---

🚧 Synchronize GitHub Repository Files with Portal (Optional: Permissions)

🧾 User Story

As a portal user and data steward,
I want to synchronize files and metadata from a GitHub repository (including Git LFS files) into the portal whenever a designated branch is updated,
And optionally synchronize user permissions based on GitHub repository roles,
So that file visibility and access control in the portal align with the source of truth in GitHub.

---

✅ Acceptance Criteria

🔁 File Sync

• When a designated branch (e.g., main or release) is updated, the portal:

  • Pulls the latest Git LFS pointers.
  • Extracts associated metadata (e.g., from .lfs-meta/*.json).
  • Displays files and metadata in the portal’s UI.

• Git LFS files are registered in the portal (e.g., as DocumentReference or file entries).

• File paths and metadata remain accurate and up to date.

🟡 Optional: Permission Sync

• If enabled, the portal synchronizes user access permissions with GitHub:

  • Users with write or admin access to the GitHub repo are granted edit/upload permissions in the portal.
  • Users with read access are granted view-only permissions.

• Permission sync occurs on a scheduled basis or on demand.

• Changes to GitHub team or collaborator roles are reflected within a configurable SLA (e.g., ≤1 hour).

---

🏗️ Technical Design

📁 GitHub Repository File Sync

Trigger:

• GitHub webhook on push to a specific branch, or scheduled polling (e.g., cron job)

Steps:

1. Clone the repository and pull Git LFS files:

   git clone --branch=main --depth=1 https://github.com/org/repo.git
   git lfs pull

   Create Gen3 Project if it doesn't exist

2. Detect Git LFS pointer files:

   version https://git-lfs.github.com/spec/v1
   oid sha256:...
   size ...
   

3. Load .lfs-meta/*.json files (if present) to extract metadata:

   {
     "drs_id": "drs:ohsu.org/abc123",
     "patient": "patient:OH123",
     "specimen": "specimen:SP456"
   }

4. Register or update:

   • DocumentReference using create_object_id(path, project_id)
   • Related FHIR resources (e.g., Patient, Specimen) from META/

---

🟡 Optional: GitHub Permissions Sync

Trigger:

• Scheduled GitHub API query (e.g., every 15–60 minutes) or admin-triggered action

Steps:

1. Query repository collaborators:

   gh api repos/<owner>/<repo>/collaborators --jq '.[] | {login, permissions}'

2. Map permissions:

   • admin or write → portal edit access
   • read → portal view-only access

3. Update portal IAM or Arborist policies accordingly

---

🔄 Optional: Webhook Handler or GitHub Action

🪝 Webhook Handler Script

• Python (FastAPI or Flask)
• Listens for GitHub push events
• Clones repo, pulls LFS, parses metadata, syncs with portal backend

🛠️ GitHub Action Example

name: Sync with Portal

on:
  push:
    branches: [ main ]

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout with LFS
        uses: actions/checkout@v3
        with:
          lfs: true

      - name: Sync metadata to portal
        run: |
          pip install gen3_util
          python scripts/sync_to_portal.py --project aced-ohsu --branch main

---

📦 Integration Summary

Component	Integration Target
Git LFS Files	DocumentReference
.lfs-meta/*.json	Patient, Specimen, etc.
Project Namespace	Derived from GitHub org/repo
GitHub Permissions	Portal IAM / Arborist (optional)

---

📝 Tasks

• Implement Git LFS file detection and metadata sync
• (Optional) Implement GitHub permission mapping logic
• Provide webhook or GitHub Action integration
• Document how to configure branch and repo sync