Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 145 additions & 0 deletions .github/workflows/collect-metrics.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
name: Collect GitHub Metrics

on:
# Run weekly on Mondays at 00:00 UTC
schedule:
- cron: '0 0 * * 1'
# Allow manual trigger
workflow_dispatch:

permissions:
contents: write

jobs:
collect-metrics:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
# Use default fetch for faster checkout
fetch-depth: 1
# Ensure we're on the latest main branch
ref: main

- name: Setup GitHub CLI
run: |
# gh is pre-installed on GitHub Actions runners
# Use METRICS_GITHUB_TOKEN if available (for traffic data), fallback to GITHUB_TOKEN
TOKEN="${{ secrets.METRICS_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}"
echo "$TOKEN" | gh auth login --with-token

- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "0.9.11"
enable-cache: true

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'

- name: Install Python dependencies
run: uv pip install --system pyyaml

- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"

- name: Ensure data directory exists
run: mkdir -p catalog/public/data

- name: Collect GitHub metrics
id: collect
run: |
python scripts/collect_github_metrics.py
if [ $? -ne 0 ]; then
echo "error=true" >> $GITHUB_OUTPUT
exit 1
fi
echo "error=false" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.METRICS_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}

- name: Validate JSON files
run: |
echo "Validating JSON files..."
python -c "import json; json.load(open('catalog/public/data/github_metrics.json'))"
python -c "import json; json.load(open('catalog/public/data/github_metrics_history.json'))"
echo "✓ JSON files are valid"

- name: Check for changes
id: check_changes
run: |
git add catalog/public/data/github_metrics.json
git add catalog/public/data/github_metrics_history.json
if git diff --cached --quiet; then
echo "changed=false" >> $GITHUB_OUTPUT
echo "No changes to commit"
else
echo "changed=true" >> $GITHUB_OUTPUT
echo "Changes detected, will commit"
fi

- name: Commit and push metrics data
if: steps.check_changes.outputs.changed == 'true'
run: |
git commit -m "chore: update GitHub metrics data

🤖 Automated weekly metrics collection

- Updated current metrics snapshot
- Added to historical metrics database

Generated by: ${{ github.workflow }}
Run ID: ${{ github.run_id }}"

# Push with retry logic
for i in {1..3}; do
if git push origin main; then
echo "✓ Successfully pushed changes"
break
else
if [ $i -lt 3 ]; then
echo "Push failed, retrying in 5 seconds... (attempt $i/3)"
sleep 5
git pull --rebase origin main
else
echo "Failed to push after 3 attempts"
exit 1
fi
fi
done

- name: Create summary
if: always()
run: |
echo "## GitHub Metrics Collection Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Workflow Run:** \`${{ github.workflow }}\`" >> $GITHUB_STEP_SUMMARY
echo "**Trigger:** \`${{ github.event_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "**Date:** \`$(date -u)\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY

if [ "${{ steps.collect.outputs.error }}" == "true" ]; then
echo "**Status:** ❌ Collection failed" >> $GITHUB_STEP_SUMMARY
exit 0
fi

if [ -f catalog/public/data/github_metrics.json ]; then
REPO_COUNT=$(python -c "import json; data=json.load(open('catalog/public/data/github_metrics.json')); print(len(data.get('repos', {})))")
echo "**Repositories Tracked:** $REPO_COUNT" >> $GITHUB_STEP_SUMMARY

LAST_UPDATED=$(python -c "import json; data=json.load(open('catalog/public/data/github_metrics.json')); print(data.get('last_updated', 'Unknown'))")
echo "**Last Updated:** \`$LAST_UPDATED\`" >> $GITHUB_STEP_SUMMARY
fi

if [ "${{ steps.check_changes.outputs.changed }}" == "true" ]; then
echo "**Status:** ✅ Metrics updated and committed" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "The deployment workflow will automatically trigger to publish the updated analytics." >> $GITHUB_STEP_SUMMARY
else
echo "**Status:** ℹ️ No changes detected" >> $GITHUB_STEP_SUMMARY
fi
9 changes: 9 additions & 0 deletions .github/workflows/deploy-catalog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ on:
- 'scripts/sync_repositories_to_json.py'
- '.github/workflows/deploy-catalog.yml'
workflow_dispatch:
# Auto-trigger when metrics are updated
workflow_run:
workflows: ["Collect GitHub Metrics"]
types:
- completed
branches:
- main

permissions:
contents: write
Expand All @@ -21,6 +28,8 @@ concurrency:
jobs:
build-and-deploy:
runs-on: ubuntu-latest
# Only deploy if metrics collection succeeded (or if triggered by push/manual)
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
steps:
- name: Checkout
uses: actions/checkout@v6
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ repos:
hooks:
- id: typos
args: []
exclude: ^(catalog/public/data/repositories.json|catalog/public/data/papers\.bib|repositories/crisp-nam\.yaml)$
exclude: ^(catalog/public/data/repositories.json|catalog/public/data/papers\.bib|catalog/public/data/github_metrics.json|catalog/public/data/github_metrics_history.json|repositories/crisp-nam\.yaml)$

- repo: local
hooks:
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,18 @@ npm install
npm run dev
```

## 📊 Analytics Dashboard

Executive dashboard for catalog performance metrics:
- **Key Metrics**: Aggregate stats across all repositories
- **Top Performers**: Highest starred, most visited, and most cloned repos
- **Complete Overview**: Sortable table of all repositories
- **Auto-Update**: Weekly collection via GitHub Actions (Mondays at 00:00 UTC)

**Manual Collection**: `python scripts/collect_github_metrics.py` (requires `gh` CLI)

> **Note**: Traffic data (views/clones) requires a Personal Access Token with `repo` permissions. Add as `METRICS_GITHUB_TOKEN` secret in repository settings. Without it, traffic metrics will show "—" but basic metrics (stars, forks) still work.

## 📋 Repository Information

The catalog contains the following information for each implementation:
Expand Down
Loading