From a3e819f8856eac83777424d537a201e5e3062d28 Mon Sep 17 00:00:00 2001 From: Eric Gundersen <85637090+gundersen-lumn@users.noreply.github.com> Date: Mon, 20 Jul 2026 09:25:12 -0600 Subject: [PATCH 1/6] [AI-ASSISTED] Test: Copilot test commit From 14d21c94030e44e998a4498b164f92a36b634736 Mon Sep 17 00:00:00 2001 From: Eric Gundersen <85637090+gundersen-lumn@users.noreply.github.com> Date: Mon, 20 Jul 2026 09:57:52 -0600 Subject: [PATCH 2/6] [AI-ASSISTED] Add AI adoption tracking with git hook and update documentation - Introduced a new git hook at `.githooks/prepare-commit-msg` to auto-tag AI-assisted commits. - Enhanced `create-copilot-instructions.yml` to include setup instructions for AI adoption tracking. - Updated `README.md` to document the AI adoption tracking system and its integration with existing templates. --- .githooks/prepare-commit-msg | 46 ++++++ .../comprehensive-codebase-review.yml | 36 ++++- .../create-copilot-instructions.yml | 144 ++++++++++++++++++ README.md | 111 ++++++++++++++ 4 files changed, 330 insertions(+), 7 deletions(-) create mode 100755 .githooks/prepare-commit-msg diff --git a/.githooks/prepare-commit-msg b/.githooks/prepare-commit-msg new file mode 100755 index 0000000..f6ddb91 --- /dev/null +++ b/.githooks/prepare-commit-msg @@ -0,0 +1,46 @@ +#!/bin/bash +# AI Adoption Signal Detection Hook +# Auto-tags commits assisted by GitHub Copilot or other AI tools +# Idempotent: safe to run on every commit + +COMMIT_MSG_FILE=$1 +COMMIT_SOURCE=$2 + +# Skip if commit is from merge, squash, rebase, or other special cases +if [ "$COMMIT_SOURCE" = "merge" ] || [ "$COMMIT_SOURCE" = "squash" ] || [ "$COMMIT_SOURCE" = "rebase" ]; then + exit 0 +fi + +# Read current commit message +COMMIT_MSG=$(cat "$COMMIT_MSG_FILE") + +# Check for AI signals in multiple places +# Signal 1: GitHub Copilot co-author metadata +HAS_COPILOT_COAUTHOR=$(echo "$COMMIT_MSG" | grep -i "co-authored-by.*github copilot" | wc -l) + +# Signal 2: Explicit AI markers at start of message +HAS_COPILOT_MARKER=$(echo "$COMMIT_MSG" | grep -iE "^(copilot|ai-assisted|@copilot):" | wc -l) + +# Signal 3: AI tool mentions in commit body +HAS_AI_PATTERN=$(echo "$COMMIT_MSG" | grep -iE "(copilot|claude|gpt|cursor|ai-assisted)" | wc -l) + +# Idempotent check: don't re-tag if already tagged +if echo "$COMMIT_MSG" | head -1 | grep -q "^\[AI-ASSISTED\]"; then + exit 0 +fi + +# Auto-tag if any AI signal detected +if [ "$HAS_COPILOT_COAUTHOR" -gt 0 ] || [ "$HAS_COPILOT_MARKER" -gt 0 ] || [ "$HAS_AI_PATTERN" -gt 0 ]; then + FIRST_LINE=$(echo "$COMMIT_MSG" | head -1) + REST=$(echo "$COMMIT_MSG" | tail -n +2) + + # Prepend tag to first line (idempotent: already checked above) + if [ -z "$REST" ]; then + echo "[AI-ASSISTED] $FIRST_LINE" > "$COMMIT_MSG_FILE" + else + echo "[AI-ASSISTED] $FIRST_LINE" > "$COMMIT_MSG_FILE" + echo "$REST" >> "$COMMIT_MSG_FILE" + fi +fi + +exit 0 diff --git a/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml b/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml index 3bb570b..18ed5ce 100644 --- a/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml +++ b/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml @@ -157,8 +157,8 @@ body: - scoring.md — tabular scoring with 1-10 ratings and justifications. - architecture.mmd — Mermaid diagram source for the architecture. - improvement-issues.md — synopsis of open improvement-focused GitHub issues with solution steps. - - ai-adoption.md — explicit AI adoption analysis for the last 6 months with an executive judgment (low/moderate/high), score, shipped-to-master section, contributor/subsystem adoption analysis, non-adopter table, next steps, and caveats. - - ai-adoption.csv — commit-level explicit AI-related commits in the last 6 months with columns: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file. + - ai-adoption.md — explicit AI adoption analysis for the last 6 months with an executive judgment (low/moderate/high), score, shipped-to-master section, contributor/subsystem adoption analysis, **squashed commits analysis**, non-adopter table, next steps, and caveats. + - ai-adoption.csv — commit-level explicit AI-related commits in the last 6 months with columns: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file, squash_merge_indicator, signal_type (explicit_keyword | github_metadata | git_hook | implicit_pattern). - ai-adoption-by-developer.csv — developer aggregation from ai-adoption.csv with columns: developer, total_ai_commits, ai_commits_on_master, ai_commits_off_master, master_commit_share_pct. - chi-compliance.md — generated only when the repository includes a Vue-based UI; * FIRST, check index.html for Chi CDN references (CSS/JS URLs) - this is the authoritative Chi version @@ -177,21 +177,43 @@ body: - AI Adoption Report Requirements (last 6 months, output to copilot-eval): * Analyze commits from now minus 6 months through today. * Include all refs for contribution analysis, and separately analyze what reached master. - * Treat explicit AI-related commits as commit subjects containing (case-insensitive): copilot, @Copilot, ai-assisted, gpt, claude, cursor. + * **Signal Detection Strategy** (multiple layers to reduce developer friction): + 1. Explicit AI Keywords (optional, zero friction if auto-tagged): Treat commit subjects containing (case-insensitive): copilot, @Copilot, ai-assisted, gpt, claude, cursor. + 2. GitHub Copilot Attribution Metadata: If available, detect commits tagged with GitHub's AI-generated indicator (via GitHub API or webhook metadata). + 3. Git Hooks (recommended for ease-of-use): If the repo includes a `.githooks/prepare-commit-msg` hook or similar, document its presence and how it auto-tags AI commits without developer intervention. + 4. IDE/CLI Integration: Note if the codebase uses Copilot CLI or IDE extensions that auto-append AI indicators at commit time (no developer action required). + 5. Implicit Patterns (fallback): Scan commit message structure for AI-generated patterns (e.g., structured formatting, typical Copilot phrasing, co-authored metadata like "Co-authored-by: GitHub Copilot"). + 6. Squashed Commit Recovery: Extract AI signals from individual commits inside squashed merges using all methods above. + * Recommended Setup for Minimal Developer Friction: + - Use a git hook (prepare-commit-msg or commit-msg) to auto-detect and auto-tag AI commits at push time. + - Example: `if git diff --cached | grep -q "^+.*TODO\|FIXME\|DEBUG"` or integrate with Copilot CLI for automatic detection. + - Developers do nothing; the system tags automatically. + - Provide a `.githooks/prepare-commit-msg` template in the repo README for teams to adopt. + * If no automation in place, fall back to explicit keyword detection; but acknowledge this undercounts adoption. * Treat on_master as commit is an ancestor of master. * Treat content commit as files_changed > 0 and merge-only commit as files_changed = 0. * Gather baseline counts: total commits in 6 months (all refs) and total commits in 6 months on master. - * Build ai-adoption.csv with one row per explicit AI-related commit and columns exactly: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file. + * **Squashed Commit Inspection** (critical for accurate adoption metrics): + - For each merge commit on master, detect if it is a squash-merge (single parent with multiple logical commits collapsed into one). + - Use `git log ^.. --pretty=format:"%H|%an|%ae|%s|%b"` to inspect the individual commits that were squashed before merge. + - Scan all squashed individual commits for AI signals (explicit keywords, metadata, patterns) in subject and body; if any contain AI signals, attribute the AI work to the original committers (not the merge author). + - Extract files_changed by analyzing the diff of each squashed commit: `git diff ^.. --name-only | wc -l`. + - For squashed commits with AI signals, preserve the original author/date and create a logical "unmerged" entry in ai-adoption.csv with the source commits listed in a supplementary column. + - If PR metadata is available (GitHub API), cross-reference the PR number from the merge commit message to retrieve the full commit history before squashing. + - Document merge strategy (squash vs. fast-forward vs. three-way merge) in the Caveats section and note any adoption metrics recovered from squashed commits. + * Build ai-adoption.csv with one row per explicit AI-related commit (including unmerged squashed commits) and columns exactly: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file, squash_merge_indicator (true/false), signal_type (explicit_keyword | github_metadata | git_hook | implicit_pattern). * Compute normalized_author by merging obvious aliases via shared email identity and clear name/username variants (e.g., corporate name vs GitHub handle), and document normalization caveat in markdown. * Build ai-adoption-by-developer.csv from ai-adoption.csv with columns exactly: developer, total_ai_commits, ai_commits_on_master, ai_commits_off_master, master_commit_share_pct. * Detect active contributors with zero explicit AI signal: all active contributors in 6 months minus normalized authors in ai-adoption.csv, and include commit counts for this group. * Avoid brittle one-liner heredocs in terminal; if scripting is needed, create small temporary scripts/files, run them, and clean up helper artifacts afterward. - * ai-adoption.md must include: Executive Judgment (moderate/low/high) with rationale; Method with exact filter terms, timeframe, and lower-bound caveat; section #2 What Actually Shipped To master with counts and table (Date, Author, SHA short, Subject, Dominant subsystem, Files changed); section #3 Developer And Subsystem Adoption with normalized contributor counts and subsystem distribution; section All AI Contributions Per Developer sourced from ai-adoption-by-developer.csv (Developer, Total AI commits, AI commits on master, AI commits off master, master share); section Developers With 0 Explicit AI Assistance (Developer, six-month commit count, recommendation priority); section Next Steps; and section Caveats. - * Quality bar: markdown numbers must reconcile with both CSVs; use normalized identities for developer reporting; distinguish all AI contributions vs merged-to-master AI contributions; remove temporary helper artifacts; validate outputs are readable and error-free. + * ai-adoption.md must include: Executive Judgment (moderate/low/high) with rationale; Method with exact filter terms, timeframe, lower-bound caveat, **merge strategy discovery notes**, and **detection strategy used** (explicit keywords only vs. multi-signal); section #2 What Actually Shipped To master with counts and table (Date, Author, SHA short, Subject, Dominant subsystem, Files changed, Squash Indicator, Signal Type); section #3 Developer And Subsystem Adoption with normalized contributor counts and subsystem distribution; section All AI Contributions Per Developer sourced from ai-adoption-by-developer.csv (Developer, Total AI commits, AI commits on master, AI commits off master, master share); section Developers With 0 Explicit AI Assistance (Developer, six-month commit count, recommendation priority); **section Squashed Commits Analysis** with summary of recovered AI signals, original vs. merge authors, and files impacted; section Setup Recommendations (e.g., "Add git hooks for zero-friction auto-tagging"; section Next Steps; and section Caveats. + * Quality bar: markdown numbers must reconcile with both CSVs and squashed commit inspection logs; use normalized identities for developer reporting; distinguish all AI contributions vs merged-to-master AI contributions; account for squashed commits when computing adoption rate; remove temporary helper artifacts; validate outputs are readable and error-free. - For Vue-based UI repositories, expand the analysis phase to review template markup across all views, pages, and components against Chi component guidance (web components or HTML boilerplate) and describe non-compliant patterns in chi-compliance.md. - When reporting Chi compliance, treat the version declared by the CDN-style script or stylesheet reference in the primary HTML entrypoint (for example, `