diff --git a/.githooks/prepare-commit-msg b/.githooks/prepare-commit-msg new file mode 100755 index 0000000..b80954a --- /dev/null +++ b/.githooks/prepare-commit-msg @@ -0,0 +1,46 @@ +#!/bin/bash +# AI Adoption Signal Detection Hook +# Auto-tags commits assisted by GitHub Copilot or other AI tools +# Idempotent: safe to run on every commit + +COMMIT_MSG_FILE=$1 +COMMIT_SOURCE=$2 + +# Skip if commit is from merge or squash (auto-generated commit messages) +if [ "$COMMIT_SOURCE" = "merge" ] || [ "$COMMIT_SOURCE" = "squash" ]; then + exit 0 +fi + +# Read current commit message +COMMIT_MSG=$(cat "$COMMIT_MSG_FILE") + +# Check for AI signals in multiple places +# Signal 1: GitHub Copilot co-author metadata +HAS_COPILOT_COAUTHOR=$(echo "$COMMIT_MSG" | grep -qi "co-authored-by.*github copilot" && echo 1 || echo 0) + +# Signal 2: Explicit AI markers at start of message +HAS_COPILOT_MARKER=$(echo "$COMMIT_MSG" | grep -qiE "^(copilot|ai-assisted|@copilot):" && echo 1 || echo 0) + +# Signal 3: AI tool mentions in commit body +HAS_AI_PATTERN=$(echo "$COMMIT_MSG" | grep -qiE "(copilot|claude|gpt|cursor|ai-assisted)" && echo 1 || echo 0) + +# Idempotent check: don't re-tag if already tagged +if echo "$COMMIT_MSG" | head -1 | grep -q "^\[AI-ASSISTED\]"; then + exit 0 +fi + +# Auto-tag if any AI signal detected +if [ "$HAS_COPILOT_COAUTHOR" -gt 0 ] || [ "$HAS_COPILOT_MARKER" -gt 0 ] || [ "$HAS_AI_PATTERN" -gt 0 ]; then + FIRST_LINE=$(echo "$COMMIT_MSG" | head -1) + REST=$(echo "$COMMIT_MSG" | tail -n +2) + + # Prepend tag to first line (idempotent: already checked above) + if [ -z "$REST" ]; then + echo "[AI-ASSISTED] $FIRST_LINE" > "$COMMIT_MSG_FILE" + else + echo "[AI-ASSISTED] $FIRST_LINE" > "$COMMIT_MSG_FILE" + echo "$REST" >> "$COMMIT_MSG_FILE" + fi +fi + +exit 0 diff --git a/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml b/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml index 3bb570b..7c6fafd 100644 --- a/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml +++ b/.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml @@ -157,8 +157,8 @@ body: - scoring.md — tabular scoring with 1-10 ratings and justifications. - architecture.mmd — Mermaid diagram source for the architecture. - improvement-issues.md — synopsis of open improvement-focused GitHub issues with solution steps. - - ai-adoption.md — explicit AI adoption analysis for the last 6 months with an executive judgment (low/moderate/high), score, shipped-to-master section, contributor/subsystem adoption analysis, non-adopter table, next steps, and caveats. - - ai-adoption.csv — commit-level explicit AI-related commits in the last 6 months with columns: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file. + - ai-adoption.md — explicit AI adoption analysis for the last 6 months with an executive judgment (low/moderate/high), score, shipped-to-master section, contributor/subsystem adoption analysis, **squashed commits analysis**, non-adopter table, next steps, and caveats. + - ai-adoption.csv — commit-level explicit AI-related commits in the last 6 months with columns: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file, squash_merge_indicator, signal_type (explicit_keyword | github_metadata | git_hook | implicit_pattern). - ai-adoption-by-developer.csv — developer aggregation from ai-adoption.csv with columns: developer, total_ai_commits, ai_commits_on_master, ai_commits_off_master, master_commit_share_pct. - chi-compliance.md — generated only when the repository includes a Vue-based UI; * FIRST, check index.html for Chi CDN references (CSS/JS URLs) - this is the authoritative Chi version @@ -177,21 +177,42 @@ body: - AI Adoption Report Requirements (last 6 months, output to copilot-eval): * Analyze commits from now minus 6 months through today. * Include all refs for contribution analysis, and separately analyze what reached master. - * Treat explicit AI-related commits as commit subjects containing (case-insensitive): copilot, @Copilot, ai-assisted, gpt, claude, cursor. + * **Signal Detection Strategy** (multiple layers to reduce developer friction): + 1. Explicit AI Keywords (optional, zero friction if auto-tagged): Treat commit subjects containing (case-insensitive): copilot, @Copilot, ai-assisted, gpt, claude, cursor. + 2. GitHub Copilot Attribution Metadata: If available, detect commits tagged with GitHub's AI-generated indicator (via GitHub API or webhook metadata). + 3. Git Hooks (recommended for ease-of-use): If the repo includes a `.githooks/prepare-commit-msg` hook or similar, document its presence and how it auto-tags AI commits without developer intervention. + 4. IDE/CLI Integration: Note if the codebase uses Copilot CLI or IDE extensions that auto-append AI indicators at commit time (no developer action required). + 5. Implicit Patterns (fallback): Scan commit message structure for AI-generated patterns (e.g., structured formatting, typical Copilot phrasing, co-authored metadata like "Co-authored-by: GitHub Copilot"). + 6. Squashed Commit Recovery: Extract AI signals from individual commits inside squashed merges using all methods above. + * Recommended Setup for Minimal Developer Friction: + - Use a git hook (prepare-commit-msg or commit-msg) to auto-detect and auto-tag AI commits at commit time. + - Provide a `.githooks/prepare-commit-msg` template in the repo README for teams to adopt. + * If no automation in place, fall back to explicit keyword detection; but acknowledge this undercounts adoption. * Treat on_master as commit is an ancestor of master. * Treat content commit as files_changed > 0 and merge-only commit as files_changed = 0. * Gather baseline counts: total commits in 6 months (all refs) and total commits in 6 months on master. - * Build ai-adoption.csv with one row per explicit AI-related commit and columns exactly: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file. + * **Squashed Commit Inspection** (critical for accurate adoption metrics): + - For each *squash-merged PR commit* on master (typically a single-parent commit with subject like `... (#123)`), extract the PR number from the commit subject. + - If PR metadata is available (GitHub API), list the PR's commits and scan each commit subject/body for AI signals (explicit keywords, metadata, patterns). + - If PR metadata is not available locally, document that individual pre-squash commits cannot be reliably recovered from git history alone. + - Attribute detected AI signals to the original PR commit authors (not the squash-merge author) when PR commit data is available. + - Document the observed merge strategy (squash vs. merge vs. rebase) and any recovery limitations in the Caveats section. + - For squashed commits with AI signals, preserve the original author/date and create a logical "unmerged" entry in ai-adoption.csv with the source commits listed in a supplementary column. + - If PR metadata is available (GitHub API), cross-reference the PR number from the merge commit message to retrieve the full commit history before squashing. + - Document merge strategy (squash vs. fast-forward vs. three-way merge) in the Caveats section and note any adoption metrics recovered from squashed commits. + * Build ai-adoption.csv with one row per explicit AI-related commit (including unmerged squashed commits) and columns exactly: sha, date, author, normalized_author, subject, on_master, files_changed, top_level_area, dominant_subsystem, dominant_file, squash_merge_indicator (true/false), signal_type (explicit_keyword | github_metadata | git_hook | implicit_pattern). * Compute normalized_author by merging obvious aliases via shared email identity and clear name/username variants (e.g., corporate name vs GitHub handle), and document normalization caveat in markdown. * Build ai-adoption-by-developer.csv from ai-adoption.csv with columns exactly: developer, total_ai_commits, ai_commits_on_master, ai_commits_off_master, master_commit_share_pct. * Detect active contributors with zero explicit AI signal: all active contributors in 6 months minus normalized authors in ai-adoption.csv, and include commit counts for this group. * Avoid brittle one-liner heredocs in terminal; if scripting is needed, create small temporary scripts/files, run them, and clean up helper artifacts afterward. - * ai-adoption.md must include: Executive Judgment (moderate/low/high) with rationale; Method with exact filter terms, timeframe, and lower-bound caveat; section #2 What Actually Shipped To master with counts and table (Date, Author, SHA short, Subject, Dominant subsystem, Files changed); section #3 Developer And Subsystem Adoption with normalized contributor counts and subsystem distribution; section All AI Contributions Per Developer sourced from ai-adoption-by-developer.csv (Developer, Total AI commits, AI commits on master, AI commits off master, master share); section Developers With 0 Explicit AI Assistance (Developer, six-month commit count, recommendation priority); section Next Steps; and section Caveats. - * Quality bar: markdown numbers must reconcile with both CSVs; use normalized identities for developer reporting; distinguish all AI contributions vs merged-to-master AI contributions; remove temporary helper artifacts; validate outputs are readable and error-free. + * ai-adoption.md must include: Executive Judgment (moderate/low/high) with rationale; Method with exact filter terms, timeframe, lower-bound caveat, **merge strategy discovery notes**, and **detection strategy used** (explicit keywords only vs. multi-signal); section #2 What Actually Shipped To master with counts and table (Date, Author, SHA short, Subject, Dominant subsystem, Files changed, Squash Indicator, Signal Type); section #3 Developer And Subsystem Adoption with normalized contributor counts and subsystem distribution; section All AI Contributions Per Developer sourced from ai-adoption-by-developer.csv (Developer, Total AI commits, AI commits on master, AI commits off master, master share); section Developers With 0 Explicit AI Assistance (Developer, six-month commit count, recommendation priority); **section Squashed Commits Analysis** with summary of recovered AI signals, original vs. merge authors, and files impacted; section Setup Recommendations (e.g., "Add git hooks for zero-friction auto-tagging"; section Next Steps; and section Caveats. + * Quality bar: markdown numbers must reconcile with both CSVs and squashed commit inspection logs; use normalized identities for developer reporting; distinguish all AI contributions vs merged-to-master AI contributions; account for squashed commits when computing adoption rate; remove temporary helper artifacts; validate outputs are readable and error-free. - For Vue-based UI repositories, expand the analysis phase to review template markup across all views, pages, and components against Chi component guidance (web components or HTML boilerplate) and describe non-compliant patterns in chi-compliance.md. - When reporting Chi compliance, treat the version declared by the CDN-style script or stylesheet reference in the primary HTML entrypoint (for example, `