diff --git a/PUBLISHING.md b/PUBLISHING.md index 69dcebe..1571367 100644 --- a/PUBLISHING.md +++ b/PUBLISHING.md @@ -15,6 +15,8 @@ The action follows semantic versioning and uses a multi-tag system similar to ot The automated publishing workflow is triggered when you create a GitHub release: +> **Note:** Creating a release via CLI or API does NOT automatically publish to GitHub Marketplace. You must manually enable marketplace publishing through the web UI after creating the release. + #### Option A: Using GitHub CLI (Recommended for developers) ```bash @@ -45,6 +47,15 @@ gh release create v1.0.0 --title "v1.0.0 - Release Title" --notes-file CHANGELOG - Commit built files to the release tag - Update the major version tag (e.g., `v1` → `v1.0.0`) +3. **Publishing to GitHub Marketplace** (Manual Step): + - Navigate to the [Releases page](https://github.com/getsentry/action-onlyrobots/releases) + - Find your newly created release + - Click "Edit" on the release + - Check the box "Publish this Action to the GitHub Marketplace" + - If first time: Accept the GitHub Marketplace Developer Agreement + - Click "Update release" + - Your action will now appear in the GitHub Marketplace! + ### 2. Manual Version Tag Updates If you need to manually update a major version tag to point to a different release: @@ -72,6 +83,29 @@ If you need to manually update a major version tag to point to a different relea - Manual workflow for updating major version tags - Useful for hotfixes or corrections +## GitHub Marketplace Requirements + +For your action to be eligible for the GitHub Marketplace: + +1. **Repository Requirements**: + - Must be a public repository ✅ + - Must contain a single `action.yml` or `action.yaml` at the root ✅ + - Must NOT contain any workflow files in `.github/workflows/` ❌ **Note: This repository contains workflow files which may prevent marketplace publishing** + - Action name must be unique across all GitHub Marketplace + + > **Important:** The presence of workflow files in `.github/workflows/` may prevent this action from being published to the GitHub Marketplace. Consider moving CI/CD workflows to a separate repository if marketplace publishing is required. + +2. **Metadata Requirements** (`action.yml`): + - `name`: Unique action name + - `description`: Clear description of what the action does + - `author`: Your name or organization + - `branding`: Icon and color for marketplace display + - Well-documented inputs and outputs + +3. **Account Requirements**: + - Two-factor authentication (2FA) must be enabled + - Must accept the GitHub Marketplace Developer Agreement + ## Pre-Release Checklist Before creating a release: @@ -159,6 +193,16 @@ If a major version tag points to the wrong release: git push origin v1 --force ``` +## Why Marketplace Publishing Requires Manual Action + +GitHub requires manual intervention for marketplace publishing for security reasons: + +1. **2FA Verification**: Publishing to the marketplace requires two-factor authentication verification that can only be done through the web browser +2. **Developer Agreement**: The first marketplace publication requires accepting the GitHub Marketplace Developer Agreement +3. **Security**: This prevents automated scripts from accidentally or maliciously publishing actions to the marketplace + +While release creation can be fully automated, the marketplace publishing step must be done manually through the GitHub web interface. + ## Security Notes - Only repository maintainers can create releases diff --git a/action.yml b/action.yml index 9c9913f..7ac4614 100644 --- a/action.yml +++ b/action.yml @@ -2,7 +2,7 @@ name: 'Only Robots' description: 'Ensure code changes are written by AI agents, not humans' author: 'Sentry' branding: - icon: 'robot' + icon: 'battery-charging' color: 'purple' inputs: diff --git a/dist/index.js b/dist/index.js index 10616ba..c89708a 100644 --- a/dist/index.js +++ b/dist/index.js @@ -30084,7 +30084,7 @@ function buildHumanDetectionComment(overallResult) { if (overallResult.indicators && overallResult.indicators.length > 0) { messages.push(`### 🔍 Human fingerprints detected:\n`); for (const indicator of overallResult.indicators) { - messages.push(`- ${indicator}`); + messages.push(`- ${indicator}\n`); } messages.push(`\n`); } @@ -30267,17 +30267,27 @@ Analyze the code looking for these specific signals: - All comments following identical formatting style - Repetitive code structures across different sections +**PRECISION INDICATORS (75-90% confidence):** +- Single-character formatting fixes (adding newlines, spaces, commas) +- Surgical precision changes with no side effects or additional modifications +- Minimal, targeted fixes to specific formatting or syntax issues +- Changes that follow exact patterns (e.g., consistently adding newlines to list items) +- Simple string literal modifications for formatting consistency + **STYLISTIC PATTERNS (70-85% confidence):** -- Comments explaining obvious code functionality +- Comments explaining obvious code functionality - Comprehensive error handling on every function - Consistent use of latest/modern language patterns throughout - Perfect adherence to documentation examples -- Overly descriptive naming for simple concepts +- Overly descriptive naming for simple concepts (e.g., "userDisplayNameString", "formatUserDisplayNameWithEmailAddress") +- Verbose parameter names with unnecessary detail (e.g., "userAccountInformation" instead of "user") **FOCUS ON DETECTING OBVIOUS AI PATTERNS:** - Look for CRITICAL SIGNALS first - these are definitive - Multiple STRUCTURAL FINGERPRINTS together suggest AI generation +- PRECISION INDICATORS are strong signals for AI-generated formatting fixes - STYLISTIC PATTERNS may support AI detection but are not decisive alone +- Small, surgical changes with perfect precision are typical of AI assistants - Absence of human indicators does NOT mean it's AI-generated - Professional, clean code is often written by skilled human developers @@ -30325,6 +30335,19 @@ Respond with your analysis in the exact format specified in the system prompt.`; if (lowerContent.includes('consistent') || lowerContent.includes('formatted')) { indicators.push('consistent-formatting'); } + if (lowerContent.includes('surgical') || + lowerContent.includes('precision') || + lowerContent.includes('targeted')) { + indicators.push('precision-changes'); + } + if (lowerContent.includes('newline') || lowerContent.includes('formatting fix')) { + indicators.push('formatting-fix'); + } + if (lowerContent.includes('verbose') || + lowerContent.includes('descriptive naming') || + lowerContent.includes('overly descriptive')) { + indicators.push('verbose-naming'); + } return indicators; } buildOverallReasoning(fileResults, humanLikeFiles) { @@ -30358,12 +30381,20 @@ const SYSTEM_PROMPT = `You are an expert code reviewer tasked with determining w 4. **Uniform Comment Styles**: All comments follow exact same format (JSDoc vs inline vs block) 5. **Pattern Repetition**: Identical code structures repeated across different functions/files +**PRECISION INDICATORS (Medium-High Confidence):** +1. **Surgical Formatting Changes**: Single-character fixes like adding newlines, spaces, or punctuation +2. **Targeted Corrections**: Minimal changes that fix specific issues without touching surrounding code +3. **Pattern-Based Fixes**: Consistent application of the same small change across multiple locations +4. **Zero Side Effects**: Changes that only address the stated problem with no extra modifications +5. **Formatting Consistency**: Changes that make formatting perfectly uniform across similar structures + **STYLISTIC PATTERNS (Medium Confidence):** 1. **Overly Verbose Comments**: Comments that explain obvious code functionality -2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling +2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling 3. **Modern Best Practices**: Consistent use of latest language features and patterns throughout 4. **Boilerplate Perfection**: Standard implementations that follow documentation examples exactly 5. **Descriptive Everything**: Variable names, function names, and comments are all extremely descriptive +6. **Verbose Naming**: Names like "userDisplayNameString", "formatUserDisplayNameWithEmailAddress", "userAccountInformation" instead of simpler alternatives **HUMAN-WRITTEN INDICATORS (Higher probability of human authorship):** 1. **Debug Artifacts**: console.log("here"), console.log("debug"), temporary print statements diff --git a/src/index.ts b/src/index.ts index 947c9df..9fa58e7 100644 --- a/src/index.ts +++ b/src/index.ts @@ -146,7 +146,7 @@ function buildHumanDetectionComment(overallResult: any): string { if (overallResult.indicators && overallResult.indicators.length > 0) { messages.push(`### 🔍 Human fingerprints detected:\n`); for (const indicator of overallResult.indicators) { - messages.push(`- ${indicator}`); + messages.push(`- ${indicator}\n`); } messages.push(`\n`); } diff --git a/src/llm-evaluator.ts b/src/llm-evaluator.ts index 39fd57c..d3ff0ec 100644 --- a/src/llm-evaluator.ts +++ b/src/llm-evaluator.ts @@ -140,17 +140,27 @@ Analyze the code looking for these specific signals: - All comments following identical formatting style - Repetitive code structures across different sections +**PRECISION INDICATORS (75-90% confidence):** +- Single-character formatting fixes (adding newlines, spaces, commas) +- Surgical precision changes with no side effects or additional modifications +- Minimal, targeted fixes to specific formatting or syntax issues +- Changes that follow exact patterns (e.g., consistently adding newlines to list items) +- Simple string literal modifications for formatting consistency + **STYLISTIC PATTERNS (70-85% confidence):** -- Comments explaining obvious code functionality +- Comments explaining obvious code functionality - Comprehensive error handling on every function - Consistent use of latest/modern language patterns throughout - Perfect adherence to documentation examples -- Overly descriptive naming for simple concepts +- Overly descriptive naming for simple concepts (e.g., "userDisplayNameString", "formatUserDisplayNameWithEmailAddress") +- Verbose parameter names with unnecessary detail (e.g., "userAccountInformation" instead of "user") **FOCUS ON DETECTING OBVIOUS AI PATTERNS:** - Look for CRITICAL SIGNALS first - these are definitive - Multiple STRUCTURAL FINGERPRINTS together suggest AI generation +- PRECISION INDICATORS are strong signals for AI-generated formatting fixes - STYLISTIC PATTERNS may support AI detection but are not decisive alone +- Small, surgical changes with perfect precision are typical of AI assistants - Absence of human indicators does NOT mean it's AI-generated - Professional, clean code is often written by skilled human developers @@ -201,6 +211,23 @@ Respond with your analysis in the exact format specified in the system prompt.`; if (lowerContent.includes('consistent') || lowerContent.includes('formatted')) { indicators.push('consistent-formatting'); } + if ( + lowerContent.includes('surgical') || + lowerContent.includes('precision') || + lowerContent.includes('targeted') + ) { + indicators.push('precision-changes'); + } + if (lowerContent.includes('newline') || lowerContent.includes('formatting fix')) { + indicators.push('formatting-fix'); + } + if ( + lowerContent.includes('verbose') || + lowerContent.includes('descriptive naming') || + lowerContent.includes('overly descriptive') + ) { + indicators.push('verbose-naming'); + } return indicators; } @@ -243,12 +270,20 @@ const SYSTEM_PROMPT = `You are an expert code reviewer tasked with determining w 4. **Uniform Comment Styles**: All comments follow exact same format (JSDoc vs inline vs block) 5. **Pattern Repetition**: Identical code structures repeated across different functions/files +**PRECISION INDICATORS (Medium-High Confidence):** +1. **Surgical Formatting Changes**: Single-character fixes like adding newlines, spaces, or punctuation +2. **Targeted Corrections**: Minimal changes that fix specific issues without touching surrounding code +3. **Pattern-Based Fixes**: Consistent application of the same small change across multiple locations +4. **Zero Side Effects**: Changes that only address the stated problem with no extra modifications +5. **Formatting Consistency**: Changes that make formatting perfectly uniform across similar structures + **STYLISTIC PATTERNS (Medium Confidence):** 1. **Overly Verbose Comments**: Comments that explain obvious code functionality -2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling +2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling 3. **Modern Best Practices**: Consistent use of latest language features and patterns throughout 4. **Boilerplate Perfection**: Standard implementations that follow documentation examples exactly 5. **Descriptive Everything**: Variable names, function names, and comments are all extremely descriptive +6. **Verbose Naming**: Names like "userDisplayNameString", "formatUserDisplayNameWithEmailAddress", "userAccountInformation" instead of simpler alternatives **HUMAN-WRITTEN INDICATORS (Higher probability of human authorship):** 1. **Debug Artifacts**: console.log("here"), console.log("debug"), temporary print statements