Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions PUBLISHING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ The action follows semantic versioning and uses a multi-tag system similar to ot

The automated publishing workflow is triggered when you create a GitHub release:

> **Note:** Creating a release via CLI or API does NOT automatically publish to GitHub Marketplace. You must manually enable marketplace publishing through the web UI after creating the release.

#### Option A: Using GitHub CLI (Recommended for developers)

```bash
Expand Down Expand Up @@ -45,6 +47,15 @@ gh release create v1.0.0 --title "v1.0.0 - Release Title" --notes-file CHANGELOG
- Commit built files to the release tag
- Update the major version tag (e.g., `v1` → `v1.0.0`)

3. **Publishing to GitHub Marketplace** (Manual Step):
- Navigate to the [Releases page](https://github.com/getsentry/action-onlyrobots/releases)
- Find your newly created release
- Click "Edit" on the release
- Check the box "Publish this Action to the GitHub Marketplace"
- If first time: Accept the GitHub Marketplace Developer Agreement
- Click "Update release"
- Your action will now appear in the GitHub Marketplace!

### 2. Manual Version Tag Updates

If you need to manually update a major version tag to point to a different release:
Expand Down Expand Up @@ -72,6 +83,29 @@ If you need to manually update a major version tag to point to a different relea
- Manual workflow for updating major version tags
- Useful for hotfixes or corrections

## GitHub Marketplace Requirements

For your action to be eligible for the GitHub Marketplace:

1. **Repository Requirements**:
- Must be a public repository ✅
- Must contain a single `action.yml` or `action.yaml` at the root ✅
- Must NOT contain any workflow files in `.github/workflows/` ❌ **Note: This repository contains workflow files which may prevent marketplace publishing**
- Action name must be unique across all GitHub Marketplace

> **Important:** The presence of workflow files in `.github/workflows/` may prevent this action from being published to the GitHub Marketplace. Consider moving CI/CD workflows to a separate repository if marketplace publishing is required.

2. **Metadata Requirements** (`action.yml`):
- `name`: Unique action name
- `description`: Clear description of what the action does
- `author`: Your name or organization
- `branding`: Icon and color for marketplace display
- Well-documented inputs and outputs

3. **Account Requirements**:
- Two-factor authentication (2FA) must be enabled
- Must accept the GitHub Marketplace Developer Agreement

## Pre-Release Checklist

Before creating a release:
Expand Down Expand Up @@ -159,6 +193,16 @@ If a major version tag points to the wrong release:
git push origin v1 --force
```

## Why Marketplace Publishing Requires Manual Action

GitHub requires manual intervention for marketplace publishing for security reasons:

1. **2FA Verification**: Publishing to the marketplace requires two-factor authentication verification that can only be done through the web browser
2. **Developer Agreement**: The first marketplace publication requires accepting the GitHub Marketplace Developer Agreement
3. **Security**: This prevents automated scripts from accidentally or maliciously publishing actions to the marketplace

While release creation can be fully automated, the marketplace publishing step must be done manually through the GitHub web interface.

## Security Notes

- Only repository maintainers can create releases
Expand Down
2 changes: 1 addition & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: 'Only Robots'
description: 'Ensure code changes are written by AI agents, not humans'
author: 'Sentry'
branding:
icon: 'robot'
icon: 'battery-charging'
color: 'purple'

inputs:
Expand Down
39 changes: 35 additions & 4 deletions dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -30084,7 +30084,7 @@ function buildHumanDetectionComment(overallResult) {
if (overallResult.indicators && overallResult.indicators.length > 0) {
messages.push(`### 🔍 Human fingerprints detected:\n`);
for (const indicator of overallResult.indicators) {
messages.push(`- ${indicator}`);
messages.push(`- ${indicator}\n`);
}
messages.push(`\n`);
}
Expand Down Expand Up @@ -30267,17 +30267,27 @@ Analyze the code looking for these specific signals:
- All comments following identical formatting style
- Repetitive code structures across different sections

**PRECISION INDICATORS (75-90% confidence):**
- Single-character formatting fixes (adding newlines, spaces, commas)
- Surgical precision changes with no side effects or additional modifications
- Minimal, targeted fixes to specific formatting or syntax issues
- Changes that follow exact patterns (e.g., consistently adding newlines to list items)
- Simple string literal modifications for formatting consistency

**STYLISTIC PATTERNS (70-85% confidence):**
- Comments explaining obvious code functionality
- Comments explaining obvious code functionality
- Comprehensive error handling on every function
- Consistent use of latest/modern language patterns throughout
- Perfect adherence to documentation examples
- Overly descriptive naming for simple concepts
- Overly descriptive naming for simple concepts (e.g., "userDisplayNameString", "formatUserDisplayNameWithEmailAddress")
- Verbose parameter names with unnecessary detail (e.g., "userAccountInformation" instead of "user")

**FOCUS ON DETECTING OBVIOUS AI PATTERNS:**
- Look for CRITICAL SIGNALS first - these are definitive
- Multiple STRUCTURAL FINGERPRINTS together suggest AI generation
- PRECISION INDICATORS are strong signals for AI-generated formatting fixes
- STYLISTIC PATTERNS may support AI detection but are not decisive alone
- Small, surgical changes with perfect precision are typical of AI assistants
- Absence of human indicators does NOT mean it's AI-generated
- Professional, clean code is often written by skilled human developers

Expand Down Expand Up @@ -30325,6 +30335,19 @@ Respond with your analysis in the exact format specified in the system prompt.`;
if (lowerContent.includes('consistent') || lowerContent.includes('formatted')) {
indicators.push('consistent-formatting');
}
if (lowerContent.includes('surgical') ||
lowerContent.includes('precision') ||
lowerContent.includes('targeted')) {
indicators.push('precision-changes');
}
if (lowerContent.includes('newline') || lowerContent.includes('formatting fix')) {
indicators.push('formatting-fix');
}
if (lowerContent.includes('verbose') ||
lowerContent.includes('descriptive naming') ||
lowerContent.includes('overly descriptive')) {
indicators.push('verbose-naming');
}
return indicators;
}
buildOverallReasoning(fileResults, humanLikeFiles) {
Expand Down Expand Up @@ -30358,12 +30381,20 @@ const SYSTEM_PROMPT = `You are an expert code reviewer tasked with determining w
4. **Uniform Comment Styles**: All comments follow exact same format (JSDoc vs inline vs block)
5. **Pattern Repetition**: Identical code structures repeated across different functions/files

**PRECISION INDICATORS (Medium-High Confidence):**
1. **Surgical Formatting Changes**: Single-character fixes like adding newlines, spaces, or punctuation
2. **Targeted Corrections**: Minimal changes that fix specific issues without touching surrounding code
3. **Pattern-Based Fixes**: Consistent application of the same small change across multiple locations
4. **Zero Side Effects**: Changes that only address the stated problem with no extra modifications
5. **Formatting Consistency**: Changes that make formatting perfectly uniform across similar structures

**STYLISTIC PATTERNS (Medium Confidence):**
1. **Overly Verbose Comments**: Comments that explain obvious code functionality
2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling
2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling
3. **Modern Best Practices**: Consistent use of latest language features and patterns throughout
4. **Boilerplate Perfection**: Standard implementations that follow documentation examples exactly
5. **Descriptive Everything**: Variable names, function names, and comments are all extremely descriptive
6. **Verbose Naming**: Names like "userDisplayNameString", "formatUserDisplayNameWithEmailAddress", "userAccountInformation" instead of simpler alternatives

**HUMAN-WRITTEN INDICATORS (Higher probability of human authorship):**
1. **Debug Artifacts**: console.log("here"), console.log("debug"), temporary print statements
Expand Down
2 changes: 1 addition & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ function buildHumanDetectionComment(overallResult: any): string {
if (overallResult.indicators && overallResult.indicators.length > 0) {
messages.push(`### 🔍 Human fingerprints detected:\n`);
for (const indicator of overallResult.indicators) {
messages.push(`- ${indicator}`);
messages.push(`- ${indicator}\n`);
}
messages.push(`\n`);
}
Expand Down
41 changes: 38 additions & 3 deletions src/llm-evaluator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -140,17 +140,27 @@ Analyze the code looking for these specific signals:
- All comments following identical formatting style
- Repetitive code structures across different sections

**PRECISION INDICATORS (75-90% confidence):**
- Single-character formatting fixes (adding newlines, spaces, commas)
- Surgical precision changes with no side effects or additional modifications
- Minimal, targeted fixes to specific formatting or syntax issues
- Changes that follow exact patterns (e.g., consistently adding newlines to list items)
- Simple string literal modifications for formatting consistency

**STYLISTIC PATTERNS (70-85% confidence):**
- Comments explaining obvious code functionality
- Comments explaining obvious code functionality
- Comprehensive error handling on every function
- Consistent use of latest/modern language patterns throughout
- Perfect adherence to documentation examples
- Overly descriptive naming for simple concepts
- Overly descriptive naming for simple concepts (e.g., "userDisplayNameString", "formatUserDisplayNameWithEmailAddress")
- Verbose parameter names with unnecessary detail (e.g., "userAccountInformation" instead of "user")

**FOCUS ON DETECTING OBVIOUS AI PATTERNS:**
- Look for CRITICAL SIGNALS first - these are definitive
- Multiple STRUCTURAL FINGERPRINTS together suggest AI generation
- PRECISION INDICATORS are strong signals for AI-generated formatting fixes
- STYLISTIC PATTERNS may support AI detection but are not decisive alone
- Small, surgical changes with perfect precision are typical of AI assistants
- Absence of human indicators does NOT mean it's AI-generated
- Professional, clean code is often written by skilled human developers

Expand Down Expand Up @@ -201,6 +211,23 @@ Respond with your analysis in the exact format specified in the system prompt.`;
if (lowerContent.includes('consistent') || lowerContent.includes('formatted')) {
indicators.push('consistent-formatting');
}
if (
lowerContent.includes('surgical') ||
lowerContent.includes('precision') ||
lowerContent.includes('targeted')
) {
indicators.push('precision-changes');
}
if (lowerContent.includes('newline') || lowerContent.includes('formatting fix')) {
indicators.push('formatting-fix');
}
if (
lowerContent.includes('verbose') ||
lowerContent.includes('descriptive naming') ||
lowerContent.includes('overly descriptive')
) {
indicators.push('verbose-naming');
}

return indicators;
}
Expand Down Expand Up @@ -243,12 +270,20 @@ const SYSTEM_PROMPT = `You are an expert code reviewer tasked with determining w
4. **Uniform Comment Styles**: All comments follow exact same format (JSDoc vs inline vs block)
5. **Pattern Repetition**: Identical code structures repeated across different functions/files

**PRECISION INDICATORS (Medium-High Confidence):**
1. **Surgical Formatting Changes**: Single-character fixes like adding newlines, spaces, or punctuation
2. **Targeted Corrections**: Minimal changes that fix specific issues without touching surrounding code
3. **Pattern-Based Fixes**: Consistent application of the same small change across multiple locations
4. **Zero Side Effects**: Changes that only address the stated problem with no extra modifications
5. **Formatting Consistency**: Changes that make formatting perfectly uniform across similar structures

**STYLISTIC PATTERNS (Medium Confidence):**
1. **Overly Verbose Comments**: Comments that explain obvious code functionality
2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling
2. **Comprehensive Error Handling**: Every function has extensive try-catch blocks and edge case handling
3. **Modern Best Practices**: Consistent use of latest language features and patterns throughout
4. **Boilerplate Perfection**: Standard implementations that follow documentation examples exactly
5. **Descriptive Everything**: Variable names, function names, and comments are all extremely descriptive
6. **Verbose Naming**: Names like "userDisplayNameString", "formatUserDisplayNameWithEmailAddress", "userAccountInformation" instead of simpler alternatives

**HUMAN-WRITTEN INDICATORS (Higher probability of human authorship):**
1. **Debug Artifacts**: console.log("here"), console.log("debug"), temporary print statements
Expand Down