Skip to content

feat(ci): auto-label issues with GPT-5 mini #1998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 27, 2025
Merged

feat(ci): auto-label issues with GPT-5 mini #1998

merged 3 commits into from
Oct 27, 2025
+22 −7

Conversation

@sanity
Copy link
Collaborator

@sanity sanity commented Oct 27, 2025

Summary

Automatically labels new GitHub issues using GPT-5 mini for consistent triaging.

Related to #1980

What Changed

  • Added workflow
  • Triggers on every new issue (including user-labeled ones for curation)
  • Uses GPT-5 mini API with JSON response format
  • Applies labels with ≥75% confidence threshold
  • Posts explanatory comment with reasoning

Label Curation

The workflow runs on all new issues, even if users add labels:

  • Sees user's initial labels in the prompt
  • Can improve incomplete or incorrect labels
  • Removes old labels before applying new ones
  • Shows 'Previous labels' for transparency

Security Measures

Anti-spam:

  • Only processes issues from accounts >7 days old
  • Rate limited to 1 concurrent run

Cost control:

  • Truncates issue bodies >8000 chars
  • Uses efficient GPT-5 mini model (~$0.01/issue)

Prompt injection protection:

  • XML delimiters around user content
  • Explicit instructions to ignore embedded commands

Label Schema

Uses prefix system from #1980:

  • T- (Type): bug, feature, enhancement, docs, question, tracking
  • P- (Priority): critical, high, medium, low
  • E- (Effort): easy, medium, hard
  • A- (Area): networking, contracts, developer-xp, documentation, crypto
  • S- (Status): needs-reproduction, needs-design, blocked, waiting-feedback

Testing

Once merged, test by creating a new issue and verifying:

  1. Workflow runs and completes
  2. Labels are applied correctly
  3. Comment explains reasoning

[AI-assisted debugging and comment]

@sanity sanity enabled auto-merge October 27, 2025 03:09
sanity added 2 commits October 27, 2025 04:28
@sanity
fix(ci): use GPT-5 correct API params and remove branding
3c57344 
- Change max_tokens to max_completion_tokens (GPT-5 requirement)
- Remove 'Powered by GPT-5 mini' footer from comments
- Simplify comment to just '🤖 Auto-labeled'

[AI-assisted debugging and comment]
@sanity
feat(ci): enable label curation for all new issues
e622654 
- Remove condition that only labels issues without T- labels
- Now runs on ALL new issues to curate/improve user-provided labels
- Removes existing labels before applying GPT-5 suggestions
- Shows 'Previous labels' in comment for transparency
- GPT-5 can see user's initial labels and improve/complete them

[AI-assisted debugging and comment]
@sanity sanity force-pushed the feat/auto-label-issues branch from 3d7c822 to e622654 Compare October 27, 2025 03:28
@sanity sanity changed the title feat(ci): Auto-label issues with GPT-5 mini feat(ci): auto-label issues with GPT-5 mini Oct 27, 2025
@sanity sanity added this pull request to the merge queue Oct 27, 2025
Merged via the queue into main with commit cf10f40 Oct 27, 2025
10 checks passed
@sanity sanity deleted the feat/auto-label-issues branch October 27, 2025 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sanity