Skip to content

CONTRIBUTING.md: GPT-5.4-only AI policy is no longer current SOTA #724

@flyinghanger

Description

@flyinghanger

Context

CONTRIBUTING.md (L12-20) currently states:

AI-Generated Code Policy (Vibe Coding)

If you are using AI tools to generate code ("Vibe Coding"), please note:

We only accept code generated by the GPT-5.4 model.

Pull requests containing code generated by other AI models (including other GPT versions, Claude, Gemini, or any other LLM) will be rejected.

Issue

GPT-5.4 is no longer current SOTA — OpenAI has shipped successors, and other vendors (Anthropic, Google, etc.) ship competitive frontier models in the same tier. As written, the policy:

  1. Excludes the current SOTA, including OpenAI's own newer releases — the pin is to a specific past version, not "the best available".
  2. Isn't reliably detectable. There is no robust way for a reviewer to verify which model produced a given diff, so the policy primarily filters honesty rather than quality.
  3. Penalizes good-faith disclosure. A contributor who states the model used (e.g. fix(claude): propagate real contextWindow from SDK to web status bar #720, where I disclosed Claude in the PR description) is auto-rejected, while undisclosed AI-assisted code goes through unflagged. This inverts the incentive the policy presumably wants to create.

Suggestions

Three options, lightest to heaviest, in case any of them is useful:

  1. Drop the model whitelist. Lean on the existing quality bar already in CONTRIBUTING (focused PRs, tests, no mega-PRs). Quality of output is what matters, and that can be judged from the diff directly.
  2. Switch to disclosure-only. Authors disclose the model in the PR description; reviewers judge the code on merit. Keeps transparency without the enforceability problem.
  3. Rolling allowlist. Maintain a short list of "currently acceptable" SOTA models, refreshed periodically (e.g. quarterly), so the policy stays calibrated to the actual frontier.

Happy to send a docs PR for whichever direction lands well, or to close this if the restriction is intentional and I'm missing the rationale (e.g. a specific past incident with non-GPT models that motivated the policy).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions