Proposal: Auto-Building and Sharing Experimental Binaries

[kt-devoss]

Hi.
What if we could instantly try community PRs with interesting features as binaries, without requiring local build environments or adding review burden to OpenAI's maintainers?
What if, instead of being limited to Issues, contributors could also participate through actual code?

This is a proposal to separate experimentation from the official repository.

Clarification

This is intended as an MVP / proof-of-concept, not a production system.
This proposal uses a GitHub repository as a submission interface, NOT as a maintained codebase.
It is primarily used to build per-PR artifacts(ready-to-run binary).
Each PR produces an isolated artifact on top of the latest official main branch, and is expected to be a small, focused patch targeting a single feature or idea.

• experiment = (official main branch + small, focused patch) + CI build + artifact
• experiment ≠ alternative codebase

Proposal: A System for Sharing Experimental Binaries to Support Issue Resolution

I understand that the recent policy change (invitation-only PRs) is reasonable from the perspective of scalability and maintaining quality.
(#9956 Updating Codex Contribution Guidelines)

However, there are some side effects that are concerning:

• Useful PRs linked to issues may be closed without being adopted, simply because the contributor was not invited
• It is difficult to validate ideas through implementation, leading to abstract discussions
• Forks exist, but they are difficult to try, and often diverge significantly from the official releases.

Especially for tools like TUI, many aspects can only be properly evaluated by actually using them. I feel that issue-based discussions alone have limitations.

e.g. A/B testing for UX, Keybinding design, Visual / TUI rendering, Interaction flow, Environment-specific issues, ...

---

Proposal

flowchart RL

    %% ===== Official =====
    subgraph Official["Codex Official Domain"]
        direction TB
        I["Issues"]
        M(["OpenAI Maintainers"])
        O[(Official Repo)]
        I -->|review / triage| M
        M -->|curated implementation| O
    end

    %% ===== Community =====
    subgraph Community["Community Domain"]
        direction TB
        R[(Community Experimental Repo)]
        B["CI Artifacts\n Custom Binaries\n (official release + small, focused changes)"]
        R -->|automated build| B
    end

    %% ===== Users =====
    UC(["Users / Community Contributors"])

    %% ===== Main flows (bold) =====
    UC ==> |Raw ideas| I
    UC ==> |"Submit small, focused PRs \n (based on official main)"| R
    B ==> |Trial| UC

    %% ===== Feedback loop =====
    UC -->|validated feedback| I

    %% ===== Weak relations =====
    O -.->|sync baseline| R

    %% ===== Style =====
    classDef actor fill:#fff3e0,stroke:#ef6c00,stroke-width:2px;
    classDef system fill:#e3f2fd,stroke:#1e88e5,stroke-width:2px;
    classDef data fill:#e8f5e9,stroke:#43a047,stroke-width:2px;

    class UC,M actor;
    class I,B system;
    class O,R data;

Loading

In addition to the official repository (product), I propose a separate system for building and sharing binaries that include experimental changes aimed at resolving issues.
This system is realized through GitHub Actions in a separate community-managed repository, independent of the official Codex project.

Separation of roles

• Official repository

  • Same as now (invitation-only PRs, No experimental PRs)
  • Focus on quality and consistency

• Community repository (experimental) (this proposal)

  • Anyone can submit PRs linked with Issue.
  • Each PR is a small, focused change based on the latest official main branch; not necessarily fully polished
  • Automatically build via CI (GitHub Actions) and distribute binaries as artifacts, with source and binary matching
  • Intended as a place for experimentation and validation
  • Issues are not created here; they should be created in the official repository
  • No merging is required; each PR only triggers a CI build, with no expectation of review or ongoing maintenance from OpenAI maintainers

---

Expected operation of the community repository

• PR submissions are open, typically issue-driven and minimal in scope
• Automatically build via CI(GitHub Actions) and distribute binaries as artifacts (source and binary match)
• Follow the latest official branch (periodic sync)
• Stability is not guaranteed (prototype-oriented)

---

Expected benefits

For OpenAI

• Reduce workload
• Prevents experimental PRs from flowing into the official repository's PRs, allowing maintainers to focus on carefully reviewed, prioritized PRs
• Unlike PRs in the official repository, experimental PRs do NOT require review, allowing maintainers to focus on prioritized work
• NO need for triage, prioritization, or merging of experimental PRs
• NO ongoing maintenance burden for experimental implementations
• Ability to real-world behavior of multiple implementations without review overhead
• Externalize UX and direction validation
• More inputs for roadmap decisions
• Easier estimation of implementation cost
• Faster validation of fixes, including issues that occur only in specific environments

For the community

• PRs do not become completely wasted
• Enables a loop of implementation → testing → discussion
• Easier to discover than scattered forks
• Focused on a single problem or idea based on latest release, rather than combining multiple features as in typical forks
• People can try ready-to-run things instantly without any build setup, and contributors can see others using their work

---

Workflow image

1. Discuss ideas in official Issues
   → people interested in the problem space
   → identify problems and gather interest

2. Contributors implement ideas in the community repository
   → submit small, experimental PRs based on the latest main branch

3. Interested users try the generated binaries
   → low-friction validation in real environments

4. Feedback is aggregated back into official Issues
   → based on actual usage, not speculation
   → discussions are driven by real, working implementations

5. Maintainers adopt only validated directions into the official repository
   → curated, high-confidence integration

• Merging is intentionally not part of the workflow; each PR triggers a CI build
• Each PR independently produces a testable runnable binary artifact

---

Notes

There are already cases where people fork and publish their own builds, but:

• They are not easily discoverable from the official repository
• They often diverge from the official codebase or bundle multiple features, making it harder to evaluate specific ideas discussed in Issues
• The build cost is high (especially for Rust)

Because of this, many implementations are not widely tested.

This proposal aims to reduce such “untested implementations” and make discussions more grounded in actual usage.

To clarify the expected involvement:

The only required contribution from OpenAI would be a one-time setup:

• Creating (or approving) the repository
• Setting up CI to build PR-based artifacts

After that, no ongoing involvement is required.

OpenAI would not:

• Review or merge PRs
• Maintain or monitor the repository
• Take responsibility for builds or artifacts

So this introduces effectively zero ongoing cost.

OpenAI would:

• Continue working only in the official repository
• Continue discussions in official issues
• (Optionally) reference implementations if useful

---

The policy of not accepting PRs is reasonable.
However, having a place for experimentation would likely improve both the quality and speed of discussions.

This is one possible approach to leverage community experimentation while maintaining the quality of the official repository.
Thanks.

---

PS:
To the OpenAI Codex team: thank you for building such a great tool. I really appreciate the effort to keep the development process open, and I’m looking forward to its continued evolution.

If you find this idea useful, please consider clicking the ↑ upvote and 👍buttons to help bring it to the attention of the OpenAI team.
May the source be with you.

[etraut-openai]

Thanks for starting the discussion. It's an interesting idea, but I see a number of challenges.

How would this differ from each contributor creating a separate fork? Experimentation is already possible (and encouraged) on forks. Is this mainly solving a build / distribution problem?

Are you envisioning a single branch that accumulates all PRs or a separate branch (and therefore independent binary) for each PR? I see problems with both approaches. If all contributions in this exploratory repo were merged onto a single branch, it would solve the discoverability problem, but it would quickly accumulate regressions. With no one responsible for maintaining and fixing these regressions, it would quickly become unusable. It would also diverge from the "main" branch in a way that will eventually make it difficult or impossible to merge changes from the official repo.

If each PR results in its own branch (and associated binaries), wouldn't that have the same discoverability problems that forks do today?

Who would maintain this? Who would fix flaky tests and keep CI clean? What versioning scheme would it use? Where would the binaries be hosted? What protections would be in place to prevent malware or spyware from being incorporated into the code and/or binaries?

One of the big problems with the old system (where anyone could post a PR) is that there was no notion of prioritization. Anyone who ran into an obscure bug or had a niche feature request could "skip the line" and request Codex team members to consider their PR in front of all the other bug-fix and feature-implementation priorities that affected many more users. This proposal seems like it would be similarly problematic.