Codex connector may silently commit truncated/corrupted data to GitHub

[topquark22]

What issue are you seeing?

GitHub Connector Data-Loss Bug: Response-Size Limit Can Result in Corrupted Repository Files

Summary

The ChatGPT GitHub connector appears to have a serious data-integrity bug involving its internal tool response-size limit.

When an existing repository file is too large to be returned completely within the tool response budget, the connector reports that the response has been truncated. However, a subsequent edit can still proceed as a full-file replacement.

Instead of failing the operation or rolling it back, the incomplete representation can apparently be committed to GitHub, resulting in silent truncation and corruption of the repository file.

A response-size limit should never result in a successful destructive write.

Concrete Example

Repository:

topquark22/ClipCrank

File:

docs/TEST_PLAN.md

Commit exhibiting the failure:

68bcb19520315b9c4b1ced94e77964648abff09a

Commit message:

Document frame-rate conversion tests

The requested operation was limited to:

1. Changing the documented smoke-test count from 50 to 52.
2. Adding documentation for two new frame-rate tests.

Those requested edits were made correctly.

However, the same commit unexpectedly deleted a substantial portion of the end of TEST_PLAN.md.

The resulting file was truncated in the middle of a sentence and ended with: For clips

The Git commit confirms that this was actual repository data loss, not merely truncation of what ChatGPT displayed to the user.

Relevant Connector Behavior

During these operations, GitHub connector reads have explicitly returned messages such as:

Response output was truncated at a line boundary to fit the tool response budget.

This appears to be an internal response-size limit rather than an account usage or billing limit.

The GitHub update operation, meanwhile, replaces the complete contents of the target file.

The observed failure is consistent with the following sequence:

1. The connector reads the existing file.
2. The complete file exceeds the internal tool response-size limit.
3. Only a partial representation of the file is made available for the operation.
4. The requested small edit is applied to that partial representation.
5. The connector performs a full-file replacement using incomplete content.
6. GitHub successfully commits that replacement.
7. Everything beyond the truncated representation is silently lost.

The precise internal mechanism would need to be confirmed by OpenAI, but this sequence is consistent with the observable behavior and the resulting Git commit.

Expected Transactional Behavior

Reaching an internal response-size limit must be treated as a failure condition if the complete source file is required to perform a safe write.

The operation should behave transactionally:

Either the complete requested edit is applied while preserving all unrelated file content, or no write occurs at all.

If the connector cannot obtain or represent the complete original file, it should abort the update and leave the repository unchanged.

It must never convert a partial read into a successful full-file replacement.

Possible safe behaviors include:

• aborting the write and reporting that the file exceeds the response-size limit,
• automatically retrieving the file in multiple chunks before constructing the replacement,
• using a patch-based editing mechanism that does not require retransmitting the entire file,
• or performing an integrity check before committing the replacement.

Any of these would be preferable to committing incomplete data.

Failure Atomicity

This is fundamentally a failure-atomicity problem.

An internal resource or response-size limit should cause the operation to fail cleanly. It should not allow a partially completed operation to become persistent repository state.

In database terminology, the expected behavior is effectively:

COMMIT the complete intended change, or ROLLBACK and leave the original file untouched.

The current behavior appears capable of committing an incomplete intermediate representation.

Reproducibility

This has occurred multiple times while editing files in the same repository.

The pattern is:

1. Request a small modification to an existing, moderately large text file.
2. The connector encounters or is subject to a tool response-size limit.
3. The requested change is made.
4. Unrelated content from later in the file is silently deleted.
5. The corrupted file is committed successfully.

The problem has therefore not behaved as an isolated editing error.

Severity

I consider this a serious data-integrity bug.

Git version history makes recovery possible in this particular application, but the existence of version control does not make silent destructive writes acceptable.

A user may reasonably assume that asking the connector to change a few lines cannot delete unrelated portions of the file.

More importantly, an internal implementation limit must never be allowed to turn into persistent user-data corruption.

Requested Fix

Please ensure that GitHub connector write operations are failure-atomic with respect to incomplete or truncated tool responses.

Specifically:

If the complete source data required for a safe full-file replacement is unavailable because of a response-size limit, the connector must refuse or roll back the write rather than commit incomplete data.

Ideally this invariant should be enforced by the connector/tooling layer itself rather than relying on the language model to recognize that a previous tool response was truncated.

What steps can reproduce the bug?

Steps to reproduce

The bug is intermittent, but the following conditions appear to increase the likelihood of reproducing it.

1. Create a GitHub repository containing a moderately large text file, preferably Markdown, with enough content that retrieving the complete file approaches or exceeds the connector's tool response-size limit.

   A useful test file would be several hundred lines long and should contain easily recognizable sections near the end, for example:

   ## SECTION_NEAR_END_DO_NOT_DELETE

   followed by several paragraphs of text.

2. Connect the repository to ChatGPT/Codex through the GitHub connector.

3. Ask the assistant to make a very small edit near the beginning or middle of the file, for example:

   "Change '50 tests' to '52 tests'. Make no other changes."

4. The connector may return the source file with a diagnostic similar to:

   Response output was truncated at a line boundary to fit the tool response budget.

5. Allow the requested GitHub edit to proceed.

6. Inspect the resulting Git commit rather than merely checking whether the requested edit succeeded.

Failure condition

The requested small edit is present, but unrelated material near the end of the file has been deleted. In observed failures, the replacement file ended abruptly in the middle of a sentence.

Expected behavior

If the connector cannot retrieve or represent the complete source file because of its response-size limit, the write operation should abort without modifying the repository.

The operation should be failure-atomic:

• complete source available → apply requested edit and commit;
• complete source unavailable → do not commit anything.

A partial source representation must never be used as the contents of a successful full-file replacement.

Reproduction notes

The problem does not occur on every edit.

To increase the chance of reproducing it:

• use a longer text file;
• request only a small edit near the beginning or middle;
• leave substantial unchanged content after the edited region;
• repeat the operation several times on fresh copies of the original file;
• inspect the resulting Git diff after every attempt;
• record whether the connector reported that its read response was truncated because of the tool response budget.

A particularly useful automated reproduction test would generate files of progressively increasing size and repeat the same one-line edit until the connector crosses its response-size threshold.

The test should assert that the file's sentinel text near the end remains present after every successful write.

For example, after changing a line near the beginning, verify that:

SECTION_NEAR_END_DO_NOT_DELETE

still exists in the committed version.

If the connector cannot preserve that sentinel because it did not receive the complete source file, the update should fail rather than commit.

What is the expected behavior?

No response

Additional information

No response

[VallesMarinerisExplorer]

I am interested in this also, how often has it happened to you?