chore: Update PR template with agent-specific testing guidance#48776
Merged
rafaeelaudibert merged 1 commit intomasterfrom Feb 24, 2026
Conversation
It's very common for agents opening new PRs against our codebase to hallucinate tests they did to guarantee the code works even when they didn't even try testing it. Let's try and steer them away from that by telling them it's fine to say they haven't done that.
rafaeelaudibert
requested review from
a team,
MattBro and
fercgomes
and removed request for
a team
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| <!-- Include automated tests if possible, otherwise describe the manual testing routine. --> | ||
|
|
||
| <!-- Docs reminder: If this change requires updated docs, please do that! Engineers are the primary people responsible for their documentation. 🙌 --> | ||
| <!-- If you are an agent writing this, do NOT include manual tasks you have NOT completed. You can clearly outline you're simply an agent and you haven't tested this manually except for code-based unit/integration tests --> |
Contributor
There was a problem hiding this comment.
Removing the documentation reminder entirely affects human contributors. Consider keeping both the docs reminder AND the agent guidance, e.g.:
Suggested change
| <!-- If you are an agent writing this, do NOT include manual tasks you have NOT completed. You can clearly outline you're simply an agent and you haven't tested this manually except for code-based unit/integration tests --> | |
| <!-- Docs reminder: If this change requires updated docs, please do that! Engineers are the primary people responsible for their documentation. 🙌 --> | |
| <!-- If you are an AI agent writing this, do NOT include manual tasks you have NOT completed. You can clearly state you're an AI agent and you haven't tested this manually except for code-based unit/integration tests --> |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/pull_request_template.md
Line: 17
Comment:
Removing the documentation reminder entirely affects human contributors. Consider keeping both the docs reminder AND the agent guidance, e.g.:
```suggestion
<!-- Docs reminder: If this change requires updated docs, please do that! Engineers are the primary people responsible for their documentation. 🙌 -->
<!-- If you are an AI agent writing this, do NOT include manual tasks you have NOT completed. You can clearly state you're an AI agent and you haven't tested this manually except for code-based unit/integration tests -->
```
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.| <!-- Include automated tests if possible, otherwise describe the manual testing routine. --> | ||
|
|
||
| <!-- Docs reminder: If this change requires updated docs, please do that! Engineers are the primary people responsible for their documentation. 🙌 --> | ||
| <!-- If you are an agent writing this, do NOT include manual tasks you have NOT completed. You can clearly outline you're simply an agent and you haven't tested this manually except for code-based unit/integration tests --> |
Contributor
There was a problem hiding this comment.
Grammar: use "you are an AI agent" instead of "you're simply an agent" for clearer, more professional wording
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/pull_request_template.md
Line: 17
Comment:
Grammar: use "you are an AI agent" instead of "you're simply an agent" for clearer, more professional wording
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.
Contributor
There was a problem hiding this comment.
actually I'll agree with the robot. The first phrase is fine, but the second might make the AI doubt itself. AI responds better to direct instructions
Contributor
|
Size Change: 0 B Total Size: 100 MB ℹ️ View Unchanged
|
fercgomes
approved these changes
Feb 24, 2026
Member
Author
Merge activity
|
rafaeelaudibert
deleted the
rafa/02-23-refactor-steer-agents-away-from-hallucinating-how-they-tested-code
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The pull request template needed to be updated to provide clearer guidance for AI agents contributing code, ensuring they don't claim to have performed manual testing they cannot actually complete.
Changes
Updated the pull request template to replace the documentation reminder with specific instructions for AI agents, clarifying that they should not include manual tasks they haven't completed and can identify themselves as agents who have only performed code-based unit/integration tests.
How did you test this code?
This is a documentation change to the pull request template. The modification was verified by reviewing the template content to ensure the new guidance is clear and appropriate for both human contributors and AI agents.
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Publish to changelog?
No - this is an internal template change that doesn't affect the product functionality.