Get rid of LLM disclosure checkboxes - #160785
Conversation
These had a bunch of issues: - They're extra work every time someone opens a PR - They didn't show up at all if people used `gh pr create` - They had "bad vibes" -- reviewers often don't want to think about LLMs, and adding a checkbox makes them very "in-your-face" for the reviewer. - Every PR now shows "1 of 2 tasks", which is useless noise. - The hovered PR description is now useless. Replace them with an HTML comment that says "remember to disclose if you used an LLM". This seems ok and low-noise for now. If we find that people are ignoring the comment, we could edit the triagebot welcome message to include a reminder and a link to the policy.
rustbot
added
the
S-waiting-on-review
|
rustbot has assigned @Mark-Simulacrum. Use Why was this reviewer chosen?The reviewer was selected based on:
|
| If you do not check a box, a reviewer may ask you whether an LLM was involved. | ||
| If you used an LLM to generate code, please disclose that according to our [guidelines][disclosure guidelines]. | ||
| LLM contributions are not banned, but are held to a higher standard of review and correctness. | ||
| If you do not want your disclosure to be part of the permanent git history, add `<!-- homu-ignore:start` before it. |
There was a problem hiding this comment.
Is the tooling smart enough to not parse the homu-ignore when it's within a comment?
|
A potential problem is: If a PR doesn't mention LLMs, we're left guessing whether it's a human who has seen the instructions, or if it's an LLM that hasn't seen the instructions. |
|
yes. that’s why the penalty is “a reviewer might ask you whether your PR is LLM-generated”, not “close the PR on sight”. |
| If you do not check a box, a reviewer may ask you whether an LLM was involved. | ||
| If you used an LLM to generate code, please disclose that according to our [guidelines][disclosure guidelines]. | ||
| LLM contributions are not banned, but are held to a higher standard of review and correctness. | ||
| If you do not want your disclosure to be part of the permanent git history, add `<!-- homu-ignore:start` before it. |
There was a problem hiding this comment.
Could we just drop this line instead, please? The disclosure is useful information.
There was a problem hiding this comment.
people should have control over what they put in a permanent record.
6 participants
These had a bunch of issues:
gh pr createReplace them with an HTML comment that says "remember to disclose if you used an LLM". This seems ok and low-noise for now.
If we find that people are ignoring the comment,
we could edit the triagebot welcome message to include a reminder and a link to the policy (cc @Kobzol, i believe you'd planned to do this already).