Draft policy on the usage of generative AI tools#177
Draft policy on the usage of generative AI tools#177santisoler wants to merge 2 commits intomainfrom
Conversation
mdtanker
left a comment
There was a problem hiding this comment.
This looks really good to me! I think it sets clear boundaries but isn't too strict. I've added a few minor suggestions.
| Issues, and any communication channel of the project. | ||
| - ❌ Pasting outputs of genAI content directly into Pull Requests, Issues, and | ||
| any communication channel of the project. | ||
| - ❌ Use of genAI tools to circumvent license restrictions. For example, |
There was a problem hiding this comment.
The other points in this section I think are good, but this seems a bit niche, at least it's not something I've heard about. But if it's a common issue, then all good to leave it!
There was a problem hiding this comment.
Yes, I know. I wanted to list it still, just to make it explicit in case someone plans to use AI tools to build a reimplementation of a piece of software, particularly old academic software that original researchers had not released under open-source licenses.
The most relevant case of "license laundry" is what happened in chardet: a maintainer overwrote the entire LGPL-licensed code base with an AI generated output and released it under a more permissive license. Leaving a link here to the Issue raising the concern: https://github.com/chardet/chardet/issues/327.
There was a problem hiding this comment.
I think this is relevant to free software but not really in the context of a project like our right? Unless you think someone could do this as a module in a package?
There was a problem hiding this comment.
I'm OK with leaving this in since it's not doing any harm.
| license. | ||
| - ❌ Use of genAI tools in code reviews in any of our repositories. We highly | ||
| value the *peer-review* process as a way for everyone to learn from each | ||
| other's expertise. |
There was a problem hiding this comment.
Maybe we could offer an alternative here, and say something like: As an alternative to genAI tools for code review, we recommend code analysis tools such as PyLint or Ruff, which can catch many common code issues.
There was a problem hiding this comment.
Sure! We mention Ruff as part of the normal workflow. We can mention it here again.
There was a problem hiding this comment.
Now that I reread this item, I think not allowing genAI tools in code reviews is too broad. What I mean is not to ask a chatbot to review a whole PR. But acceptable uses of genAI during a code review process would be to check for grammar, write snippets that would show how the solution is not covering all cases, helping to write additional tests, etc.
I'm open to suggestions to improve this sentences.
leouieda
left a comment
There was a problem hiding this comment.
@santisoler thanks for getting this started. I think a lot of the policies I've seen around are way too lenient. I'd be a bit stricter on a few things and make it clear that the use of these tools is discouraged in many cases.
| - [Testing your code](#testing-your-code) | ||
| - [Documentation](#documentation) | ||
| - [Code Review](#code-review) | ||
| * [Policy on Usage of Generative AI Tools](#policy-on-usage-of-generative-ai-tools) |
There was a problem hiding this comment.
| * [Policy on Usage of Generative AI Tools](#policy-on-usage-of-generative-ai-tools) | |
| * [Usage of Generative AI Tools](#usage-of-generative-ai-tools) |
| [get in touch](#how-can-i-talk-to-you). | ||
|
|
||
|
|
||
| ## Policy on Usage of Generative AI Tools |
There was a problem hiding this comment.
| ## Policy on Usage of Generative AI Tools | |
| ## Usage of Generative AI Tools |
| Generative AI tools (genAI tools) are being widely used for generating these | ||
| types of content. We understand that some contributors might find them useful, | ||
| and would like to include them in their workflows and use them when | ||
| contributing to open-source projects. |
There was a problem hiding this comment.
Move this paragraph to the start of this section. Then everything else flows a bit better.
There was a problem hiding this comment.
Also might be worth
| Generative AI tools (genAI tools) are being widely used for generating these | |
| types of content. We understand that some contributors might find them useful, | |
| and would like to include them in their workflows and use them when | |
| contributing to open-source projects. | |
| Generative AI tools (genAI tools, including chatbots and coding agents) are being widely used for generating these | |
| types of content. We understand that some contributors might find them useful, | |
| and would like to include them in their workflows and use them when | |
| contributing to open-source projects. |
|
|
||
| Usage of genAI tools in contributions to Fatiando a Terra are not forbidden, | ||
| but when being used as an aid in developing code and editing documentation, | ||
| **contributors must**: |
There was a problem hiding this comment.
I'd rephrase this to make it clear that the usage is not forbidden exactly but strongly discouraged.
| **contributors must**: | |
| Usage of genAI tools in contributions to Fatiando a Terra is **strongly discouraged** but are not forbidden (see below). | |
| When such tools are used, **contributors must**: |
| 1. ensure that they fully understand the proposed changes and can explain why | ||
| they are correct, and | ||
| 2. communicate that genAI tools have been used in the process. |
There was a problem hiding this comment.
| 1. ensure that they fully understand the proposed changes and can explain why | |
| they are correct, and | |
| 2. communicate that genAI tools have been used in the process. | |
| 1. Ensure that they **fully understand the proposed changes** and can explain why | |
| they are correct with the aid of a bot. | |
| 2. **Declare** that genAI tools have been used in the process (this is covered in our PR template). |
| Maintainers reserve the right, at their sole discretion, to close Pull | ||
| Requests, Issues, and delete comments that incur in unacceptable usage of genAI | ||
| tools. |
There was a problem hiding this comment.
| Maintainers reserve the right, at their sole discretion, to close Pull | |
| Requests, Issues, and delete comments that incur in unacceptable usage of genAI | |
| tools. | |
| The lists above are not exhaustive and serve as examples. | |
| Maintainers reserve the right, at their sole discretion, to close Pull | |
| Requests, Issues, and delete comments that they judge to incur in unacceptable usage of genAI | |
| tools. |
| Issues, and any communication channel of the project. | ||
| - ❌ Pasting outputs of genAI content directly into Pull Requests, Issues, and | ||
| any communication channel of the project. | ||
| - ❌ Use of genAI tools to circumvent license restrictions. For example, |
There was a problem hiding this comment.
I think this is relevant to free software but not really in the context of a project like our right? Unless you think someone could do this as a module in a package?
| Issues, and any communication channel of the project. | ||
| - ❌ Pasting outputs of genAI content directly into Pull Requests, Issues, and | ||
| any communication channel of the project. | ||
| - ❌ Use of genAI tools to circumvent license restrictions. For example, |
There was a problem hiding this comment.
I'm OK with leaving this in since it's not doing any harm.
| - ❌ Pasting outputs of genAI content directly into Pull Requests, Issues, and | ||
| any communication channel of the project. |
There was a problem hiding this comment.
| - ❌ Pasting outputs of genAI content directly into Pull Requests, Issues, and | |
| any communication channel of the project. | |
| - ❌ Pasting outputs of genAI content without human revision directly into Pull Requests, Issues, and | |
| any communication channel of the project. |
| - ⚠️ Write code for us. While they can save some *typing time*, one needs to | ||
| spend time reviewing the generated output and making sure we fully understand | ||
| the proposed solution. |
There was a problem hiding this comment.
| - ⚠️ Write code for us. While they can save some *typing time*, one needs to | |
| spend time reviewing the generated output and making sure we fully understand | |
| the proposed solution. | |
| - ⚠️ Write code for us. While they can save you some *typing time*, we need to | |
| spend time reviewing the generated output to make sure we fully understand | |
| the proposed solution and then maintain it in the future. Typing time is not the bottleneck in our projects. |
Add a policy on the usage of generative AI within the project to
CONTRIBUTING.md.Relevant issues/PRs:
Fixes #171