[Research] Add toxicity detection pipeline

[TheDutchDevil]

Description (*)

Did you ever consider the addition of a toxicity detection bot to this project?

We’re Nathan Cassee and Alexander Serebrenik (Eindhoven University of Technology, The Netherlands), Nicole Novielli (University of Bari, Italy), Christian Kastner (Carnegie Mellon University, USA), and Bogdan Vasilescu (Carnegie Mellon University, USA). And we are conducting research to understand the effectiveness of toxicity detection bots on GitHub. As part of our research we want to understand the impact of a state of the art toxicity detection bot in your project. We hope that a better knowledge and understanding of how these toxicity bots operate can be used to further improve the health of open-source projects.

To participate in this experiment we ask you to adopt a toxicity bot in magento/magento2. You can adopt the bot by merging this pull-request. This bot will monitor issues and pull-requests for comments containing toxicity, and will post a comment if it detects toxicity. Additionally, the bot will securely store comments and edits or deletions made to those comments. This will allow us to study the frequency of toxic comments, and whether and how toxic comments are edited or deleted.

We expect that the toxicity bot reduces toxicity in issues or in pull-requests. However, there might be cases where the bot responds in issues or pull-requests where there is no toxicity (a false positive) and this might distract on-topic discussions.

Practicalities

Your participation in this study is completely voluntary, at any point in time you can retract your project from this study. This can be done by disabling the toxicity bot, or by sending us an email. Additionally, if you don’t want us to use the results of your project in the analysis of the study, you can always email us to inform us that you want to retract your data from the study.

The study itself will run for roughly three months, at the end of this time we will open a PR in your project to remove the toxicity bot from the project. If after the experiment you want to keep using the toxicity bot we can also provide you with a version of the toxicity bot that does not record telemetry.

The data collected for this study will be stored securely on a private server, and the raw data will only be available to the researchers involved in this study. When we release or publicize results of the study the results will be released anonymously or in an aggregated way.

This study has been approved by the Ethical Review Board of the Eindhoven University of Technology.

Closing and Survey

If you have any questions about the bot feel free to ask them here, or mail them to n.w.cassee@tue.nl.

If you are not interested in participating we would really appreciate it if you would let us know why you are not participating.

Secondly, it would be really helpful if everyone involved in the project could respond to the following survey on your expectations of the bot, especially if you are not interested in adopting the bot (https://docs.google.com/forms/d/e/1FAIpQLSdaioKzNeYjeYqbo2MpAvCGBgClo4zeSqDlA2Lx4o5KJKJ24A/viewform)!

Questions or comments

Note: We are not sure how best to approach projects to participate in this study, if you thought this PR was spammy, or unhelpful, please let us know so we can modify how we invite projects!

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

[m2-assistant]

Hi @TheDutchDevil. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE,
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel