Detect Hate Speech

A small solution for targeting Homophobic and Sexist Tweets to be reported to Twitter by Data For Good, Israel.

Contents

• 1   Data Collection
• 2   Solution
• 3   Webapp
  • 3.1   Locally
  • 3.2   Live example with Heroku
• 4   Metric Results
• 5   Installation
• 6   Contributing
• 7   Research
• 8   Improvement and Next Steps
• 9   Installing for development
• 10   Credits

1   Data Collection

We merge multiple dataset:

• We are using hate speech dataset from

  • https://github.com/ZeerakW/hatespeech and
  • https://data.world/thomasrdavidson/hate-speech-and-offensive-language

• We are scrapping sexist and homophobic tweets thanks to hashtags and doing special annotation.

2   Solution

A Text Classification Model using a Calibrated SGD model and TF-IDF features.

3   Webapp

We are using Streamlit for this tool.

3.1   Locally

cd DetectHateSpeech
streamlit run webapp/webapp.py

3.2   Live example with Heroku

For the webapp to work on heroku, 3 files are added: Procfile, runtime.txt and setup.sh.

You can access to the live webapp here.

4   Metric Results

            precision    recall  f1-score   support

homophobia       0.88      0.82      0.85        17
      none       0.93      0.98      0.95      4949
    sexism       0.84      0.53      0.65       836

 micro avg       0.92      0.92      0.92      5802
 macro avg       0.88      0.78      0.82      5802
 weighted avg       0.91      0.92      0.91      5802

---

5   Installation

pip install -r requirements.txt

6   Contributing

Author and current maintainer are the Data For Good Team.

You are more than welcome to approach us for help and contributions are very welcomed!

7   Research

You can find our research notebook here.

We tried different methods to tackle this problem: Word2Vec, Transformers, NN.

To be continued...

8   Improvement and Next Steps

• Collect more data with less biased labelling.
• Use this article: Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "Deep Learning Models for Multilingual Hate Speech Detection". https://arxiv.org/pdf/2004.06465.pdf. We used it in the research part, let's implement it!
• Working on the improving the model infrastructure.
• Creating a way to integrate our model and WebApp with Twitter or other system for social media moderators (Add-On, API)

9   Installing for development

Clone:

git clone https://github.com/DataforGoodIsrael/DetectHateSpeech.git

10   Credits

Created by Jeremy Atia and Samuel Jefroykin from Data For Good Israel.

Contact us at hello@dataforgoodisrael.com