Real Toxicity Prompts

Analysis of the toxicity of various models.

Compared to the original repository, adds support for:

• Detoxify open-source toxicity classifier
• GPT-J models, using the HuggingFace port from finetuneanon

Misc changes:

• Pinned transformers versions as the code is not compatible with recent releases

Documentation/scratch pad on Notion: https://www.notion.so/Toxicity-b63e23946ab8483686a86b1acf586db1

Usage

$ python run_prompts_experiment.py --help
Usage: run_prompts_experiment.py [OPTIONS] OUTPUT_DIR

Options:
  --dataset-file TEXT             JSONL file containing prompts data. Each row
                                  must contain a prompt at
                                  `row["prompt"]["text"]`.
  --use-eos / --use-dataset       Whether to use EOS or a dataset file for
                                  generation.
  --model TEXT                    Equivalent to `model_name_or_path` in
                                  transformers.  [required]
  --model-type [gpt3|gpt2|gpt2-affect|gpt2-ctrl|gpt2-greedy|gpt2-naughty-list|pplm|ctrl|openai-gpt|xlnet]
                                  [required]
  --perspective-rate-limit INTEGER
  --n INTEGER                     Number of samples to generate for each
                                  prompt. When used with --eos
  --max-tokens INTEGER            Number of tokens (usually BPE) to generate
                                  for each prompt.
  --batch-size INTEGER
  --resume / --no-resume
  --help                          Show this message and exit.

Prompted generation

Unprompted generation

Original README

This repository is forked from https://github.com/allenai/real-toxicity-prompts. Below is the original README from that repo.

[Paper link] [Demo link] [Data link]

Work in progress, please revisit this page soon for more in-depth instructions.

Installing

Run the commands provided in order to set up your environment:

1. git clone https://github.com/allenai/real-toxicity-prompts.git
2. cd real-toxicity-prompts
3. Install Anaconda, then run conda env create -f environment.yml to create a new Anaconda environment named rtp with all necessary dependencies.
4. Run conda activate rtp before running any further scripts or notebooks.

Getting Started

Getting the data

Data processing steps are forthcoming.

Getting API access

We used the following two APIs to run our experiments:

1. Perspective API to score text for toxicity
2. OpenAI API to generate text with the GPT-3 model

Reproducing the experiments

The prompts experiments can be run from scripts/run_prompts_experiments.py.

Type python -m scripts.run_prompts_experiments --help to get a list of the available models and arguments that can be used.