Skip to content
No description, website, or topics provided.
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
argument-classification
argument-similarity Update README.md Jun 25, 2019
.gitignore
LICENSE Initial commit Jun 24, 2019
NOTICE.txt
README.md Update README.md Aug 22, 2019

README.md

Argument Classification and Clustering using BERT

In our publication Classification and Clustering of Arguments with Contextualized Word Embeddings (ACL 2019) we fine-tuned the BERT network to:

  • Perform sentential argument classification (i.e., given a sentence with an argument for a controversial topic, classify this sentence as pro, con, or no argument). Details can be found in argument-classification/README.md
  • Estimate the argument similarity (0...1) given two sentences. This argument similarity score can be used in conjuction with hierarchical agglomerative clustering to perform aspect-based argument clustering. Details can be found in argument-similarity/README.md

Citation

If you find the implementation useful, please cite the following paper: Classification and Clustering of Arguments with Contextualized Word Embeddings

@InProceedings{Reimers:2019:ACL,
  author    = {Reimers, Nils, and Schiller, Benjamin and Beck, Tilman and Daxenberger, Johannes and Stab, Christian and Gurevych, Iryna},
  title     = {{Classification and Clustering of Arguments with Contextualized Word Embeddings}},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  month     = {07},
  year      = {2019},
  address   = {Florence, Italy},
  pages     = {567--578},
  url       = {https://arxiv.org/abs/1906.09821}
}

Contact person: Nils Reimers, Rnils@web.de

https://www.ukp.tu-darmstadt.de/

Don't hesitate to send us an e-mail or report an issue, if something is broken (and it shouldn't be) or if you have further questions.

This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.

Setup

This repository requires Python 3.5+ and PyTorch 0.4.1/1.0.0. It uses pytorch-pretrained-BERT version 0.6.2. See the pytorch-pretrained-BERT readme for further details on the installation. Usually, you can install as follows:

pip install pytorch-pretrained-bert==0.6.2 sklearn scipy

Argument Classification

Please see argument-classification/README.md for full details.

Given a sentence and a topic, classify if the sentence is a pro, con, or no argument. For example:

Topic: zoo
Sentence: Zoo confinement is psychologically damaging to animals.
Output Label: Argument_against

You can download pre-trained models from here, which were trained on all eight topics of the UKP Sentential Argument Mining Corpus:

See argument-classification/inference.py how to use these models for classifying new sentences.

In a leave-one-topic out evaluation, the BERT model achieves the following performance.

Classification Performance

Argument Similarity & Clustering

See argument-similarity/README.md for full details.

Given two sentences, the code in argument-similarity returns a value between 0 and 1 indicating the similarity between the arguments. This can be used for agglomorative clustering to find & cluster similar arguments.

You can download two pre-trained models:

See argument-similarity/inference.py for an example. This example computes the pairwise similarity between arguments on different topics. The output should be something like this for the model trained on the UKP corpus:

Predicted similarities (sorted by similarity):
Sentence A: Eating meat is not cruel or unethical; it is a natural part of the cycle of life.
Sentence B: It is cruel and unethical to kill animals for food when vegetarian options are available
Similarity: 0.99436545

Sentence A: Zoos are detrimental to animals' physical health.
Sentence B: Zoo confinement is psychologically damaging to animals.
Similarity: 0.99386144

[...]

Sentence A: It is cruel and unethical to kill animals for food when vegetarian options are available
Sentence B: Rising levels of human-produced gases released into the atmosphere create a greenhouse effect that traps heat and causes global warming.
Similarity: 0.0057242378

With the Misra AFS model, the output should be something like this:

Predicted similarities (sorted by similarity):
Sentence A: Zoos are detrimental to animals' physical health.
Sentence B: Zoo confinement is psychologically damaging to animals.
Similarity: 0.8723387

Sentence A: Eating meat is not cruel or unethical; it is a natural part of the cycle of life.
Sentence B: It is cruel and unethical to kill animals for food when vegetarian options are available
Similarity: 0.77635074

[...]

Sentence A: Zoos produce helpful scientific research.
Sentence B: Eating meat is not cruel or unethical; it is a natural part of the cycle of life.
Similarity: 0.20616204

Argument Similarty Performance

UKP Aspects Performance

AFS Performance

You can’t perform that action at this time.