Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 

README.md

SAMSA

SAMSA Evaluation Metric and Human Evaluation Benchmark for Text Simplification

If you use the metric or the benchmark, please cite the following paper:

  Semantic Structural Evaluation for Text Simplification
  Elior Sulem, Omri Abend and Ari Rappoport
  Proc. of NAACL 2018

Dataset

./human_evaluation_benchmark.ods

Human evaluation scores given by 5 annotators for the 4 elicitation questions described in the paper. Each annotator scored the same 700 (input,output) pairs.

The source sentences and the system outputs can be found at http://homepages.inf.ed.ac.uk/snaraya2/data/simplification-2016.tgz.

Code

Requirements:

  1. Python NLTK

  2. UCCA toolkit

  3. Monolingual word aligner

  4. The TUPA parser for parsing the source side.

Contents:

./scene_sentence_extraction.py

./scene_sentence_alignment.py

./SAMSA_score.py: The SAMSA metric

./SAMSA_abl_score.py: The SAMSA metric without the non-splitting penalty

About

Simplification Automatic evaluation Measure through Semantic Annotation

Resources

License

Releases

No releases published

Packages

No packages published

Languages

You can’t perform that action at this time.