UiO System for Negation Analysis Based on Sequence Labeling
Perl Python Awk
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
evaluation
patterns
sherlock
.gitignore
README.md
sherlock.py

README.md

sherlock

Train and test a system for negation scope resolution. This system was initially submitted to 2012 *sem shared task on negation scope resolution, ranking second in the closed track and first in the open track.

Requirements

Sherlock requires python 2.7 and up, and a working installation of Wapiti.

Usage

$> python sherlock.py -h 
usage: sherlock.py [-h] [--pos POS] [--lemma LEMMA] [--training TRAINING]
                   --pattern_scope PATTERN_SCOPE --pattern_event PATTERN_EVENT
                   [--scope_model SCOPE_MODEL] [--event_model EVENT_MODEL]
                   --testing TESTING --output OUTPUT --wapiti WAPITI
                   [--cleanup]

Negation scope resolution.

optional arguments:
  -h, --help            show this help message and exit
  --pos POS             specify the epe node property to use as pos. Defaults
                        to 'pos'
  --lemma LEMMA         specify the epe node property to use as pos. Defaults
                        to 'lemma'
  --training TRAINING   path to training data
  --pattern_scope PATTERN_SCOPE
                        path to a wapiti pattern for negation scopes
  --pattern_event PATTERN_EVENT
                        path to a wapiti pattern for negated events
  --scope_model SCOPE_MODEL
                        path to a pre-trained wapiti model for scopes
  --event_model EVENT_MODEL
                        path to a pre-trained wapiti model for negated events
  --testing TESTING     path to testing data
  --output OUTPUT       output file name
  --wapiti WAPITI       path to a wapiti executable, assumed to be in cwd if
                        absent
  --cleanup             if enabled, sherlock will remove all intermediate
                        files

Getting started

A minimal invocation of Sherlock:

python sherlock.py --training trainingset.epe+ --testing testset.epe+ --wapiti ~/path/to/wapiti --pattern_scope ~/patterns/scope.pattern --pattern_event ~/patterns/event.pattern --output output.*sem --cleanup --pos pos --lemma lemma

When run with the --cleanup flag, Sherlock will produce two files: gold_testset_test.*sem (the *sem version of the test set, with gold negation annotations) and output.*sem (the system output).

To evaluate, invoke the 2012 *sem shared task evaluation script:

perl evaluation/eval.cd-sco.pl -g gold_testset_test.*sem -s output.*sem

Data

This version of Sherlock accepts .epe+-formatted files as input. For more information, please see http://epe.nlpl.eu/.

Tweaking the system

Some pre- and post-processing heuristics, as well Wapiti hyperparameters, can be tweaked in sherlock/config.py. Please see the helpful comments in the file itself.

Cite

Copy and paste to your .bib:

@inproceedings{Lap:Vel:Ovr:12,
  author = {Emanuele Lapponi and Erik Velldal and Lilja Øvrelid and Jonathon Read},
  title = {{U}i{O}2:\ Sequence-labeling Negation Using Dependency Features},
  booktitle = {Proceedings of the 1st {J}oint {C}onference on {L}exical and {C}omputational {S}emantics},
  pages = {\fromto{319}{327}},
  year = 2012,
  address = {Montr\'{e}al, Canada},
}