UCCA preprocessing

UCCA preprocessing

Producing UCCA training data for the AM parser

Step 1: Extract tokens from companion data

mkdir -p data_ucca/companion_tokens
python ucca/get_companion_tokenization.py data_ucca/companion/ucca/ data_ucca/companion_tokens/

Step 2: Create Alto corpora

mkdir -p data_ucca/alto_corpus
python ucca/convert_training_into_alto_corpus.py data_ucca/ucca/ data_ucca/companion_tokens/ data_ucca/alto_corpus/

This will take a few minutes, and will then write two files in Alto format, training.txt and test.txt, to the alto_corpus directory.

Step 3: Decompose and write AM-CoNLL file

Once we run CreateCorpus from am-tools, we obtain an am-conll file. This file will serve as input to the am-parser model.

Contraction sanity check

To find out how accurately the contractions can be reversed, continue as follows:

Step 4: Evaluate AM-CoNLL file to MRP graphs

This am-conll file can be converted back to an MRP file using EvaluateMRP in am-tools.

Step 5: Uncontract edges in MRP graphs

python3 decompress_mrp.py mrp/with/contracted/edges outdir/

Step 6: Compare against original corpus

using mtool