Error analysis: visualization of AM dependency trees

When you want to perform error analysis on AM dependency trees, this can be achieved with the following steps:

cd to analyzers/ and call:

python prepare_visualize.py system_output.amconll gold_file.amconll some/directory/

Where system_output.amconll are the parser outputs, gold_file.amconll contains gold AM dependency trees and some/directory/ directory will serve as storage of intermediate files (conll-like files that MaltEval can read). This command intersects the two amconll files and sorts them by sentence length so you can look at short examples first.

Now, we can call MaltEval.jar to visuzalize the trees and explore in the errors:

java -jar analyzers/MaltEval.jar -g some/directory/gold_file.amconll -s some/directory/system_output.amconll -v 1

If you point your cursor on a word, a little label will pop up, telling you its graph fragment and type.

MaltEval is quite powerful, see here for a documentation. There's also a paper with some usage examples from Nilsson and Nivre 2008.

Full default comparison + visualization

In the analyzers folder, run

bash am_comparison_full.sh file1 file2 folder/

where file1 and file2 are the original amconll files you want to compare (the preparation python script is called within) and folder is an empty, existing folder where the results will be put. The bash file is documented/self-explanatory. This will print f-score and LAS/UAS, and will open the visualization tool. More detailed f-scores per dependency labels are printed to a file in the folder.

Non-comparative visualization

If you don't want to compare with a gold standard but just want to have a look at AM dependency trees, nearly the same procedure can be used but condensed into a bash script (again, cd to analyzers/):

bash visualize_am_trees.sh your_file.amconll