-
Notifications
You must be signed in to change notification settings - Fork 9
Error analysis: visualization of AM dependency trees
When you want to perform error analysis on AM dependency trees, this can be achieved with the following steps:
cd to analyzers/ and call:
python prepare_visualize.py system_output.amconll gold_file.amconll some/directory/
Where system_output.amconll are the parser outputs, gold_file.amconll contains gold AM dependency trees and some/directory/ directory will serve as storage of intermediate files (conll-like files that MaltEval can read). This command intersects the two amconll files and sorts them by sentence length so you can look at short examples first.
Now, we can call MaltEval.jar to visuzalize the trees and explore in the errors:
java -jar analyzers/MaltEval.jar -g some/directory/gold_file.amconll -s some/directory/system_output.amconll -v 1
If you point your cursor on a word, a little label will pop up, telling you its graph fragment and type.
MaltEval is quite powerful, see here for a documentation. There's also a paper with some usage examples from Nilsson and Nivre 2008.
cd to analyzers/ and call:
bash am_comparison_full.sh file1 file2 folder/
where file1 and file2 are the original amconll files you want to compare (this script calls the preparation python script for you) and folder is an empty, existing folder where the results will be put.
The bash script is documented/self-explanatory. It prints f-score and LAS/UAS, and will open the visualization tool. More detailed f-scores per dependency labels are printed to a file in the folder.
On the Coli servers, amconll files that are appropriate for the comparative visualization are in /proj/irtg/sempardata/mrp/amconll/PSD (replace PSD by DM or EDS for the other graphbanks). Note that not all frameworks have the same corpus as train and dev set.
If you don't want to compare with a gold standard but just want to have a look at AM dependency trees, nearly the same procedure can be used but condensed into a bash script (again, cd to analyzers/):
bash visualize_am_trees.sh your_file.amconll