Prediction and evaluation on test data

Also compare the section on reproducing the experiments with the pretrained models

To evaluate your model, you can run the bash script scripts/predict.sh which is also used for our default models. You simply need to give it an additional option to specify you own model (which is an output of the training step):

bash scripts/predict.sh -i <input corpus file> -T <graph formalism> -o <output directory> -m <your model>

In detail, the options of scripts/predict.sh are as follows:

Required arguments:

• -i : Graph corpus in the original format. For example, the DM dev set in .sdp format. For EDS, make this the test.amr file that contains the gold graphs in PENMAN notation. For AMR, use the directory which contains all the test corpus files (e.g. data/amrs/split/test in the official AMR corpora)
• -o : where the results will be stored.
• -T . Possible options: DM, PAS, PSD, AMR-2017 and EDS.

Optional arguments:

• -m file in .tar.gz format. If not given, the default model path $defaultmodel is used. If that file does not exist, it will be downloaded automatically.
• -f faster, less accurate evaluation (flag; default false)
• -g which GPU to use (its ID, i.e. 0, 1 or 2 etc). Default is -1, using CPU instead