In the test data will slots in system turn be released too? In the evaluation scripts only user turns are evaluated.

[shaomai00]

"The fields marked with * will be excluded in the test data released to the participants."
I'm wondering whether the slots in system turn should be predicted.
And could we use the former and the later system turn frames to predict the user turn?

[abhirast]

We just updated the README to clarify that only user turns will have missing annotations in the test set. Thank you for pointing out the ambiguity. The evaluation metrics are only defined on the user utterance, so slots in the system turn don't need to be predicted. Using the later system turns would imply using the future to predict the past. We don't allow this since it makes the model unsuitable for use in real world systems. Using the slot annotations in the previous system turns is allowed.