Added DST Teacher for multiwoz_v22 task #4656
Conversation
| @@ -373,6 +379,212 @@ def get_id_task_prefix(self): | |||
| return "MultiwozV22" | |||
|
|
|||
|
|
|||
| class MultiWOZv22DSTTeacher(tod_agents.TodUserSimulatorTeacher): | |||
There was a problem hiding this comment.
might be better to add a comment saying this teacher is needed for reproducing the Joint Goal Accuracy values reported in the simpleTOD & SOLOIST papers.
There was a problem hiding this comment.
Is it better now ? @chinnadhurai
|
Looks okay code-wise, though I haven't done a through review. Should this be in ParlAI-Internal rather than public ParlAI though? |
parlai/tasks/multiwoz_v22/agents.py
Outdated
| @@ -393,5 +605,5 @@ class SingleApiSchemaAgent(MultiwozV22Parser, tod_agents.TodSingleApiSchemaAgent | |||
| pass | |||
|
|
|||
|
|
|||
| class DefaultTeacher(SystemTeacher): | |||
There was a problem hiding this comment.
would be better to keep the default to SystemTeacher as is.
we need this teacher for the actual DST task in Multiwoz and reproduce JGA reported in the SOTA models. So, figured it might make sense to push to the public one. |
|
@stephenroller Can you please look into this ? It has been here for sometime now. |
There was a problem hiding this comment.
I'll defer to @moyapchen ans @klshuster here. Don't see any issue though
There was a problem hiding this comment.
deferring to @moyapchen for final say but looks OK to me
could we generate test files for it as well?
|
@moyapchen Can you please see this ? |
The DST Teacher for
multiwoz_v22is not present. Adding this teacher would allowmultiwoz_v22to be usable for Dialog State Tracking task.cc @chinnadhurai