about baseline score

[everks]

I wonder where the baseline score came from? I don't find the corresponding score in the original paper, and the reproducible experiment results are quite different from that. For example, the Roberta-classifier performance on clinc 5-shot is much lower than the score here cause it is just a simple RobertaForSequenceClassification call. I don't think my code is wrong. Can you please tell me where the score comes from?

[jianguoz]

@everks Which baselines score are you targeting? The Roberta-classifier baseline score is reported by DNNC. The intent accuracy is when the threshold is set to 0 in the paper, and based on the DNNC code, the RoBERTa-classifier performs reasonably well.

[everks]

I am sorry, but I look through the paper again and don't find any experiments about BANKING77 and HWU64.

[jianguoz]

@everks Yes, you are right. They are not reported on the DNNC paper. We asked the authors (I am also one author of the DNNC paper) to provide numbers on the additional two datasets. Sorry that I did not state it clearly in the previous reply.