in a supervised setting， why are the results of the test set tested during training,

[Gyyyym]

Hello, in a supervised setting, why are the results of the test set tested during training, so that the results of the test set will not be high?
I don't quite understand this part, please advise, thanks.

[YJiangcm]

Hi, I am a little confused about your question. During training, we only use the dev set for evaluation in order to save the best checkpoint. Do you mean that you cannot reproduce the results under supervised settings?

[Gyyyym]

Hi, I am a little confused about your question. During training, we only use the dev set for evaluation in order to save the best checkpoint. Do you mean that you cannot reproduce the results under supervised settings?

I use the dataset of another task. The accuracy rate on the training set, validation set and test set can reach 100%, and the loss value is above 4.0. I feel this result is very strange. I see that there is a 5-fold cross in the code. is it also trained on the test set, resulting in a very high accuracy rate? and Why is 5-fold cross-validation used in the evaluation process? Thank you very much for your reply

[YJiangcm]

Why is 5-fold cross-validation is used:
After the model is well trained, it can derive sentence embedding vectors, which can be directly used to compute the cosine similarity for STS tasks. However, for transfer tasks like text classification tasks, we still need to train a logistic regression classifier on top of (frozen) sentence embeddings. So, after the training of DCPCSE is finished, we use the 5-fold cross-validation following SentEval toolkit (https://github.com/facebookresearch/SentEval) to evaluate the performance on transfer tasks.

Why "The accuracy rate on the training set, validation set, and test set can reach 100%, and the loss value is above 4.0":
I haven't come across such a phenomenon yet. Which dataset did you use?

[Gyyyym]

Why is 5-fold cross-validation is used: After the model is well trained, it can derive sentence embedding vectors, which can be directly used to compute the cosine similarity for STS tasks. However, for transfer tasks like text classification tasks, we still need to train a logistic regression classifier on top of (frozen) sentence embeddings. So, after the training of DCPCSE is finished, we use the 5-fold cross-validation following SentEval toolkit (https://github.com/facebookresearch/SentEval) to evaluate the performance on transfer tasks.

Why "The accuracy rate on the training set, validation set, and test set can reach 100%, and the loss value is above 4.0": I haven't come across such a phenomenon yet. Which dataset did you use?

I use CMV datasets. I don't know why the loss value doesn't drop and the accuracy improves still.