You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
a good job!
I have a very concerned question that I hope can be answered. Reading the article, the standard for measuring model capabilities should be the test set of the dataset? What if we directly select the sample in the train dataset that is most similar to the test set as the baseline result?
For example, if they are all text, directly embed them into vectors and then calculate the sample that is most similar to the test set.
The text was updated successfully, but these errors were encountered:
We select examples that are similar to the validation examples (which are also the in-context learning examples) from the training set, and we have the representation data selection baseline presented in table 3
In machine learning generally, it's prohibited to use any information from test set (including inputs and outputs) as it's basically cheating, and we adhere to this rule by using a small validation set for data selection
Maybe I misunderstood something here.. Let me know if it helps address your question.
a good job!
I have a very concerned question that I hope can be answered. Reading the article, the standard for measuring model capabilities should be the test set of the dataset? What if we directly select the sample in the train dataset that is most similar to the test set as the baseline result?
For example, if they are all text, directly embed them into vectors and then calculate the sample that is most similar to the test set.
The text was updated successfully, but these errors were encountered: