Are test_eval and test_llama the same data?

[Camellia-hz]

Dear Author, Hello, When I followed the data preparation in the challenge/readme documentation, I realized that test_eval.json and test_llama.json are essential the same data (derived from test.json),
and if I train my model using test_llama.json and then generate the output.json, and then ultimately evaluate it according to the documented methods (using output.json and test_eval.json), wouldn't that be equivalent to assess my model with the training set? Is my understanding wrong?

[Camellia-hz]

@DevLinyan

[Camellia-hz]

@ChonghaoSima

[DevLinyan]

The files test_eval.json and test_llama.json contain the same data but in different formats. The evaluation can only be conducted using the specific format in test_eval.json.

[Camellia-hz]

Thanks for your reply, if so is the evaluation valid? Because I am using test_llama.json to train my model, if I then use test_eval.json to evaluate it, what about the training set and validation set are the same?

[Camellia-hz]

@DevLinyan

[ChonghaoSima]

Not sure what you mean by "the training set and validation set are the same".

The evaluationo is valid as long as you use our provided test file and submit to our official test server.