Validation script missing?

[kupcsika]

Describe the issue

Hi,

I'd like to create my own dataset for benchmarking. I'm looking at the documentation about the collection pipeline here. I managed to create my own dataset (the jsonl files) and as I understand the next step would be the execution based validation. However, I don't see the harness/run_validation.py or the validation_engine.py. Were these removed on purpose, or moved somewhere else?

Cheers!

Suggest an improvement to documentation

No response

[john-b-yang]

Please see the note here. We're unfortunately not actively supporting task validation efforts right now, but we have some forthcoming work that should resolve this. Please stay tuned!