Why engine_validation.py and ./harness/run_validation.sh files are misssing

[skj092]

Describe the issue

I am trying to reproduce the dataset but seems like these file are missing

Suggest an improvement to documentation

No response

[zlpure]

I also encountered this problem. The ./harness/run_validation.sh file is missing in the main branch.

[john-b-yang]

Hi @skj092 thanks for this issue. We are actively working on some ongoing efforts that will address this + standardize the process. The creation process is complicated, and writing a tutorial would take quite some effort.

Therefore, to focus our efforts, we are temporarily pausing actively supporting queries around collection. See the tutorial for more details.

If you're intent on reproducing the dataset, the tutorial mentions the last commit at which engine_validation.py was made available in the repository.

I'm closing this issue for now. Thanks again for the query, and we will have a resolution for such questions in the coming weeks.