We have the definition (in the Rules.md file) of the directory hierarchy that the benchmark will produce for Training and Checkpointing runs, but we still need the definition for VDB and KVCache. Those definitions are (primarily) used by the submission validation checker code that Pablo is writing.
Note that the directory hierarchy is one piece of what's needed, but it should be the first and easiest thing to document. The other things include defining any fields in yaml/json files that the validation checker should look at. Eg: Russ should add "data_access_api: [file|s3]" to one of the json files that DLIO generates so that we know that this run used S3 rather than POSIX.
Defining the yaml/json fields to validate is part of developing the Rules for your workload.
We have the definition (in the Rules.md file) of the directory hierarchy that the benchmark will produce for Training and Checkpointing runs, but we still need the definition for VDB and KVCache. Those definitions are (primarily) used by the submission validation checker code that Pablo is writing.
Note that the directory hierarchy is one piece of what's needed, but it should be the first and easiest thing to document. The other things include defining any fields in yaml/json files that the validation checker should look at. Eg: Russ should add "data_access_api: [file|s3]" to one of the json files that DLIO generates so that we know that this run used S3 rather than POSIX.
Defining the yaml/json fields to validate is part of developing the Rules for your workload.