v2.0-rc1

As the existing benchmarks in DCPerf v2 are mostly ready to use, we're publishing this v2.0-rc1 release ahead of the formal DCPerf v2 release to provide a stable version handle for users who would like to have more reproducibility. Now, the ./benchpress_cli.py --version command and benchmark result JSON reports will have v2.0-rc1 as the version. We will release newer RC versions if we land more fine-tuning changes to the benchmarks, and the final v2.0 when everything is done.

DCPerf v2.0 intends to support CentOS 9/10 and Ubuntu 22.04/24.04. We plan to add Docker option in the future.

Currently the following benchmarks are ready to use:

Main Benchmarks:

• TaoBench v2
  • TaoBench v2 job: tao_bench_autoscale_v2_beta
  • The v1 job tao_bench_autoscale and tao_bench_standalone is preserved.
  • New feature: Auto-warmup and support for memory file to reduce execution time
• FeedSim v2
  • FeedSim v2 job: feedsim_dlrm
  • Software architecture introduction: ARCHITECTURE_v2.md
  • NOTE: the legacy feedsim_autoscale job has been removed; if you would like to run FeedSim v1, please check out the main branch (or v1 branch in the future) and install & run from there.
• DjangoBench v2
  • DjangoBench v2 job: django_workload_default. No separate job for ARM.
  • Software architecture introduction: here
  • NOTE: the same django_workload_default job will run DjangoBench v2, please checkout the main or v1 branch if you wish to run DjangoBench v1
• Mediawiki
  • The workload is unchanged. The main update to Mediawiki is a scalability fix for CPUs of >=200 logical cores.
• SparkBench v2
  • Changed Java runtime from OpenJDK 8 to GraalVM
  • SparkBench v2 job: spark_standalone_remote_3x
  • The original v1 job spark_standalone_remote job is preserved.
• VideoTranscodeBench
  • The workload is unchanged. The main update is SVT-AV1 library version upgrade to expose newer optimizations especially for ARM CPUs.

Micro-benchmarks:

• Datacenter Tax and WDL
  • WDLBench v2
• Graph and Tiered Memory Benchmarking
  • GABPS Bench
• Kernel & HW Performance
  • Schbench
  • Syscall
  • HealthCheck

AI Micro-benchmarks:

• FBGEMM Embedding and Matmul
• Rebatching
• Tensor Deserialization
• Concurrent Hashmap
• Pytorch GEMM Dispatch