-
Notifications
You must be signed in to change notification settings - Fork 69
Description
What would you like to be added:
Config file auto-generation feature.
Why is this needed:
Creating a configuration file from scratch is overly complicated. Users must manually select valid benchmarks and configure the appropriate parameters for each test, which increases the likelihood of errors (e.g., misspellings).
Without this feature, how does current superbenchmark work:
Users need to manually write superbench
config YAML file (and/or rule files) to run benchmarks, diagnosis, and result summary.
Components that may involve changes:
cli
Brief description of your proposal if any:
Can we have a feature similar to nhc
's config file auto-generation, which can automatically generate an initial config file based on the node info scan?
It can scan the basic node info and generate an initial sample config by:
- Selecting all the valid benchmarks for the node.
- Setting parameters for each benchmark test (e.g., set
num_proc
based on the number of GPUs).
And then the user can easily modify the config for some customization (e.g., remove some unnecessary benchmarks and adjust parameters).
It can be a new subcommand (e.g., sb config generate
) in the CLI, or just some utility scripts.