Skip to content

Config file auto-generation #707

@fno2010

Description

@fno2010

What would you like to be added:

Config file auto-generation feature.

Why is this needed:

Creating a configuration file from scratch is overly complicated. Users must manually select valid benchmarks and configure the appropriate parameters for each test, which increases the likelihood of errors (e.g., misspellings).

Without this feature, how does current superbenchmark work

Users need to manually write superbench config YAML file (and/or rule files) to run benchmarks, diagnosis, and result summary.

Components that may involve changes:

cli

Brief description of your proposal if any:

Can we have a feature similar to nhc's config file auto-generation, which can automatically generate an initial config file based on the node info scan?

It can scan the basic node info and generate an initial sample config by:

  1. Selecting all the valid benchmarks for the node.
  2. Setting parameters for each benchmark test (e.g., set num_proc based on the number of GPUs).

And then the user can easily modify the config for some customization (e.g., remove some unnecessary benchmarks and adjust parameters).

It can be a new subcommand (e.g., sb config generate) in the CLI, or just some utility scripts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions