System and Platform Objects

[douglasjacobsen]

Project Name: System and Platform Objects

Contributors: douglasjacobsen

Project Summary

The goal is this project is to implement first party objects in Ramble to
represent individual compute nodes (i.e. platforms) along with high level
clusters (i.e. systems) that are built out of platforms.

The aim is to simplify reproducibility for end users by encapsulating the
important aspects of their compute environment into these system and platform
objects.

Background

When exploring performance differences across various hardware types, users are
forced to configure their workspaces for many aspects of the underlying system.
Ramble currently supports including one or more YAML files, which can be
composed to create a set of experiments targeting a specific system. These
complex configurations can be more error prone for users to maintain, and can
be more difficult for users to reasonable (due to all of the indirection
involved).

Recent successes in the addition of package manager and workflow manager
objects has shown that we are able to encapsulate more information,
specifically information that never changes, in these portable objects and this
results in more easily re-used artifacts than straight YAML files.

Users still have to piece together the objects they want to use, but the python
objects provide a more rigid definition for users that make it easier to help
guide them along a working path.

Historically, Ramble has shied away from encoding system information. Primarily
because this changes frequently from one user to another. However, as Ramble's
user base grows within various sites it becomes more important to encode this
information in concrete artifacts that can be reused more easily (and are more
easily maintained).

This design aims to promote systems and platforms, as important components of a
successful experiment definition, into first party features that can simplify
the end-user experience on known systems.

Requirements

• Users retain the ability to describe their environment in YAML (i.e. user-managed options)
• Systems:
  • System objects should be able to specify default variants (i.e. package manager, workflow manager)
  • System objects should be able to restrict the possible values for platforms (i.e. only an available platform can be selected)
  • System objects should be able to define a mapping of platform to variable values (i.e. platform => partition name)
  • System objects should be able to provide an experiment with auxiliary software files (i.e. spack configuration files)
• Platforms:
  • Platform objects should define expected attributes of a compute node (i.e. memory_per_node, cores_per_node, accelerator_type, accelerators_per_node)

Proposal

New Object Types

Within Ramble's repository, we will add default object types for system and
platform objects. Additionally, we will add base_system and base_platform
objects to remain consistent with other objects. The base classes can then be
used to construct inheritance chains, simplifying development within some
ecosystem (i.e. all GCP platforms inherit from a gcp-platform base platform
class.

Ramble object repositories will be allowed to have directories matching these
object types, similar to other object types.

Object Precedence

An important aspect of constructing an experiment is the precedence of the
object definitions. As mentioned in the requirements, system objects will be
allowed to specify default values for variants including the package manager
and workflow manager. Users are expected to be able to override whatever these
defaults are, but providing defaults helps simplify the user experience.

To help ensure this works properly, objects will adhere to the following
precedence order when setting variant and variable values:

1. YAML Definition (Highest Precedence)
2. Modifier Objects
3. System Objects
4. Platform Objects
5. Workflow Manager Objects
6. Package Manager Objects
7. Application Objects (Lowest Precedence)

This precedence ordering will allow system and platform objects to define
default workflow manager and package manager objects before their definitions
are applied to the experiment.

Modifier objects are not expected to manipulate the system, platform, workflow
manager, or package manager object variants. However, to ensure they are able
to change variables they have a higher precendence order.

User Controls

The user will select the system and platform classes through variant values.
For example:

ramble:
    variants:
        system: our-slurm-cluster
        platform: c2

These variants can be parameterized the same way as other variants, through
variable indirection. For example:

ramble:
    variants:
        system: our-slurm-cluster
        platform: '{experiment_platform}'
    variables:
        experiment_platform: [c2, c2d]

Can be used to sweep multiple platforms within a given system. Systems can be
swept similarly.

Required Variables

Within the new base classes, several variables will be required to be defined.
These are primarily to help ease the end-user experience, but also to
standardize the definitions somewhat. Required variables will be defined using
the required_variable directive in the shared language.

Within the system class, the platform_nodes variable will be required to be
defined. Through directives defined later, platforms within a system can have
different default values, assuming the developer wants to provide a default
value. If a value is not provided within the system class, end-users will be
required to define one in their workspace.

Within the platform class, the following variables will be marked as required:

• cores_per_node
• memory_per_node

Base platform classes can be created to augment this with additional
information. An accelerated-node base platform will be created that will also
define a variant named accelerator_type, and then a variable
accelerators_per_node will be marked as required. This can be used to
standardize the number of accelerators (i.e. GPUs) within a node.

Specific system and platform implementations can add additional requirements as
needed to ensure they function properly together.

Standard validation

The base system and platform classes will have variants to control validation.
These will be named validate_system and validate_platform and will have a
default value of True. Users can turn these off to use configurations that
would not validate normally.

Some default validators will be registered into these base classes, such as:

• {n_nodes} <= {platform_nodes}
• {n_threads} * {processes_per_node} <= {cores_per_node}
• {n_ranks} <= {cores_per_node} * {n_nodes}

This can help end users avoid configuration issues that can negatively impact
their experiment's performance.

Directives Changes

In support of these new classes, some modifications to directives are required.
Each object type will get its own language, similar to how other objects
behave in Ramble.

Platform Object Language

The platform object language will be the simplest. It will primarily be the
shared language, with the addition of platform_family variant directives.

System Object Language

The system object language will build off of the shared language, and add the
following new directives.

Default Workflow Manager

Sets the default for variants:workflow_manager in this experiment

default_workflow_manager(str: workflow_manager_name)

Example:

default_workflow_manager("slurm")

Makes experiments using this system default to using slurm.

Default Package Manager

Sets the default for variants:package_manager in this experiment

default_package_manager(str: package_manager_name)

Example:

default_package_manager("spack")

Makes experiments using this system default to using spack.

Default Platform

Sets the default for variants:platform in this experiment

default_platform(str: platform_name)

Example:

default_platform("c2")

Makes experiments using this system default to using c2.

Platform Controls

Defines the platforms that are available in this system. Will validate that the
selection is one of these.

available_platforms([list, of, platforms, supported])

Defines a mapping of platform to value, for a given variable name. Can be used
multiple times to set several variables for each platform.

platform_variable_map(
    str: variable_name,
    dict[str, str]: var_map = {
        "platform1_name": "platform1_value",
        "platform2_name": "platform2_value",
        "platform3_name": "platform3_value",
    }
)

Example:

platform_variable_map(
    "slurm_partition",
    var_map = {
        "c2": "c2standard",
        "c2d": "c2dstandard"
    }
)

Causes the value for slurm_partition to be c2standard or c2dstandard
based on the platform selected.

Defines a mapping of variable name to value for a specific platform.

variable_defaults(
    dict[str, str]: variable_definitions = {
        "variable1_name": "variable1_val",
        "variable2_name": "variable2_val",
        "variable3_name": "variable3_val",
    }
)

Example:

with when("platform=c2"):
    variable_defaults(
        variable_definitions = {
            "slurm_partition": "c2standard",
        }
    )


with when("platform=c2d"):
    variable_defaults(
        variable_definitions = {
            "slurm_partition": "c2dstandard",
        }
    )

Allows for grouping variable definitions based on platform rather than by
variable name.

Auxiliary Software Files

Behaves very similar to register_template however we render these within the
package manager so they are properly injected into the software environment.

auxiliary_software_file(
    str: name,
    str: src_path,
    str: dest_path,
)

Example:

auxiliary_software_file(
    "packages",
    src_path="packages-{platform_name}.yaml.tpl",
    dest_path="{env_path}/packages.yaml"
)

Will render the local (to the system.py file)
packages-{platform_name}.yaml.tpl file into the software environment, to help
configure the environment for the given platform.

Package managers are free to implement how auxiliary files should be combined.

Additional Functionality

Other aspects, like validating configurations and defining default variables,
will be handled through the shared language of objects. It is also expected
that existing features (like when and versioning) will work properly within
the system and platform objects.

[douglasjacobsen]

PR for this is: #1541