initial config for partial conv in JET of esm2 in bionemo2

[dorotat-nv]

Adding the first config of partial convergence training in JET for ESM2 model in BioNeMo 2

The JET pipeline can be submitted for this config by running

python jet/cli.py submit --root-dir ci/benchmarks/partial-conv/ --manifest-template-config-file <PATH_TO_CI_REPO>/jet/manifest_template_config.yaml

where PATH_TO_CI_REPO is an absolute path to the CI repository storing JET CLI tools: https://gitlab-master.nvidia.com/clara-discovery/bionemo-github-ci/-/blob/master/jet/cli.py?ref_type=heads#L78

The PR it is still a draft. I need to adjust number of steps (due to 4h time limit and reducing nodes from 32 to 4). Also, I need to add test section.

In order to make the management of configs more robust, I will add a unit test in BioNeMo2 which checks if the training command can be executed.

TODO in the future:

1. Run trainings on different random seeds
2. Use full convergence max steps in lr scheduler: Add support on num steps for learning rate scheduler #489

[skothenhill-nv]

Did we decide to do this with the CLI/argparse entrypoints? This is doable with the pydantic interface as well.

[sichu2023]

LGTM.

[dorotat-nv]

/build-ci