Skip to content

Conversation

sleaux
Copy link
Collaborator

@sleaux sleaux commented Jun 9, 2025

This adds support for generator tasks (excluding burn-in) that run resmoke suites via bazel.

Generating the subtask suite configs depends on the original suite configs having already been created like bazel build --build_tag_filters=resmoke_config //... , and their locations passed in via the new option --bazel-suite-configs resmoke_suite_configs.yml.

In the mongo Evergreen YAML, generator tasks use the same functions, but with the bazel target as the suite parameter. Some examples (some details omitted for brevity):

  - <<: *gen_task_template
    name: aggregation_secondary_reads_gen
    commands:
      - func: "generate resmoke tasks"
        vars:
          suite: //buildscripts/resmokeconfig:aggregation_secondary_reads
          bazel_args: >-
              --test_arg=--sanityCheck

  - <<: *gen_task_template
    name: multiversion_sanity_check_gen
    tags:
      [
        "multiversion",
      ]
    commands:
      - func: "generate resmoke tasks"
      - func: "initialize multiversion tasks"
        vars:
          //buildscripts/resmokeconfig:multiversion_sanity_check_last_continuous_new_new_old: last_continuous
          //buildscripts/resmokeconfig:multiversion_sanity_check_last_lts_new_new_old: last_lts

The generated tasks end up looking like:

    {
      "name": "aggregation_secondary_reads_0-linux-enterprise",
      "commands": [
        {
          "func": "do setup"
        },
        {
          "func": "configure evergreen api credentials"
        },
        {
          "func": "run generated tests via bazel",
          "vars": {
            "bazel_args": "--test_arg=--suites=generated_resmoke_config/aggregation_secondary_reads_0-linux-enterprise.yml --test_arg=--sanityCheck",
            "suite": "generated_resmoke_config/aggregation_secondary_reads_0-linux-enterprise.yml",
            "gen_task_config_location": "mongodb-mongo-master/d02a958ad6c1a4a303cb2ac1640d64321ead1330/generate_tasks/generated-config-683a1600942b9f0007980662.tgz",
            "require_multiversion_setup": false,
            "targets": "//buildscripts/resmokeconfig:aggregation_secondary_reads",
            "compiling_for_test": true
          }
        }
      ],
      "depends_on": [
        {
          "name": "archive_dist_test"
        }
      ]
    },

    {
      "name": "multiversion_sanity_check_last_continuous_new_new_old_0-linux-enterprise",
      "commands": [
        {
          "func": "git get project no modules"
        },
        {
          "func": "add git tag"
        },
        {
          "func": "do setup"
        },
        {
          "func": "configure evergreen api credentials"
        },
        {
          "func": "do multiversion setup"
        },
        {
          "func": "run generated tests via bazel",
          "vars": {
            "gen_task_config_location": "mongodb-mongo-master/da3394e46073d46dc62f438d0db276141d7a2074/generate_tasks/generated-config-68474760170ba6000713e14c.tgz",
            "suite": "generated_resmoke_config/multiversion_sanity_check_last_continuous_new_new_old_0-linux-enterprise.yml",
            "require_multiversion_setup": true,
            "bazel_args": "--test_arg=--tagFile=generated_resmoke_config/multiversion_exclude_tags.yml --test_arg=--excludeWithAnyTags=multiversion_incompatible,backport_required_multiversion,bazel_multiversion_sanity_check_backport_required_multiversion,requires_fcv_82 --test_arg=--suites=generated_resmoke_config/multiversion_sanity_check_last_continuous_new_new_old_0-linux-enterprise.yml",
            "targets": "//buildscripts/resmokeconfig:multiversion_sanity_check_last_continuous_new_new_old",
            "multiversion_exclude_tags_version": "last_continuous",
            "compiling_for_test": true
          }
        }
      ],
      "depends_on": [
        {
          "name": "archive_dist_test"
        },
        {
          "name": "select_multiversion_binaries"
        }
      ]
    }

@sleaux
Copy link
Collaborator Author

sleaux commented Jun 9, 2025

Relies on changes in https://github.com/10gen/mongo/compare/master...sleaux/SERVER-102655, which introduces the Evergreen functions and task definitions referenced here.

Required set + all bazel resmoke tasks are run in a mongodb-mongo-master patch build.

@sleaux sleaux marked this pull request as ready for review June 9, 2025 22:11
@sleaux sleaux requested a review from Trevor159 June 9, 2025 22:12
@Trevor159
Copy link
Collaborator

Do you have a patch build from mongodb-mongo-master that uses this?

@sleaux
Copy link
Collaborator Author

sleaux commented Jun 10, 2025

Do you have a patch build from mongodb-mongo-master that uses this?

Shared outside of this PR, since this repo is public.

src/main.rs Outdated
&evg_expansions.project,
gen_sub_tasks_config_file.as_ref(),
);

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] random newline

@sleaux sleaux changed the title Generate tasks for suites that run via bazel DEVPROD-18259 Generate tasks for suites that run via bazel Jun 10, 2025
@sleaux sleaux enabled auto-merge June 10, 2025 16:14
@sleaux sleaux added this pull request to the merge queue Jun 10, 2025
Merged via the queue into master with commit 6a69fc5 Jun 10, 2025
2 checks passed
@sleaux sleaux deleted the sleaux/besmoke branch June 10, 2025 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants