Skip to content

Create dynamic Gitlab pipelines from ReFrame tests #1523

@vkarak

Description

@vkarak

The idea is very similar to spack ci generate. ReFrame will generate a Gitlab pipeline dynamically that would run in each stage different set of tests as pipeline jobs. For the moment, we don't treat dependencies a proper test restore mechanism is need, which will be addressed in #1222. Here is the basic idea:

stages: [generate, build]

generate-pipeline: 
  stage: generate
  tags: 
    - <custom-tag>
  script: 
    - reframe -c /path/to/repo/checks --ci-generate="${CI_PROJECT_DIR}/jobs_scratch_dir/pipeline.yml" --ci-tags=unittest,verification,production
  artifacts: 
    paths: 
      - "${CI_PROJECT_DIR}/jobs_scratch_dir/pipeline.yml"

test-jobs: 
  stage: build
  trigger: 
    include: 
      - artifact: "jobs_scratch_dir/pipeline.yml"
        job: generate-pipeline
    strategy: depend

The pipeline.yaml would then define a Gitlab pipeline with different stages, one per CI tag (in this example, unittest, verification and production) and each pipeline job will execute a single ReFrame test for that set of tags: reframe -c /path/to/repo/checks -t unittest -n test_0, reframe -c /path/to/repo/checks -t unittest -n test_1 etc.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions