Dynamically compute scenarios to run

[cbeauchesne]

Motivation

Changes

1. compute scenarios to run based on modifications on manifests
   • get the map of test_method -> attached scenario
   • get the list of modified node in manifest
   • on each modified node, get attached scenarios to run
2. compute scenarios to run based on modifications in tests/ folder
   • get the list of modified file
   • with the map above, get the attached scenarios to run
   • ⚠️ it may run more scenario than necessary, but it's better than running all of them
3. modify the CI API
   • rather having a run-scenario-group inputs (lot of them), uses two inputs:
   • scenarios: a list of scenario to run
   • scenarios_groups: a list of scenario groups to run
   • define scenarios_groups a the scneario level
   • and add script that build a list of scenario based on scenarios_groups

Last note, scenarios and scenarios_groups are strings, in JSON format, like '["AAA", "BBB"]'. I choose this foirmat, rather a simple comma separated list, to be able to easily use them as matrix inputs in a near future.

Edit: Actually use a comma separated list : those inputs needs to be computed by a script, so, technically JSON and a comma-separated list are equally simple. But the second is easier to use for system-tests.yml API.

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Relevant label (run-parametric-scenario, run-profiling-scenario...) are presents
• If PR title starts with [<language>], double-check that only <language> is impacted by the change
• No system-tests internal is modified. Otherwise, I have the approval from R&P team
• CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added (or removed)?
  • Get a review from R&P team

[robertomonteromiguel]

I added some comments/questions.
I think this PR is good for our CI performance, but I also think we are adding a lot of complexity to our pipeline.

[cbeauchesne]

Last note, scenarios and scenarios_groups are strings, in JSON format, like '["AAA", "BBB"]'. I choose this foirmat, rather a simple comma separated list, to be able to easily use them as matrix inputs in a near future.

Edit: Actually use a comma separated list : those inputs needs to be computed by a script, so, technically JSON and a comma-separated list are equally simple. But the second is easier to use for system-tests.yml API.