Open up test host orchestrator for public consumption experimentally

[Youssef1313]

Summary

Currently, test host orchestration is an internal feature only. It's used by Retry today.

Background and Motivation

I see a lot of potential in exposing test host orchestrator to be publicly usable, for third-party extensions.

For example:

• A third-party extension could, if it needs to, distribute test runs across multiple processes, or even multiple machines (similar to Arcade's Helix - maybe this could even be implemented by Arcade itself in future).

• A third-party extension could implement a retry feature, similar to ours, but with more features suited for very specific needs that we might not see worth implementing into our own retry extension.

Proposed Feature

Open-up the orchestrator with experimental attribute

Alternative Designs

[thomhurst]

This sounds very good.

Imo if you want MTP to be as flexible and as powerful as possible, anything internal that your own 'extensions' or 'filters' are using should be public, allowing users to create similar features!

[Evangelink]

Yeah I was going to say that we are probably good enough now to not rely on any IVTs and just go with public experimental.