Allow setting a ExecutionClusterLabel when triggering a Launchplan/Workflow/Task

[RRap0so]

Why are the changes needed?

This is an alternative implementation to the #4956. Instead of trying to implement the clusterPool functionality we reuse the ClusterAssignment struct and Execution Label.

This was changes are kept to a minimal and would cover the needs of the linked RFC

Closes #5081

What changes were proposed in this pull request?

How was this patch tested?

This was tested with two additional unit tests.

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[codecov]

Codecov Report

Attention: Patch coverage is 89.83051% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 59.00%. Comparing base (feb4777) to head (72d52ca).

Files	Patch %	Lines
...g/executioncluster/impl/random_cluster_selector.go	85.00%	2 Missing and 1 partial ⚠️
flyteadmin/pkg/manager/impl/execution_manager.go	90.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4998      +/-   ##
==========================================
+ Coverage   58.98%   59.00%   +0.02%     
==========================================
  Files         645      645              
  Lines       55648    55670      +22     
==========================================
+ Hits        32824    32850      +26     
+ Misses      20229    20224       -5     
- Partials     2595     2596       +1     

Flag	Coverage Δ
unittests	59.00% <89.83%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[davidmirror-ops]

cc @gdabisias

[wild-endeavor]

Won't we also need to add the message to the incoming request? It'll need to start from there no?

[wild-endeavor]

hey @RRap0so do you have a real multi-cluster setup available to help test this? It's a fair bit of work for us to set one up to test. We can fake it, but it'd be nice to have a real setup to test against. I pulled this pr, built a new sandbox image for it, and ran a local/normal test, and it works, but i'm obviously not testing any of the new functionality.

[RRap0so]

@wild-endeavor I think we might have something that we can use. I'll report back.

[RRap0so]

Hi @wild-endeavor.

All done and tested. I've created executions via grpc and manage to reproduce everything perfectly:

• if a label is found, goes through the proper cluster
• if not found then default behaviours applies (in our case the execution would randomly hit one of the two clusters we have)

I've also added a Debug line to help out and done another comparison fix.

[katrogan]

thank you for testing this out end to end!

do you mind also updating https://github.com/flyteorg/flyte/blob/master/flyteadmin/pkg/executioncluster/impl/in_cluster.go#L25 to fail fast when a ExecutionTargetSpec.ExecutionClusterLabel is specified (like we do for TargetID)?