Arcade changes are validated against itself and consumer repos

[markwilkie]

There should be a process/mechanism in place with which we can validate that changes done in Arcade won't break any consumer repos. There are several levels of validation:

1. Self validation

• Arcade's dependencies version are always the latest (Versions.props os always up to date)
• Arcade builds itself by using the latest bits in the repo (self build)
• Build which was just validated is added to the "Validation" channel

2. Sample repo validation

• Create a "arcade-validation" blob feed that is going to be used for publishing validation
• A samples repo (arcade-validation), based of arcade-minimalci-sample, is created
  • Signing samples are added covering most of the signing scenarios
  • Publishing samples publishing to the "test" blob feed
  • Packaging samples
  • Helix (testing) samples
  • Publish to BAR step is added
• Create a PR build definition where basic scenarios are validated once Maestro++ updates the versions from the "Pre-Latest" channel
• Create an official build definition to validate "complex" scenarios like real signing and publishing
• As the official build completes, the build is added to the "Latest" channel

3. Tier one repos validation (post preview 2)

• Possibly by the use of darc branch (depending on whether if it is completed or not) update the versions of a repo+branch from the "Latest" channel
• If the builds from all tier 1 repos succeed, take the build from the "Latest" channel and make it "LKG"

Basic things that Arcade repo has to have:

• Arcade build sufficiently often such that it gives an early read
• Arcade build failures are high visibility and a top priority for FR as they likely represent wider problems.
• Arcade build uses or runs tests for every important piece of infrastructure and each service we rely on.

[jcagme]

@riarenas I'm closing this Epic. I moved all the active issues to #1915