Re-integrate release branches into main branch

[michaelsauter]

In the finalize stage (assembly), we merge back into the main branch.
That ensures master has all changes of the release branch, ensuring that
newer releases do not accidentally loose changes done on earlier release
branches. However, if two concurrent releases are being worked on, then
it can still happen that the newer release branch does not contain
changes done in the older release branch. This is mitigated by a check
at the pipeline start (init stage). Every promotion to Q or P checks if
the commits that are to be deployed are descendants of commits already
deployed to Q and P. In case of Q, this might be OK (concurrent releases
fighting over Q are allowed), so the build is marked only as unstable.
In case of P, the build fails.

To enable the functionality described above, there is now a directory
"ods-state" in the release manager repository, which, on the master
branch, contains up-to-date information which version, change and
repository commits are deployed in which environment at the moment.

Apart from those changes, there are quite a few other improvements / bug
fixes which were identified in the process:

• Remove repo.data.odsBuildArtifact in favour of repo.data.openshift
• Only run resurrection logic in assemble mode
• Change resurrection detection to look at latest deployed commit in environment
  instead of looking for diff since last deployment descriptor commit.
  It also does not check for deployment version anymore. The better
  check would be to compare the exports, but that proved to be difficult
  with the agent setup we have right now. In any case, the export is
  always happening in the finalize stage, even when the component was
  resurrected so skipping this check seems fine to me.
• Change format of deployment descriptor (new top-level “deployments” key)
• Extracted logic of deployment descriptor in its own class
• Extract logic of deploying/finalizing ODS components in its own class
• Move more Git tasks into GitService
• Move hardcoded name of image puller secret to project to make it easier in the
  future to cusomtize this
• Speed up retrieval of pod data (previously it needed at least two API
  calls, now just one)
• Push all commits together in the finalize stage - this helps to
  prevent inconsistent tags between repositories. However, it still
  isn't an atomic transaction. To prevent pipelines from getting stuck
  (as the tag of a repository is not writable because it already
  exists), the Git tag used for promotion now includes the Jenkins build
  number of the assembling job.
• Introduce new pipeline parameter 'rePromote'. Using this flag it is
  possible to promote a tag to Q again that was previously already
  deployed. Re-deployment to P is never possible.

Closes #367.

[metmajer]

@michaelsauter I'll need a few days to dig through. Quick comment: some teams use "dev" as an integration branch. Will need to check that such custom flows remain supported (will be if you're using the branches configured in metadata.yml).

[michaelsauter]

@metmajer I know this is a lot to digest. Sorry for this. One reason that the diff is huge is that I moved some code out of MROPipelineUtil to DeployOdsComponent and FinalizeOdsComponent. To your question: I use the branches configured in metadata (repo.branch). Only for the release manager repository, master is hardcoded (as it is already hardcoded in the pipeline config) ... potentially we could read that from the branch of the pipeline run, but I am not sure we really want this ... so I kept it simple with master being the main branch of the RM repo.

[metmajer]

@michaelsauter makes sense, and the component branches were what I was thinking of only. Thanks for the quick update, I'll try my best to have a look by the end of the week.

[martsec]

lgtm. Great job!