Add GitHub tagging and release creation automation using GitHub and Azure Pipelines APIs

[dagood]

I recommend reviewing with "hide whitespace" enabled. I changed the subcommand implementation to use an interface rather than have a func in a struct, so a lot of code is unindented by one level.

---

• For Create a pipeline for microsoft/go releases go#423
• Also implements Include build asset JSON in GitHub releases for easy access to all artifacts go#463

Finish off post-build release process automation for microsoft/go. This gets rid of a lot of copy-pasting version numbers so the process is less tedious and error-prone.

Here's an example of the comments that the pipeline (optionally) leaves on the release issue: https://github.com/dagood/go-release-test/issues/3.
Example release/tag: https://github.com/dagood/go-release-test/releases/tag/v1.18.1-1

Pipeline run UI looks like this:

I plan to have the variable group pass through all the release steps to make it reasonable to run a fake release without worrying too much about messing up the main repos.

Individual steps are toggleable so if a build partially fails, you can run a retry that only includes the steps that still need to happen. If the steps were each idempotent, these toggles wouldn't be needed, but I didn't want to try to rig up reliable idempotency on top of the GitHub APIs--at least not yet. I think keeping it simple will work out fine.

This adds a few libraries:

• https://github.com/google/go-github - API to create the tag, GitHub release, and comment on the issue.
  • https://golang.org/x/oauth2 - For go-github auth.
• https://github.com/microsoft/azure-devops-go-api - Get more details about the build being released, like commit to tag.

I didn't switch the existing code over to use go-github yet. Note: go-github uses the GitHub v3 API, so it actually can't do auto-merges, which requires v4. But the GitHub v4 API is incomplete (it doesn't support releases), so we can't simply use a newer library that supports v4.

---

github.com/microsoft/go-infra/subcmd

This package makes it easier to implement subcommands. I moved the subcommand functionality from git-go-patch over here and changed it to use interfaces, which are more typical for this kind of sharing.

I changed the existing update-aka-ms command into the releasego akams subcommand, which meant some tweaks like adding the ability to handle arbitrary args by implementing OptionArgTaker instead of only Option.

I wanted to use subcommands for releasego to give the commands a clear namespace (so subcommand names don't have to be as descriptive and can trivially share code) and so we can build a single app that gets used throughout the steps of the pipeline for simplicity.

[qmuntal]

Did an initial review and everything looks great!

I didn't find anything to comment, so I will let it sink in and tomorrow I'll give it another shot 😄

[jaredpar]

The extract, applyandrebase` commands have the following pattern:

        rootDir, err := findOuterRepoRoot()
	if err != nil {
		return err
	}

	goDir := filepath.Join(rootDir, "go")

Is it worth factoring that out?

[dagood]

I can see rootDir, goDir, err := findProjectRoots() working out.

I'd probably want to rename to projectDir, submoduleDir, err := findProjectRoots() to be more forward-looking, because I think we'll be generalizing the patching tools beyond our forks. The findProjectRoots function will then let us migrate off the hard-coded go subdir in one place (finding/reading a config file instead).

These lines are only touched by this PR to change the indentation, but it would be pretty hard to mess up a refactor like this, so I might as well while it's in mind.

[jaredpar]

One thought I had reading through this is there is a lot of IO happening and when it fails we resort to

return err

I'm considering how that will look when we have an error in an AzDO run. It will result in a failure that has an IO message in the logs but it seems like it will be really hard to track that back down to what failed. Basically if we have a "Access Denied" error in the logs how will we tell which of the various IO paths in this code was responsible?

Is it worth adding logs on the failure cases so we have a trail to look at?

[dagood]

In a lot of cases I think we can pretty easily figure out where an error happened by looking at the last positive/progress log line. But, I try to always remember to wrap errors when I think there might be some ambiguity, like return fmt.Errorf("unable to chop the carrot: %w", err) when err could be knife is dull but the func is making an entire bowl of soup.

In my experience, IO errors always mention the filename, so there are a decent number of situations where even without wrapping, the error would still be obvious.

This seems pretty subjective to me, and wrapping too much is definitely better than not wrapping enough, so I'd be happy to put more in. (Just maybe not in git-go-patch because this isn't new code, only unindented.)

[chsienki]

If we fail to upload a file, we should fail the release. Does that happen? Possibly it does, just my lack of go knowledge showing :)

[chsienki]

Ah, we're outside of the retry so this will return from Handle which will kick in the defered cleanup?

[dagood]

Yep, err := retry(...) gets the last inner error if it exceeds the retry count, then if err := retry(...); err != nil { continues here, and this return is in the function's scope.

[chsienki]

LGTM!

[ghost]

All CLA requirements met.