roachtest: allow running roachtests on AWS

[petermattis]

Add a --cloud flag (default "gce") which allows running roachtests on
"aws". Refactor some of the GCE specific code to also support AWS.

Fixes #29334

Release note: None

[cockroach-teamcity]

This change is 

[cockroach-teamcity]

This change is 

[petermattis]

This is still a WIP as roachprod fails when trying to create a cluster on i3 machines as it is using the Google device names. We'll also need to add another teamcity config for running some subset of the roachtests on AWS.

[petermattis]

This is still a WIP as roachprod fails when trying to create a cluster on i3 machines as it is using the Google device names. We'll also need to add another teamcity config for running some subset of the roachtests on AWS.

[petermattis]

Oh, I think roachprod is ok. The problem is RemountNoBarrier.

[petermattis]

Oh, I think roachprod is ok. The problem is RemountNoBarrier.

[petermattis]

Fixed up RemountNoBarrier and switched to specifying --aws-machine-type-ssd instead of --aws-machine-type. Everything seems to be working in simple testing. For example, roachtest run --cloud=aws kv95/encrypt=false/nodes=1.

[petermattis]

Fixed up RemountNoBarrier and switched to specifying --aws-machine-type-ssd instead of --aws-machine-type. Everything seems to be working in simple testing. For example, roachtest run --cloud=aws kv95/encrypt=false/nodes=1.

[andreimatei]

LGTM

Reviewable status: complete! 0 of 0 LGTMs obtained

---

pkg/cmd/roachtest/cluster.go, line 393 at r1 (raw file):

	fmt.Fprintf(os.Stderr, "unknown machine type: %s\n", s)
	os.Exit(1)

here and elsewhere: log.Fatalf()? Is there a reason there's a lot of os.Exit(1) in roachtest?

---

pkg/cmd/roachtest/main.go, line 152 at r1 (raw file):

			&artifacts, "artifacts", "artifacts", "path to artifacts directory")
		cmd.Flags().StringVar(
			&cloud, "cloud", cloud, "cloud provider to use (local aws gce)")

what's "local"? How does that, and this flag in general, interact with "--local"? Should that flag go away?

[petermattis]

Reviewable status: complete! 0 of 0 LGTMs obtained

---

pkg/cmd/roachtest/cluster.go, line 393 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

here and elsewhere: log.Fatalf()? Is there a reason there's a lot of os.Exit(1) in roachtest?

I can't recall offhand. I used fmt.Fprintf and os.Exit for consistency with other roachtest code. I'd like to leave this as-is for this PR. Feel free to determine if there is a good reason for this (i.e. spelunk git blame) and switch to using log.Fatal if there isn't. Note that using util/log isn't a good idea for command line tools as we don't want a stacktrace.

---

pkg/cmd/roachtest/main.go, line 152 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

what's "local"? How does that, and this flag in general, interact with "--local"? Should that flag go away?

I stole part of the flag description from roachprod. Specifying --cloud=local would not have worked. Perhaps we can make that work, though I think I'd want to keep --local as a synonym for --cloud=local. You're doing a bunch of work in this area right now and if I muck with it concurrently we'll likely just have merge conflicts. Feel free to sanitize this in the rest of your efforts, or leave it and I'll fix it up once you're done.

I've removed local from the flag description.

[petermattis]

bors r=andreimatei

[petermattis]

bors r=andreimatei

[craig]

Build succeeded

• GitHub CI (Cockroach)

[craig]

Build succeeded

• GitHub CI (Cockroach)