roachtest: workload fails to connect to a node - connection timed out

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/a22a3780ebb5482be04b6f31344c764b2fb5f765

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=kv0/enc=false/nodes=3/cpu=96 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1257365&tab=buildLog

The test failed on release-19.1:
	cluster.go:1423,kv.go:69,cluster.go:1761,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1257365-kv0-enc-false-nodes-3-cpu-96:4 -- ./workload run kv --init --splits=1000 --histograms=logs/stats.json --concurrency=192 --duration=10m --read-percent=0 {pgurl:1-3} returned:
		stderr:
		
		stdout:
		Error: dial tcp 10.12.41.6:26257: connect: connection timed out
		Error:  exit status 1
		: exit status 1
	cluster.go:1782,kv.go:72,kv.go:137,test.go:1245: Goexit() was called

[kenliu]

see related downstream failures reported in issues:

• all: update dependencies #30774
• roachtest: kv0/enc=false/nodes=3/cpu=96 failed #37262
• roachtest: invalid node spec 4, cluster contains 3 nodes #37179
• roachtest: kv95/enc=false/nodes=3/cpu=96 failed #37442

[github-actions]

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
5 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!