roachtest: network/authentication/nodes=4 failed

[cockroach-teamcity]

roachtest.network/authentication/nodes=4 failed with artifacts on release-23.1.21-rc @ 3b9077db19465ffe6e4aabc6a0cd05fc7073be3e:

(assertions.go:262).Fail: 
	Error Trace:	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/network.go:104
	            				github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/network.go:306
	            				main/pkg/cmd/roachtest/test_runner.go:1114
	            				GOROOT/src/runtime/asm_amd64.s:1594
	Error:      	Received unexpected error:
	            	pq: replica unavailable: (n1,s1):1 unable to serve request to r5:/{Systemtse-Table/0} [(n1,s1):1, (n3,s3):2, next=3, gen=2]: lost quorum (down: (n3,s3):2); closed timestamp: 1714370546.270632308,0 (2024-04-29 06:02:26); raft status: {"id":"1","term":6,"vote":"1","commit":17,"lead":"0","raftState":"StatePreCandidate","applied":17,"progress":{},"leadtransferee":"0"}: have been waiting 63.00s for slow proposal RequestLease [/System/"tse",/Min)
	Test:       	network/authentication/nodes=4
(require.go:1264).NoError: FailNow called
test artifacts and logs in: /artifacts/network/authentication/nodes=4/run_1

Parameters:

• ROACHTEST_arch=amd64
• ROACHTEST_cloud=gce
• ROACHTEST_cpu=4
• ROACHTEST_encrypted=false
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

• roachtest: network/authentication/nodes=4 failed #120468 roachtest: network/authentication/nodes=4 failed [A-testing C-bug C-test-failure O-roachtest O-robot P-3 T-kv branch-master]

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-38236

[miraradeva]

Duplicate of #120468. Assigning the same labels.

Nodes 1 and 2 get their circuit breakers tripped at 06:02:32 and 06:08:34 respectively because they are unable to look up descriptor for n3. But n3 only starts up at 06:08:35.