kv/kvserver/loqrecovery: TestRetrieveRangeStatus failed

[cockroach-teamcity]

kv/kvserver/loqrecovery.TestRetrieveRangeStatus failed with artifacts on release-23.2.5-rc @ e7576d10aaaa61940cebf8bd8f157ce0e1651e7b:

=== RUN   TestRetrieveRangeStatus
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/95fc0dff57a5bf3a6a9cc9666d301bb7/logTestRetrieveRangeStatus3244487076
    test_log_scope.go:81: use -show-logs to present logs inline
    server_integration_test.go:536: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/kv/kvserver/loqrecovery_test/pkg/kv/kvserver/loqrecovery/server_integration_test.go:536
        	Error:      	Not equal: 
        	            	expected: "replica unavailable: (n1,s1):1 unable to serve request to r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2, (n3,s3):3, next=4, gen=4]: lost quorum (down: (n2,s2):2,(n3,s3):3); closed timestamp: 1713711145.328480738,0 (2024-04-21 14:52:25); raft status: {\"id\":\"1\",\"term\":6,\"vote\":\"1\",\"commit\":71,\"lead\":\"0\",\"raftState\":\"StatePreCandidate\",\"applied\":71,\"progress\":{},\"leadtransferee\":\"0\"}: encountered poisoned latch /Local/RangeID/4/r/AbortSpan/\"8d26607e-a616-47d4-9adc-0b075262222c\"@0,0"
        	            	actual  : "found more failed ranges than limit 1"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-replica unavailable: (n1,s1):1 unable to serve request to r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2, (n3,s3):3, next=4, gen=4]: lost quorum (down: (n2,s2):2,(n3,s3):3); closed timestamp: 1713711145.328480738,0 (2024-04-21 14:52:25); raft status: {"id":"1","term":6,"vote":"1","commit":71,"lead":"0","raftState":"StatePreCandidate","applied":71,"progress":{},"leadtransferee":"0"}: encountered poisoned latch /Local/RangeID/4/r/AbortSpan/"8d26607e-a616-47d4-9adc-0b075262222c"@0,0
        	            	+found more failed ranges than limit 1
        	Test:       	TestRetrieveRangeStatus
    panic.go:523: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/95fc0dff57a5bf3a6a9cc9666d301bb7/logTestRetrieveRangeStatus3244487076
--- FAIL: TestRetrieveRangeStatus (111.51s)

Parameters:

• TAGS=bazel,gss,deadlock
• stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/replication

This test on roachdash | Improve this report!

Jira issue: CRDB-38296

[cockroach-teamcity]

kv/kvserver/loqrecovery.TestRetrieveRangeStatus failed with artifacts on release-23.2.5-rc @ e7576d10aaaa61940cebf8bd8f157ce0e1651e7b:

=== RUN   TestRetrieveRangeStatus
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/95fc0dff57a5bf3a6a9cc9666d301bb7/logTestRetrieveRangeStatus3529059258
    test_log_scope.go:81: use -show-logs to present logs inline
    server_integration_test.go:536: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/kv/kvserver/loqrecovery_test/pkg/kv/kvserver/loqrecovery/server_integration_test.go:536
        	Error:      	Not equal: 
        	            	expected: "replica unavailable: (n1,s1):1 unable to serve request to r4:/System{/tsd-tse} [(n1,s1):1, (n5,s5):2, (n2,s2):3, next=4, gen=4]: lost quorum (down: (n5,s5):2,(n2,s2):3); closed timestamp: 1714141378.153025780,0 (2024-04-26 14:22:58); raft status: {\"id\":\"1\",\"term\":6,\"vote\":\"1\",\"commit\":71,\"lead\":\"0\",\"raftState\":\"StatePreCandidate\",\"applied\":71,\"progress\":{},\"leadtransferee\":\"0\"}: encountered poisoned latch /Local/RangeID/4/r/AbortSpan/\"8e91a185-f94d-4e23-879f-fe8eb7efee29\"@0,0"
        	            	actual  : "found more failed ranges than limit 1"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-replica unavailable: (n1,s1):1 unable to serve request to r4:/System{/tsd-tse} [(n1,s1):1, (n5,s5):2, (n2,s2):3, next=4, gen=4]: lost quorum (down: (n5,s5):2,(n2,s2):3); closed timestamp: 1714141378.153025780,0 (2024-04-26 14:22:58); raft status: {"id":"1","term":6,"vote":"1","commit":71,"lead":"0","raftState":"StatePreCandidate","applied":71,"progress":{},"leadtransferee":"0"}: encountered poisoned latch /Local/RangeID/4/r/AbortSpan/"8e91a185-f94d-4e23-879f-fe8eb7efee29"@0,0
        	            	+found more failed ranges than limit 1
        	Test:       	TestRetrieveRangeStatus
    panic.go:523: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/95fc0dff57a5bf3a6a9cc9666d301bb7/logTestRetrieveRangeStatus3529059258
--- FAIL: TestRetrieveRangeStatus (109.91s)

Parameters:

• TAGS=bazel,gss,deadlock
• stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!