roachtest: restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8 failed

[cockroach-teamcity]

roachtest.restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8 failed with artifacts on release-23.1 @ ed045dc72631760a58a58d46de01d6ffacd3335a:

(monitor.go:153).Wait: monitor failure: full command output in run_075422.983552447_n1_cockroach-sql-insecu.log: COMMAND_PROBLEM: exit status 1
test artifacts and logs in: /artifacts/restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8/run_1

Parameters:

• ROACHTEST_arch=amd64
• ROACHTEST_cloud=aws
• ROACHTEST_cpu=8
• ROACHTEST_encrypted=false
• ROACHTEST_fs=ext4
• ROACHTEST_localSSD=false
• ROACHTEST_metamorphicBuild=false
• ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Grafana is not yet available for aws clusters

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-38668

[msbutler]

looks like the connection while reading an sst timed out. i wonder why the restore did not retry.

ERROR: importing 1023 ranges: read backups/tpc-e/customers=25000/v22.2.0/inc-count=48/2022/12/20-225543.90/data/824137888054378500.sst: read tcp 10.12.11.188:59822->3.5.132.110:443: read: connection timed out

[stevendanna]

This is 23.1 so I don't think it has your retry improvements during restore. So perhaps it did retry at the sdk level but still timed out.

[msbutler]

ah right, i say we let this issue ride out into the sunset.

[msbutler]

closing as 23.1 is out of support level.