roachtest: schemachange/random-load failed [revert stuck forever] #122659
Assignees
Labels
branch-master
Failures on the master branch.
C-test-failure
Broken test (automatically or manually discovered).
O-roachtest
O-robot
Originated from a bot.
P-1
Issues/test failures with a fix SLA of 1 month
T-sql-foundations
SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
Milestone
Comments
cockroach-teamcity
added
branch-master
rafiss
removed
the
release-blocker
|
It looks like these client-side goroutines were stuck waiting for a response: |
rafiss
changed the title
|
This is the main thing that's suspicious from the CRDB logs: That's logged at 18:23 on n2, which is when the operations on the cluster stopped. It happens again at 18:50 and 20:25, when I think is the schemachanger retrying the revert, but failing in the same way. @fqazi do you think we should make "type with ID 1112 does not exist" a non-retriable error? |
|
roachtest.schemachange/random-load failed with artifacts on master @ 5d02bd9ff6b2bccecf6d43fc6cd647167b91f782: Parameters:
Same failure on other branches
|
fqazi
added a commit
to fqazi/cockroach
that referenced
this issue
Apr 30, 2024
Previously, when rolling back type descriptor schema changes if the descriptor was already dropped we would keep retrying the schema change. This would happen because we introduced a regression where the internal structured error was replaced with user facing pgerror based error. To address this, this patch will properly handle the UndefinedObject pgcode and avoid retrying during a rollback of a typedesc schema change. Fixes: cockroachdb#122958, cockroachdb#122659 Release note (bug fix): TYPEDESC SCHEMA CHANGE jobs could end up retrying forever if the descriptor targeted by them was already dropped.
blathers-crl bot
pushed a commit
that referenced
this issue
Apr 30, 2024
Previously, when rolling back type descriptor schema changes if the descriptor was already dropped we would keep retrying the schema change. This would happen because we introduced a regression where the internal structured error was replaced with user facing pgerror based error. To address this, this patch will properly handle the UndefinedObject pgcode and avoid retrying during a rollback of a typedesc schema change. Fixes: #122958, #122659 Release note (bug fix): TYPEDESC SCHEMA CHANGE jobs could end up retrying forever if the descriptor targeted by them was already dropped.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
roachtest.schemachange/random-load failed with artifacts on master @ 2d67111f0db7bec9bfed537542c60f37f8340f69:
Parameters:
ROACHTEST_arch=amd64ROACHTEST_cloud=gceROACHTEST_coverageBuild=falseROACHTEST_cpu=4ROACHTEST_encrypted=falseROACHTEST_fs=ext4ROACHTEST_localSSD=trueROACHTEST_metamorphicBuild=falseROACHTEST_ssd=0Help
See: roachtest README
See: How To Investigate (internal)
See: Grafana
This test on roachdash | Improve this report!
Jira issue: CRDB-38016
The text was updated successfully, but these errors were encountered: