Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: TestDropTableWhileUpgradingFormat failed #108340

Closed
cockroach-teamcity opened this issue Aug 7, 2023 · 6 comments · Fixed by #108880
Closed

sql: TestDropTableWhileUpgradingFormat failed #108340

cockroach-teamcity opened this issue Aug 7, 2023 · 6 comments · Fixed by #108880
Assignees
@annrpom
Labels
branch-master Failures on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
SQL Foundations
Milestone
23.2

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Aug 7, 2023 •
edited by cockroach-jira-scripts
Loading

sql.TestDropTableWhileUpgradingFormat failed with artifacts on master @ 8c1ff5188f37d8b3b4b1f153c16a38e876e53973:

=== RUN   TestDropTableWhileUpgradingFormat
    test_log_scope.go:167: test logs captured to: /artifacts/tmp/_tmp/765430912bd62afa1f42a127d71d4b8a/logTestDropTableWhileUpgradingFormat3691027404
    test_log_scope.go:81: use -show-logs to present logs inline
    drop_test.go:828: expecting 100 keys, but found 0
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/765430912bd62afa1f42a127d71d4b8a/logTestDropTableWhileUpgradingFormat3691027404
--- FAIL: TestDropTableWhileUpgradingFormat (2.72s)
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-30431
@cockroach-teamcity cockroach-teamcity added branch-master Failures on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Aug 7, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.2 milestone Aug 7, 2023
This was referenced Aug 8, 2023
Closed
Merged
@yuzefovich
Copy link
Member

Another instance here.

@cockroach-teamcity
Copy link
Member Author

sql.TestDropTableWhileUpgradingFormat failed with artifacts on master @ a629092bb552144ec05954ad8c000e35c8da8ef0:

=== RUN   TestDropTableWhileUpgradingFormat
    test_log_scope.go:167: test logs captured to: /artifacts/tmp/_tmp/1061c85d1b78da61f97933da6fe84ebc/logTestDropTableWhileUpgradingFormat2788016457
    test_log_scope.go:81: use -show-logs to present logs inline
    drop_test.go:828: expecting 100 keys, but found 0
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/1061c85d1b78da61f97933da6fe84ebc/logTestDropTableWhileUpgradingFormat2788016457
--- FAIL: TestDropTableWhileUpgradingFormat (1.62s)
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@renatolabs
Copy link
Collaborator

Another flake: https://teamcity.cockroachdb.com/viewLog.html?buildId=11319284&buildTypeId=Cockroach_BazelEssentialCi

@exalate-issue-sync exalate-issue-sync bot assigned annrpom and unassigned fqazi Aug 15, 2023
@cockroach-teamcity
Copy link
Member Author

sql.TestDropTableWhileUpgradingFormat failed with artifacts on master @ 6716e92795a8362a7b49cb33443d1b07e536f325:

=== RUN   TestDropTableWhileUpgradingFormat
    test_log_scope.go:167: test logs captured to: /artifacts/tmp/_tmp/230b14ff9b70e2729ccc9b9deabcc166/logTestDropTableWhileUpgradingFormat875673394
    test_log_scope.go:81: use -show-logs to present logs inline
    drop_test.go:832: expecting 100 keys, but found 0
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/230b14ff9b70e2729ccc9b9deabcc166/logTestDropTableWhileUpgradingFormat875673394
--- FAIL: TestDropTableWhileUpgradingFormat (0.96s)
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@rafiss rafiss mentioned this issue Aug 17, 2023
Merged
craig bot pushed a commit that referenced this issue Aug 17, 2023
@annrpom
Merge #108880
a0c30d2 
108880: sql: deflake TestDropTableWhileUpgradingFormat r=annrpom a=annrpom

Previously, the `CheckKeyCountIncludingTombstoned` call in TestDropTableWhileUpgradingFormat flaked with a key count mismatch error (expected: 100, actual: 0). This was odd because this key counting failure was directly after rows were committed. To address this, this patch adds a retry for the `CheckKeyCountIncludingTombstoned` logic in case this is due to a race condition.

Epic: none
Fixes: #108340

Release note (sql change): deflake TestDropTableWhileUpgradingFormat

Co-authored-by: Annie Pompa <annie@cockroachlabs.com>
@craig craig bot closed this as completed in 29bf82f Aug 17, 2023
SQL Foundations automation moved this from Triage to Done Aug 17, 2023
@jayshrivastava
Copy link
Contributor

👋🏽 Can you please backport this to the relevant branches? I got a failure on release-23.1.9-rc. Probably worth backporting to 22.2 and 23.1 too. https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_BazelEssentialCi/11381798?hideProblemsFromDependencies=false&hideTestsFromDependencies=false&expandBuildChangesSection=true&expandBuildProblemsSection=true&expandBuildTestsSection=true

SQL Foundations automation moved this from Done to Triage Aug 18, 2023
@blathers-crl blathers-crl bot mentioned this issue Aug 18, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Aug 18, 2023
@annrpom
sql: deflake TestDropTableWhileUpgradingFormat
5769725 
Previously, the `CheckKeyCountIncludingTombstoned` call in
TestDropTableWhileUpgradingFormat flaked with a key count
mismatch error (expected: 100, actual: 0). This was odd
because this key counting failure was directly after rows
were committed. To address this, this patch adds a retry
for the `CheckKeyCountIncludingTombstoned` logic in case
this is due to a race condition.

Epic: none
Fixes: #108340

Release note (sql change): deflake TestDropTableWhileUpgradingFormat
@blathers-crl blathers-crl bot mentioned this issue Aug 18, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Aug 18, 2023
@annrpom
sql: deflake TestDropTableWhileUpgradingFormat
9a1d4fc 
Previously, the `CheckKeyCountIncludingTombstoned` call in
TestDropTableWhileUpgradingFormat flaked with a key count
mismatch error (expected: 100, actual: 0). This was odd
because this key counting failure was directly after rows
were committed. To address this, this patch adds a retry
for the `CheckKeyCountIncludingTombstoned` logic in case
this is due to a race condition.

Epic: none
Fixes: #108340

Release note (sql change): deflake TestDropTableWhileUpgradingFormat
@cockroach-teamcity cockroach-teamcity mentioned this issue Aug 18, 2023
Closed
annrpom added a commit to annrpom/cockroach that referenced this issue Aug 18, 2023
@annrpom
sql: deflake TestDropTableWhileUpgradingFormat
df5b655 
Previously, the `CheckKeyCountIncludingTombstoned` call in
TestDropTableWhileUpgradingFormat flaked with a key count
mismatch error (expected: 100, actual: 0). This was odd
because this key counting failure was directly after rows
were committed. To address this, this patch adds a retry
for the `CheckKeyCountIncludingTombstoned` logic in case
this is due to a race condition.

Epic: none
Fixes: cockroachdb#108340

Release note: none

Release justification: test only change
@annrpom annrpom mentioned this issue Aug 18, 2023
Closed
@rafiss
Copy link
Collaborator

rafiss commented Aug 18, 2023

backports are done

@rafiss rafiss closed this as completed Aug 18, 2023
SQL Foundations automation moved this from Triage to Done Aug 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@annrpom annrpom

Labels
branch-master Failures on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
SQL Foundations
  
Done
Milestone
23.2
Development

Successfully merging a pull request may close this issue.

sql: deflake TestDropTableWhileUpgradingFormat annrpom/cockroach
7 participants
@rafiss @fqazi @yuzefovich @jayshrivastava @cockroach-teamcity @annrpom @renatolabs