Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql/importer: TestRandomParquetExports failed #109431

Closed
cockroach-teamcity opened this issue Aug 24, 2023 · 5 comments
Closed

sql/importer: TestRandomParquetExports failed #109431

cockroach-teamcity opened this issue Aug 24, 2023 · 5 comments
Labels
branch-release-22.2 Used to mark release blockers and technical advisories for 22.2 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-queries SQL Queries Team
Projects
SQL Queries
Milestone
22.2

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Aug 24, 2023 •
edited by cockroach-jira-scripts
Loading

sql/importer.TestRandomParquetExports failed with artifacts on release-22.2 @ 36d58d7b06b3d38da67f514979cae140810ec8bd:

        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next.func1
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:427
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:478
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).queryInternalBuffered
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:592
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).QueryRowExWithCols
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:644
        	            	  | github.com/cockroachdb/cockroach/pkg/sql/importer_test.TestRandomParquetExports.func1
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/importer_test/pkg/sql/importer/exportparquet_test.go:248
        	            	  | github.com/cockroachdb/cockroach/pkg/sql/importer_test.TestRandomParquetExports
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/importer_test/pkg/sql/importer/exportparquet_test.go:268
        	            	  | [...repeated from below...]
        	            	Wraps: (2)
        	            	Wraps: (3)
        	            	Wraps: (4) candidate pg code: 22003
        	            	Wraps: (5) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/sql/sem/tree.init
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/tree/eval.go:33
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6348
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.doInit
        	            	  | 	GOROOT/src/runtime/proc.go:6325
        	            	  | runtime.main
        	            	  | 	GOROOT/src/runtime/proc.go:233
        	            	  | runtime.goexit
        	            	  | 	GOROOT/src/runtime/asm_amd64.s:1594
        	            	Wraps: (6) integer out of range
        	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *colexecerror.notInternalError (4) *pgerror.withCandidateCode (5) *withstack.withStack (6) *errutil.leafError
        	Test:       	TestRandomParquetExports
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/73dfce30b9a5630b1b4dabed3c94b32c/logTestRandomParquetExports3859300761
--- FAIL: TestRandomParquetExports (1.20s)

Parameters: TAGS=bazel,gss

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

Jira issue: CRDB-30922
@cockroach-teamcity cockroach-teamcity added branch-release-22.2 Used to mark release blockers and technical advisories for 22.2 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Aug 24, 2023
@cockroach-teamcity cockroach-teamcity added this to the 22.2 milestone Aug 24, 2023
@cockroach-teamcity cockroach-teamcity added this to Triage (Old) in SQL Queries Aug 24, 2023
@lunevalex lunevalex added the T-sql-queries SQL Queries Team label Aug 24, 2023
@mgartner mgartner added the O-rsg Random Syntax Generator label Aug 25, 2023
@msirek
Copy link
Contributor

msirek commented Aug 26, 2023

Found error in logs:

failed building declarative schema change targets for ALTER TABLE with error: "table1_col1_5_key" is referenced by foreign key from table "table6"

I230824 14:07:52.643262 46279 5@util/log/event_log.go:32 ⋮ [n1,client=127.0.0.1:43704,user=root] 79 ={"Timestamp":1692886072557187644,"EventType":"create_table","Statement":"CREATE TABLE ‹rand›.public.‹table20› (‹col20_0› INET NOT NULL, PRIMARY KEY (‹col20_0› ASC), UNIQUE (‹col20_0›), UNIQUE (‹col20_0› ASC) PARTITION BY LIST (‹col20_0›) (PARTITION ‹table20_part_0› VALUES IN ((‹'103.121.39.134/8'›:::INET,), (‹'0.0.0.0/0'›:::INET,), (‹'25d:fffc:8d3d:7757:1879:6071:ddc8:523c/21'›:::INET,), (‹'77.2.173.72/1'›:::INET,), (‹'b205:5fa8:b159:5125:4f51:39de:115c:c8fb/124'›:::INET,), (‹'6f7b:672a:b609:5853:ed41:dd56:a34f:e9fd/39'›:::INET,), (‹'11.135.52.87/13'›:::INET,), (‹'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff'›:::INET,), (‹'2b74:2524:1728:1aee:6622:a611:52c4:8a1b/86'›:::INET,)), PARTITION ‹table20_part_1› VALUES IN ((‹'35bf:f42e:826:1795:2514:ebf2:e38f:227b/4'›:::INET,), (‹'7389:770:7457:579:6272:e145:f1d7:c743/19'›:::INET,), (‹'247.223.167.91/9'›:::INET,), (‹'223.25.113.221/16'›:::INET,), (‹'c3bf:2daa:c3b6:9bf3:9fd5:fcdc:2e5:e832/127'›:::INET,), (‹'6.158.114.162/2'›:::INET,), (‹'191.11.113.201/10'›:::INET,), (‹'4d1f:20ac:c211:87ab:cf60:47a6:c248:487a/50'›:::INET,), (‹'cc75:f788:e6a2:3e13:da89:1645:6934:3b62/110'›:::INET,)), PARTITION ‹table20_part_2› VALUES IN ((‹'79f4:b91a:e41:d691:b539:57ec:d002:f231/61'›:::INET,), (‹'134.178.11.215/29'›:::INET,), (‹'150.221.44.209/10'›:::INET,), (‹'179.171.24.36/2'›:::INET,), (‹'e30b:ed0a:7448:cede:ba63:47d2:b728:fa27/45'›:::INET,), (‹'90.61.135.82/31'›:::INET,), (‹'df3e:5809:9d60:f81f:2770:5b7:502b:a576/87'›:::INET,)), PARTITION ‹table20_part_3› VALUES IN ((‹'19.152.151.215/12'›:::INET,), (‹'248.220.177.187/30'›:::INET,), (‹'2e28:8b44:aba1:c725:3e70:7652:c21a:81a0/6'›:::INET,), (‹'db07:fba4:5150:c28:653c:3a69:1c39:9e20/121'›:::INET,), (‹'d881:2262:a5a1:aa98:b189:d56:f967:50eb/28'›:::INET,), (‹'41:841f:8466:b9d2:c18f:6477:b748:c7e2/114'›:::INET,), (‹'677:a44a:6aa:fba4:5c91:4160:a1d3:ba8c/74'›:::INET,))), UNIQUE (‹col20_0› ASC), INDEX (‹col20_0› ASC) NOT VISIBLE)","Tag":"CREATE TABLE","User":"root","DescriptorID":125,"TableName":"‹rand.public.table20›"}
I230824 14:07:52.643296 46279 5@util/log/event_log.go:32 ⋮ [n1,client=127.0.0.1:43704,user=root] 80 ={"Timestamp":1692886072557187644,"EventType":"alter_table","Statement":"ALTER TABLE ‹rand›.public.‹table6› ADD FOREIGN KEY (‹col6_2›) REFERENCES ‹rand›.public.‹table1› (‹col1_5›)","Tag":"ALTER TABLE","User":"root","DescriptorID":111,"TableName":"‹rand.public.table6›"}
I230824 14:07:52.784998 46279 sql/schemachanger/scbuild/build.go:38 â‹® [n1,client=127.0.0.1:43704,user=root] 81  building declarative schema change targets for ALTER TABLE
W230824 14:07:52.793555 46279 sql/schemachanger/scbuild/internal/scbuildstmt/drop_index.go:274 ⋮ [n1,client=127.0.0.1:43704,user=root] 82  failed building declarative schema change targets for ALTER TABLE with error: ‹"table1_col1_5_key" is referenced by foreign key from table "table6"›
I230824 14:07:53.433657 45735 sql/stats/automatic_stats.go:509 â‹® [n1] 83  quiescing auto stats refresher
W230824 14:07:53.434150 46300 kv/kvserver/intentresolver/intent_resolver.go:799 ⋮ [-] 84  failed to gc transaction record: could not GC completed transaction anchored at /Table/30/1/‹104›/‹105›/‹"table1"›/‹4›/‹1›: ‹node unavailable; try another peer›
W230824 14:07:53.434341 45741 jobs/registry.go:827 â‹® [n1] 85  canceling all adopted jobs due to stopper quiescing
W230824 14:07:53.434553 45721 sql/sqlliveness/slinstance/slinstance.go:242 â‹® [n1] 86  exiting heartbeat loop
I230824 14:07:53.434798 45997 jobs/registry.go:1227 â‹® [n1] 87  AUTO SPAN CONFIG RECONCILIATION job 893962395121483777: stepping through state succeeded
W230824 14:07:53.434858 45997 kv/txn.go:705 ⋮ [n1] 88  failure aborting transaction: ‹node unavailable; try another peer›; abort caused by: txn exec: context canceled
W230824 14:07:53.439993 45997 jobs/adopt.go:454 ⋮ [n1] 89  could not clear job claim: ‹clear-job-claim: failed to send RPC: sending to all replicas failed; last error: ba: Scan [/Table/15/1/893962395121483777,/Table/15/1/893962395121483778), [txn: 804d78f5], [can-forward-ts] RPC error: node unavailable; try another peer›

@msirek
Copy link
Contributor

msirek commented Aug 29, 2023

@cockroachdb/sql-foundations Does anything look suspicious in the declarative schema changer here?

@msirek msirek added this to Triage in SQL Foundations via automation Aug 29, 2023
@blathers-crl blathers-crl bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Aug 29, 2023
@mgartner mgartner removed the O-rsg Random Syntax Generator label Aug 29, 2023
@rafiss
Copy link
Collaborator

rafiss commented Aug 29, 2023

I don't think that is the problem here. That's just saying the schema changer could not execute that statement since you aren't allowed to remove a key while it's still being referenced by a foreign key.

The error here is:

        	Error Trace:	github.com/cockroachdb/cockroach/pkg/sql/importer_test/pkg/sql/importer/exportparquet_test.go:256
        	            				github.com/cockroachdb/cockroach/pkg/sql/importer_test/pkg/sql/importer/exportparquet_test.go:268
        	Error:      	Received unexpected error:
        	            	integer out of range
        	            	(1) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal.func3.1
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:948
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next.func1
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:427
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:478
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).queryInternalBuffered
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:592
        	            	  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).QueryRowExWithCols
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/internal.go:644
        	            	  | github.com/cockroachdb/cockroach/pkg/sql/importer_test.TestRandomParquetExports.func1
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/importer_test/pkg/sql/importer/exportparquet_test.go:248
        	            	  | github.com/cockroachdb/cockroach/pkg/sql/importer_test.TestRandomParquetExports
        	            	  | 	github.com/cockroachdb/cockroach/pkg/sql/importer_test/pkg/sql/importer/exportparquet_test.go:268

from this query:

cockroach/pkg/sql/importer/exportparquet_test.go

Lines 248 to 255 in 36d58d7

_, cols, err := ie.QueryRowExWithCols(
ctx,
"",
nil,
sessiondata.InternalExecutorOverride{
User: username.RootUserName(),
Database: dbName},
fmt.Sprintf("SELECT * FROM %s LIMIT 1", tableName))

@rafiss rafiss removed the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Aug 29, 2023
@rafiss rafiss removed this from Triage in SQL Foundations Aug 29, 2023
@rharding6373
Copy link
Collaborator

I was unable to repro using the random seed at that commit.

rharding6373 added a commit to rharding6373/cockroach that referenced this issue Sep 6, 2023
@rharding6373
sql/randgen: make insert values deterministic with a seed
c868b7c 
Previously, when randgen built values for an INSERT statement it used
the rand library. This PR uses the passed in rand.Rand created from the
test seed so that test failures can be reproduced deterministically
using COCKROACH_RANDOM_SEED.

Epic: None
Informs: cockroachdb#109431

Release note: None
@rharding6373 rharding6373 mentioned this issue Sep 7, 2023
Merged
@rharding6373
Copy link
Collaborator

I'm going to close this issue as not reproducible. There's a PR open to make flakes like this reproducible so we can investigate next time.

craig bot pushed a commit that referenced this issue Sep 11, 2023
@rharding6373
Merge #110148
83bdbcc 
110148: sql/randgen: make insert values deterministic with a seed r=rharding6373 a=rharding6373

Previously, when randgen built values for an INSERT statement it used the rand library. This PR uses the passed in rand.Rand created from the test seed so that test failures can be reproduced deterministically using COCKROACH_RANDOM_SEED.

Epic: None
Informs: #109431

Release note: None

Co-authored-by: rharding6373 <rharding6373@users.noreply.github.com>
@blathers-crl blathers-crl bot mentioned this issue Sep 12, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Sep 12, 2023
@rharding6373
sql/randgen: make insert values deterministic with a seed
37c730f 
Previously, when randgen built values for an INSERT statement it used
the rand library. This PR uses the passed in rand.Rand created from the
test seed so that test failures can be reproduced deterministically
using COCKROACH_RANDOM_SEED.

Epic: None
Informs: #109431

Release note: None
@yuzefovich yuzefovich mentioned this issue Dec 21, 2023
Closed
@michae2 michae2 mentioned this issue Jan 23, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-release-22.2 Used to mark release blockers and technical advisories for 22.2 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-queries SQL Queries Team
Projects
SQL Queries
Archived in project
SQL Queries
Triage (Old)
Milestone
22.2
Development

No branches or pull requests
6 participants
@mgartner @rafiss @msirek @rharding6373 @cockroach-teamcity @lunevalex