Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

stats: dropping enum values may corrupt stats #67050

Open
ajwerner opened this issue Jun 30, 2021 · 2 comments
Open

stats: dropping enum values may corrupt stats #67050

ajwerner opened this issue Jun 30, 2021 · 2 comments
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-queries SQL Queries Team

Comments

@ajwerner
Copy link
Contributor

ajwerner commented Jun 30, 2021 •
edited by cockroach-jira-scripts
Loading

Describe the problem

We now drop enum values. See this comment:

cockroach/pkg/sql/stats/stats_cache.go

Lines 496 to 504 in d4627db

// The metadata accessed here is never older than the metadata used when
// collecting the stats. Changes to types are backwards compatible across
// versions, so using a newer version of the type metadata here is safe.
// Given that we never delete members from enum types, a descriptor we
// get from the lease manager will be able to be used to decode these stats,
// even if it wasn't the descriptor that was used to collect the stats.
// If have types that are not backwards compatible in this way, then we
// will need to start writing a timestamp on the stats objects and request
// TypeDescriptor's with the timestamp that the stats were recorded with.

Jira issue: CRDB-8341
@ajwerner ajwerner added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Jun 30, 2021
@ajwerner
Copy link
Contributor Author

cc @arulajmani

@yuzefovich yuzefovich mentioned this issue Dec 1, 2022
Closed
@knz knz added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Jul 27, 2023
@exalate-issue-sync exalate-issue-sync bot added T-sql-queries SQL Queries Team and removed T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels May 8, 2024
@michae2 michae2 mentioned this issue May 22, 2024
Closed
@michae2
Copy link
Collaborator

michae2 commented May 23, 2024

Here's one way to trigger this:

SET CLUSTER SETTING sql.stats.automatic_collection.enabled = off;
CREATE TYPE e AS ENUM ('a', 'b', 'c');
CREATE TABLE x (x e PRIMARY KEY);
INSERT INTO x VALUES ('a'), ('b'), ('c');
ANALYZE x;
DELETE FROM x WHERE x = 'a';
ALTER TYPE e DROP VALUE 'a';
SHOW STATISTICS USING JSON FOR TABLE x;

This fails with an internal error with a stacktrace like:

ERROR: could not find [64] in enum "defaultdb.public.e" representation PhysicalReps: [[128] [192]]; LogicalReps: [b c] goroutine 6686 [running]:
runtime/debug.Stack()
	GOROOT/src/runtime/debug/stack.go:24 +0x64
github.com/cockroachdb/cockroach/pkg/sql/types.(*T).EnumGetIdxOfPhysical(0x14001ea1380, {0x1400b4585c9, 0x1, 0x7})
	github.com/cockroachdb/cockroach/pkg/sql/types/pkg/sql/types/types.go:2693 +0x1b0
github.com/cockroachdb/cockroach/pkg/sql/sem/tree.GetEnumComponentsFromPhysicalRep(0x14001ea1380, {0x1400b4585c9?, 0x0?, 0x0?})
	github.com/cockroachdb/cockroach/pkg/sql/sem/tree/datum.go:5248 +0x24
github.com/cockroachdb/cockroach/pkg/sql/rowenc/keyside.Decode(0x14007f91ec0, 0x14001ea1380, {0x1400b4585c8, 0x4, 0x8}, 0x1)
	github.com/cockroachdb/cockroach/pkg/sql/rowenc/keyside/decode.go:292 +0x1d58
github.com/cockroachdb/cockroach/pkg/sql/stats.decodeUpperBound(0x14001ea1380?, 0x1099122e0?, {0x1400b4585c8?, 0x1035d3ab4?, 0x1?})
	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/histogram.go:99 +0x50
github.com/cockroachdb/cockroach/pkg/sql/stats.(*JSONStatistic).SetHistogram(0x14007f92600, 0x14005efebd0)
	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/json.go:80 +0x1d8
github.com/cockroachdb/cockroach/pkg/sql/stats.(*JSONStatistic).DecodeAndSetHistogram(0x10995b440?, {0x1099a6400, 0x140065ccde0}, 0x140032eca50, {0x109a0aaf8?, 0x14004dd25a0?})
	github.com/cockroachdb/cockroach/pkg/sql/stats/pkg/sql/stats/json.go:127 +0x1bc
github.com/cockroachdb/cockroach/pkg/sql.(*planner).ShowTableStats.func1({0x1099a6400, 0x140065ccde0}, 0x140032ec678)
	github.com/cockroachdb/cockroach/pkg/sql/show_stats.go:303 +0x6f0
github.com/cockroachdb/cockroach/pkg/sql.(*delayedNode).startExec(0x14004fdc780, {{0x1099a6400?, 0x140065ccde0?}, 0x140089f9800?, 0x140032ec678?})
	github.com/cockroachdb/cockroach/pkg/sql/delayed.go:50 +0x5c
github.com/cockroachdb/cockroach/pkg/sql.startExec.func2({0x14007f928b8?, 0x105c83b50?}, {0x1099a9928, 0x14004fdc780})
	github.com/cockroachdb/cockroach/pkg/sql/plan.go:520 +0xf4
github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal.func1()
	github.com/cockroachdb/cockroach/pkg/sql/walk.go:112 +0x44
github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal(0x14007f929d0, {0x1099a9928?, 0x14004fdc780?}, {0x1073f1ec0?, 0xd?})
	github.com/cockroachdb/cockroach/pkg/sql/walk.go:299 +0x10a0
github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visit(0x14007f929d0, {0x1099a9928, 0x14004fdc780})
	github.com/cockroachdb/cockroach/pkg/sql/walk.go:79 +0xe8
github.com/cockroachdb/cockroach/pkg/sql.walkPlan(...)
	github.com/cockroachdb/cockroach/pkg/sql/walk.go:43
github.com/cockroachdb/cockroach/pkg/sql.startExec({{0x1099a6400?, 0x140065ccde0?}, 0x140089f9800?, 0x140032ec678?}, {0x1099a9928, 0x14004fdc780})
	github.com/cockroachdb/cockroach/pkg/sql/plan.go:523 +0xdc
github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start(0x14003f3adc0, {0x1099a6400, 0x140065ccdb0})
	github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:175 +0x104
github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Columnarizer).Init(0x14006ffd200, {0x1099a6400, 0x140065ccd80})
	github.com/cockroachdb/cockroach/pkg/sql/colexec/columnarizer.go:183 +0xdc
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).init(...)
	github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:94
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError(0x1400b456c80?)
	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92 +0x58
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Init(0x14008a91f00, {0x1099a6400?, 0x140065ccd80?})
	github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:103 +0x74
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).Run.(*BatchFlowCoordinator).init.func2()
	github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:245 +0x34
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError(0x1099a63c8?)
	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92 +0x58
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).init(...)
	github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:244
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).Run(0x140051aa960, {0x1099a63c8?, 0x14004fdc820?})
	github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:278 +0xbc
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run(0x1400708cb40, {0x1099a63c8, 0x14004fdc820}, 0x60?)
	github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:320 +0x1a0
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0x140014aeb40, {0x1099a6400, 0x140065cc660}, 0x1400729c8f0, 0x14004f86960, 0x14005fc5e00, 0x1400405aa80, 0x140032ecb40, 0x14007f938b0)
	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:905 +0x8a8
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun(0x140014aeb40, {0x1099a6400, 0x140065cc660}, 0x140032ecb40, 0x1400729c8f0, 0x1099a9928?, {{0x1099a9928, 0x14004fdc780}, 0x0}, 0x1400405aa80, ...)
	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1945 +0x170
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3(0x1400608fca0?, 0x0?, {0x1099a6400?, 0x140065cc660?}, 0x14007f93928?, 0x102daf214?, 0x10?)
	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1660 +0xa0
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll(0x1400729c8f0?, {0x1099a6400?, 0x140065cc660}, 0x140032ecb40, 0x1400729c8f0, 0x140032ec678, 0x1400405aa80, 0x0)
	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1663 +0x180
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine(0x140032ec000, {0x1099a6400?, 0x140065cc660}, 0x140032ec678, 0x0?, {0x1391484b0?, 0x1400379ff28}, 0x14007f93e30?, 0x140073f3898)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:2405 +0x320
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine(0x140032ec000, {0x1099a6400, 0x14006a9c420}, 0x140032ec678, {0x1391484b0?, 0x1400379ff28})
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1944 +0xe18
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState(0x140032ec000, {0x1099a6400, 0x14006a9c420}, {{0x1099cc2e0, 0x14006a9c1b0}, {0x0, 0x0, 0x0}, {0x140079b5253, 0x26}, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1125 +0x3560
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func1({0x1099a6400?, 0x14006a9c210?})
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:145 +0x8c
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling(0x1099a6400?, {0x1099a6400?, 0x14006a9c210?}, {0x1099cc2e0?, 0x14006a9c1b0?}, 0x1400379f800?, 0x100000000?)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:3398 +0x27c
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt(0x140032ec000, {0x1099a6400, 0x14006a9c210}, {{0x1099cc2e0, 0x14006a9c1b0}, {0x0, 0x0, 0x0}, {0x140079b5253, 0x26}, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:144 +0x318
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1(0x14007f95968, 0x14007f95580, 0x140032ec000, 0x14007f954a8, 0x14007f955a0, 0x14007f95590, 0x14007f955b0, 0x14007f955c0)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2255 +0x2c0
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0x140032ec000)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2260 +0x2a8
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0x140032ec000, {0x1099a6400, 0x14005ad9860}, 0x10375a988?, 0x1400272aea0?, 0x140020dac30?)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2177 +0x168
github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn(0x140070683c0?, {0x1099a6400?, 0x14005ad9860?}, {0x1400476a02a?}, 0x9?, 0x14005ad96e0?)
	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:964 +0xb4
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommands(0x1400379f800, {0x1099a6400?, 0x14005ad9860}, {0x0, 0x4, {0x1, {0x1073b7ffb, 0x3}, {0x14007916610, 0xb}, ...}, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:247 +0x2ac
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveImpl.func3()
	github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:999 +0xcc
created by github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveImpl in goroutine 6685
	github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:996 +0x4a0

@michae2 michae2 mentioned this issue May 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-queries SQL Queries Team
Projects
SQL Queries
Status: 25.1 Release
Milestone
No milestone
Development

No branches or pull requests
3 participants
@knz @ajwerner @michae2