Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

release-20.2: vendor: bump Pebble to c3ef93f9a9ed #58304

Merged
merged 1 commit into from
Dec 30, 2020
Merged

release-20.2: vendor: bump Pebble to c3ef93f9a9ed #58304

merged 1 commit into from
Dec 30, 2020

Conversation

jbowens
Copy link
Collaborator

@jbowens jbowens commented Dec 28, 2020 

c3ef93f9 db: scale point tombstone averages according to file size
9b9c1284 db: add point tombstone compensation

Release note (bug fix): Fix a storage layer bug that could cause deleted
system.jobs rows to remain on-disk indefinitely. The bloated system.jobs
table could make jobs completely unavailable and prevent DDL statements
from executing. This bug can be detected by examining the system.jobs
table size from the DB Console. This change fixes the bug for Pebble
only and the bug still persists on RocksDB.

@jbowens
vendor: bump Pebble to c3ef93f9a9ed
e428ce7 
```
c3ef93f9 db: scale point tombstone averages according to file size
9b9c1284 db: add point tombstone compensation
```

Release note (bug fix): Fix a storage layer bug that could cause deleted
system.jobs rows to remain on-disk indefinitely. The bloated system.jobs
table could make jobs completely unavailable and prevent DDL statements
from executing. This bug can be detected by examining the system.jobs
table size from the DB Console. This change fixes the bug for Pebble
only and the bug still persists on RocksDB.
@cockroach-teamcity
Copy link
Member

This change is Reviewable

itsbilal
itsbilal approved these changes Dec 28, 2020
View reviewed changes
Copy link
Member

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @petermattis and @sumeerbhola)

@jbowens
Copy link
Collaborator Author

jbowens commented Dec 28, 2020

The acceptance/version-upgrade roachtest failed in CI with:

I201228 16:35:06.841433 201 server/server.go:1119 â‹® active cluster version is now 20.2 (up from 20.1)
I201228 16:35:08.772686 793 sql/sqlliveness/slstorage/slstorage.go:342 ⋮ [n2] inserted sqlliveness session ‹a418c755aebf4c96b7bbefa64d909a9b›
I201228 16:35:08.772738 793 sql/sqlliveness/slinstance/slinstance.go:143 ⋮ [n2] created new SQL liveness session ‹a418c755aebf4c96b7bbefa64d909a9b›
I201228 16:35:08.777500 5035 sql/table.go:102 ⋮ [n2,client=‹127.0.0.1:58956›,hostnossl,user=root] queued new database schema change job 619652411466579970 for database 162
F201228 16:35:08.777559 5035 sql/database.go:112 ⋮ [n2,client=‹127.0.0.1:58956›,hostnossl,user=root] invalid attempted write of database descriptor
goroutine 5035 [running]:
github.com/cockroachdb/cockroach/pkg/util/log.getStacks(0x95a7701, 0x742236, 0xc00130eb80, 0x20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/get_stacks.go:25 +0xb8
github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(0x95a69a0, 0x4, 0x1654ee993bb9f652, 0x13ab, 0x874e1bf, 0xf, 0x70, 0xc0021cc750, 0x2e, 0xc00086eac0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:243 +0x9e5
github.com/cockroachdb/cockroach/pkg/util/log.addStructured(0x618ba80, 0xc002726400, 0x4, 0x2, 0x512df6f, 0x2e, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/structured.go:52 +0x1ac
github.com/cockroachdb/cockroach/pkg/util/log.logDepth(0x618ba80, 0xc002726400, 0x1, 0xc000000004, 0x512df6f, 0x2e, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:44 +0x8c
github.com/cockroachdb/cockroach/pkg/util/log.Fatal(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:173
github.com/cockroachdb/cockroach/pkg/sql.(*planner).writeDatabaseChangeToBatch(0xc001e9b418, 0x618ba80, 0xc002726400, 0xc0029f7980, 0xc000f9e000, 0x4e, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/database.go:112 +0x13f
github.com/cockroachdb/cockroach/pkg/sql.(*planner).writeNonDropDatabaseChange(0xc001e9b418, 0x618ba80, 0xc002726400, 0xc0029f7980, 0xc0009453b0, 0x4e, 0x4e, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/database.go:99 +0xeb
github.com/cockroachdb/cockroach/pkg/sql.(*planner).createUserDefinedSchema(0xc001e9b418, 0x618ba80, 0xc002726400, 0xc001f47340, 0xc001e9b418, 0xc00169f230, 0x7b5601, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/create_schema.go:136 +0x83f
github.com/cockroachdb/cockroach/pkg/sql.(*createSchemaNode).startExec(0xc001a69f08, 0x618ba80, 0xc002726400, 0xc001f47340, 0xc001e9b418, 0xc001c35300, 0x2749212)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/create_schema.go:36 +0x5b
github.com/cockroachdb/cockroach/pkg/sql.startExec.func2(0x5092b12, 0xd, 0x618d900, 0xc001a69f08, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/plan.go:506 +0xb9
github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal.func1(0xc0027996e8, 0x5092b12, 0xd, 0x618d900, 0xc001a69f08)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:119 +0x5d
github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visitInternal(0xc001c356e8, 0x618d900, 0xc001a69f08, 0x5092b12, 0xd)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:298 +0xbf
github.com/cockroachdb/cockroach/pkg/sql.(*planVisitor).visit(0xc001c356e8, 0x618d900, 0xc001a69f08, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:86 +0x8d
github.com/cockroachdb/cockroach/pkg/sql.walkPlan(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/walk.go:50
github.com/cockroachdb/cockroach/pkg/sql.startExec(0x618ba80, 0xc002726400, 0xc001f47340, 0xc001e9b418, 0x618d900, 0xc001a69f08, 0x0, 0x2)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/plan.go:509 +0x11f
github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start(0xc000f9ef00, 0x618ba80, 0xc002726400, 0x95f4e20, 0x48a1840)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:125 +0xd6
github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBase).Run(0xc000f9ef00, 0x618ba80, 0xc002726400)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:774 +0x52
github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run(0xc0019c6c60, 0x618ba80, 0xc002726400, 0x567fc80, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:392 +0x24d
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0xc0008914a0, 0xc0008477a0, 0xc001934a20, 0xc002a2ec00, 0xc00012ea80, 0xc001e9b518, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:422 +0x5d9
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun(0xc0008914a0, 0x618bb40, 0xc00169f290, 0xc001e9b518, 0xc0008477a0, 0xc001934a20, 0x618d900, 0xc001a69f08, 0x0, 0xc00012ea80, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1002 +0x1d1
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine(0xc001e9b000, 0x618bb40, 0xc00169f290, 0xc001e9b418, 0x1, 0x7f277540da78, 0xc001fda680, 0xc001c36100, 0xc000910ef8, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1001 +0x3c0
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine(0xc001e9b000, 0x618bb40, 0xc00169f290, 0xc001e9b418, 0x7f277540da78, 0xc001fda680, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:872 +0x6ff
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState(0xc001e9b000, 0x618bb40, 0xc00169f290, 0x61a9440, 0xc00169f230, 0xc002ca63b5, 0x24, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:639 +0xbb1
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt(0xc001e9b000, 0x618bb40, 0xc00169f290, 0x61a9440, 0xc00169f230, 0xc002ca63b5, 0x24, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:114 +0x924
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1(0x61a9440, 0xc00169f230, 0xc002ca63b5, 0x24, 0x0, 0x0, 0x362d543f, 0xed77c003a, 0x0, 0x362d5cb9, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1465 +0x2d5
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc001e9b000, 0x618ba80, 0xc002726340, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1467 +0x148d
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc001e9b000, 0x618ba80, 0xc002c8a080, 0xc000e34fa0, 0x5400, 0x15000, 0xc000e35040, 0xc002c82090, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1391 +0x1f2
github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn(0xc0010b4680, 0x618ba80, 0xc002c8a080, 0xc001e9b000, 0x5400, 0x15000, 0xc000e35040, 0xc002c82090, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:508 +0x104
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1(0xc0026a003e, 0xc002a82080, 0x618ba80, 0xc002c8a080, 0xc002c82090, 0xc0010b4680, 0xc001fda000, 0x61bb380, 0xc0023cc330, 0xc00275a3c0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:626 +0x317
created by github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:554 +0x17e

@jbowens
Copy link
Collaborator Author

jbowens commented Dec 28, 2020

Re-run succeeded and it seems unlikely to be related to this change. I don't see an issue tracking a related problem though. I'll create one.

@jbowens jbowens mentioned this pull request Dec 28, 2020
Closed
@jbowens
Copy link
Collaborator Author

jbowens commented Dec 30, 2020

TFTR!

@jbowens jbowens merged commit 18ed1ec into cockroachdb:release-20.2 Dec 30, 2020
@jbowens jbowens deleted the jackson/pebble-c3ef93f9a9ed branch December 30, 2020 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@itsbilal itsbilal itsbilal approved these changes

@petermattis petermattis Awaiting requested review from petermattis

@sumeerbhola sumeerbhola Awaiting requested review from sumeerbhola

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@jbowens @cockroach-teamcity @itsbilal