Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sentry: table_meta.go:443: partial index predicate does not exist in table metadata (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/sql... #124239

Closed
cockroach-sentry opened this issue May 15, 2024 · 1 comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented May 15, 2024

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/5355667853/?referrer=webhooks_plugin

Panic Message:

table_meta.go:443: partial index predicate does not exist in table metadata
(1) assertion failure
Wraps: (2) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/opt.(*TableMeta).PartialIndexPredicate
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/table_meta.go:443
  | github.com/cockroachdb/cockroach/pkg/sql/opt/norm.deriveScanRejectNullCols
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/norm/reject_nulls_funcs.go:372
  | github.com/cockroachdb/cockroach/pkg/sql/opt/norm.DeriveRejectNullCols
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/norm/reject_nulls_funcs.go:214
  | github.com/cockroachdb/cockroach/pkg/sql/opt/norm.(*CustomFuncs).RejectNullCols
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/norm/reject_nulls_funcs.go:25
  | github.com/cockroachdb/cockroach/pkg/sql/opt/norm.(*Factory).ConstructLeftJoin
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/opt/norm/factory.og.go:4278
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*mutationBuilder).buildLeftJoinForUpsertArbiter
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/mutation_builder_arbiter.go:431
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*mutationBuilder).buildInputForUpsert.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/insert.go:798
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*arbiterSet).ForEach.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/arbiter_set.go:123
  | github.com/cockroachdb/cockroach/pkg/util/intsets.Fast.ForEach
  | 	github.com/cockroachdb/cockroach/pkg/util/intsets/fast.go:154
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*arbiterSet).ForEach
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/arbiter_set.go:112
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*mutationBuilder).buildInputForUpsert
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/insert.go:769
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildInsert
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/insert.go:337
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmt.func2
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:349
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).processWiths
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/with.go:116
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmt
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:348
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmtAtRoot
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:277
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).Build
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:251
  | github.com/cockroachdb/cockroach/pkg/sql.(*optPlanningCtx).buildReusableMemo
  | 	github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:441
  | github.com/cockroachdb/cockroach/pkg/sql.(*planner).prepareUsingOptimizer
  | 	github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:169
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).populatePrepared
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:334
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare.func2
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:272
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:277
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).addPreparedStmt
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:116
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPrepare
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:77
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2312
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2103
  | github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:902
  | github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:310
  | runtime.goexit
  | 	src/runtime/asm_arm64.s:1172
Wraps: (3) partial index predicate does not exist in table metadata
Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *errutil.leafError
-- report composition:
*errutil.leafError: partial index predicate does not exist in table metadata
table_meta.go:443: *withstack.withStack (top exception)
*assert.withAssertionFailure
Stacktrace (expand for inline code snippets):

src/runtime/asm_arm64.s#L1171-L1173

reservedOwned = false // We're about to pass ownership away.
retErr = sqlServer.ServeConn(
ctx,

}(ctx, h)
return h.ex.run(ctx, s.pool, reserved, cancel)
}

var err error
if err = ex.execCmd(); err != nil {
// Both of these errors are normal ways for the connExecutor to exit.

stmtCtx := withStatement(ctx, ex.curStmtAST)
ev, payload = ex.execPrepare(stmtCtx, tcmd)
case DescribeStmt:

stmt := makeStatement(parseCmd.Statement, ex.server.cfg.GenerateID())
_, err := ex.addPreparedStmt(
ctx,

// Prepare the query. This completes the typing of placeholders.
prepared, err := ex.prepare(ctx, stmt, placeholderHints, rawTypeHints, origin)
if err != nil {

// Use the existing transaction.
if err := prepare(ctx, ex.state.mu.txn); err != nil {
if origin != PreparedStatementOriginSessionMigration {

p.semaCtx.Annotations = tree.MakeAnnotations(stmt.NumAnnotations)
flags, err = ex.populatePrepared(ctx, txn, placeholderHints, p, origin)
return err

// and execute paths.
flags, err := p.prepareUsingOptimizer(ctx, origin)
if err != nil {

memo, err := opc.buildReusableMemo(ctx)
if err != nil {

}
if err := bld.Build(); err != nil {
return nil, err

// and physical properties.
outScope := b.buildStmtAtRoot(b.stmt, nil /* desiredTypes */)

b.ctes = nil
outScope = b.buildStmt(stmt, desiredTypes, inScope)
// Build With operators for any CTEs hoisted to the top level.

case *tree.Insert:
return b.processWiths(stmt.With, inScope, func(inScope *scope) *scope {
return b.buildInsert(stmt, inScope)

inScope.atRoot = false
outScope := buildStmt(inScope)
outScope.expr = b.buildWiths(outScope.expr, correlatedCTEs)

return b.processWiths(stmt.With, inScope, func(inScope *scope) *scope {
return b.buildInsert(stmt, inScope)
})

// as the join condition.
mb.buildInputForUpsert(inScope, ins.OnConflict, ins.OnConflict.Where)

var canaryCol *scopeColumn
mb.arbiters.ForEach(func(name string, conflictOrds intsets.Fast, pred tree.Expr, canaryOrd int) {
// If the arbiter has a partial predicate, project a new column that

// Call the callback for each index arbiter.
a.indexes.ForEach(func(i int) {
index := a.mb.tab.Index(i)

i := bits.TrailingZeros64(v)
f(i)
v &^= 1 << uint(i)

f(string(index.Name()), conflictOrds, pred, canaryOrd)
})

// Create a left-join for the arbiter.
mb.buildLeftJoinForUpsertArbiter(
inScope, conflictOrds, pred,

// Construct the left join.
mb.outScope.expr = mb.b.factory.ConstructLeftJoin(
mb.outScope.expr,

https://github.com/cockroachdb/cockroach/blob/37f75744ac468fe5a97b343a188ea248bce8fb4a/bazel-out/k8-opt/bin/pkg/sql/opt/norm/factory.og.go#L4277-L4279
func (c *CustomFuncs) RejectNullCols(in memo.RelExpr) opt.ColSet {
return DeriveRejectNullCols(in, c.f.disabledRules)
}

case opt.ScanOp:
relProps.Rule.RejectNullCols.UnionWith(deriveScanRejectNullCols(in))
}

for i, n := 0, md.Table(scan.Table).IndexCount(); i < n; i++ {
if pred, isPartialIndex := md.TableMeta(scan.Table).PartialIndexPredicate(i); isPartialIndex {
predFilters := *pred.(*memo.FiltersExpr)

if !ok {
panic(errors.AssertionFailedf("partial index predicate does not exist in table metadata"))
}

src/runtime/asm_arm64.s in runtime.goexit at line 1172
pkg/sql/pgwire/conn.go in pkg/sql/pgwire.(*conn).processCommandsAsync.func1 at line 310
pkg/sql/conn_executor.go in pkg/sql.(*Server).ServeConn at line 902
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2103
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2312
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).execPrepare at line 77
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).addPreparedStmt at line 116
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).prepare at line 277
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).prepare.func2 at line 272
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).populatePrepared at line 334
pkg/sql/plan_opt.go in pkg/sql.(*planner).prepareUsingOptimizer at line 169
pkg/sql/plan_opt.go in pkg/sql.(*optPlanningCtx).buildReusableMemo at line 441
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).Build at line 251
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).buildStmtAtRoot at line 277
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).buildStmt at line 348
pkg/sql/opt/optbuilder/with.go in pkg/sql/opt/optbuilder.(*Builder).processWiths at line 116
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).buildStmt.func2 at line 349
pkg/sql/opt/optbuilder/insert.go in pkg/sql/opt/optbuilder.(*Builder).buildInsert at line 337
pkg/sql/opt/optbuilder/insert.go in pkg/sql/opt/optbuilder.(*mutationBuilder).buildInputForUpsert at line 769
pkg/sql/opt/optbuilder/arbiter_set.go in pkg/sql/opt/optbuilder.(*arbiterSet).ForEach at line 112
pkg/util/intsets/fast.go in pkg/util/intsets.Fast.ForEach at line 154
pkg/sql/opt/optbuilder/arbiter_set.go in pkg/sql/opt/optbuilder.(*arbiterSet).ForEach.func1 at line 123
pkg/sql/opt/optbuilder/insert.go in pkg/sql/opt/optbuilder.(*mutationBuilder).buildInputForUpsert.func1 at line 798
pkg/sql/opt/optbuilder/mutation_builder_arbiter.go in pkg/sql/opt/optbuilder.(*mutationBuilder).buildLeftJoinForUpsertArbiter at line 431
bazel-out/k8-opt/bin/pkg/sql/opt/norm/factory.og.go in pkg/sql/opt/norm.(*Factory).ConstructLeftJoin at line 4278
pkg/sql/opt/norm/reject_nulls_funcs.go in pkg/sql/opt/norm.(*CustomFuncs).RejectNullCols at line 25
pkg/sql/opt/norm/reject_nulls_funcs.go in pkg/sql/opt/norm.DeriveRejectNullCols at line 214
pkg/sql/opt/norm/reject_nulls_funcs.go in pkg/sql/opt/norm.deriveScanRejectNullCols at line 372
pkg/sql/opt/table_meta.go in pkg/sql/opt.(*TableMeta).PartialIndexPredicate at line 443

Tags

Tag Value
Command server
Environment v23.1.16
Go Version go1.19.13
Platform linux arm64
Distribution CCL
Cockroach Release v23.1.16
Cockroach SHA 37f7574
# of CPUs 16
# of Goroutines 896

Jira issue: CRDB-38789

@cockroach-sentry cockroach-sentry added O-sentry Originated from an in-the-wild panic report. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels May 15, 2024
@yuzefovich
Copy link
Member

dup of #118913

@exalate-issue-sync exalate-issue-sync bot changed the title Sentry: table_meta.go:443: partial index predicate does not exist in table metadata (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/sql... Sentry: table_meta.go:443: partial index predicate does not exist in table metadata (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/sql... May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.
Projects
None yet
Development

No branches or pull requests

2 participants