spanconfig: v23.1.12: nil pointer dereference in findNextOverlap

[cockroach-sentry]

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/5047259415/?referrer=webhooks_plugin

Panic Message:

panic.go:884: runtime error: invalid memory address or nil pointer dereference
(1) attached stack trace
  -- stack trace:
  | runtime.gopanic
  | 	GOROOT/src/runtime/panic.go:884
  | runtime.panicmem
  | 	GOROOT/src/runtime/panic.go:260
  | runtime.sigpanic
  | 	GOROOT/src/runtime/signal_unix.go:839
  | github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore.(*iterator).findNextOverlap
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/spanconfig/spanconfigstore/entry_interval_btree.go:1109
  | github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore.(*iterator).FirstOverlap
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/spanconfig/spanconfigstore/entry_interval_btree.go:1071
  | github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore.(*spanConfigStore).computeSplitKey
  | 	github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore/span_store.go:118
  | github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore.(*Store).ComputeSplitKey
  | 	github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore/store.go:130
  | github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore.(*Store).NeedsSplit
  | 	github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigstore/store.go:117
  | github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigkvsubscriber.(*KVSubscriber).NeedsSplit
  | 	github.com/cockroachdb/cockroach/pkg/spanconfig/spanconfigkvsubscriber/kvsubscriber.go:343
  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).maybeAdd
  | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:672
  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.baseQueueHelper.MaybeAdd
  | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:573
  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).MaybeAddAsync.func1
  | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:625
  | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).Async.func1
  | 	github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:612
  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470
  | runtime.goexit
  | 	GOROOT/src/runtime/asm_amd64.s:1594
Wraps: (2) runtime error: invalid memory address or nil pointer dereference
Error types: (1) *withstack.withStack (2) runtime.errorString
-- report composition:
runtime.errorString
panic.go:884: *withstack.withStack (top exception)

Stacktrace (expand for inline code snippets):

GOROOT/src/runtime/asm_amd64.s#L1593-L1595

cockroach/pkg/util/stop/stopper.go

Lines 469 to 471 in d7e9824

	sp.UpdateGoroutineIDToCurrent()
	f(ctx)
	}()

https://github.com/cockroachdb/cockroach/blob/d7e9824b4cd6ebf7a8548156f2a772ae6648257d/pkg/kv/kvserver/pkg/kv/kvserver/queue.go#L611-L613
https://github.com/cockroachdb/cockroach/blob/d7e9824b4cd6ebf7a8548156f2a772ae6648257d/pkg/kv/kvserver/pkg/kv/kvserver/queue.go#L624-L626
https://github.com/cockroachdb/cockroach/blob/d7e9824b4cd6ebf7a8548156f2a772ae6648257d/pkg/kv/kvserver/pkg/kv/kvserver/queue.go#L572-L574
https://github.com/cockroachdb/cockroach/blob/d7e9824b4cd6ebf7a8548156f2a772ae6648257d/pkg/kv/kvserver/pkg/kv/kvserver/queue.go#L671-L673

cockroach/pkg/spanconfig/spanconfigkvsubscriber/kvsubscriber.go

Lines 342 to 344 in d7e9824

	return s.mu.internal.NeedsSplit(ctx, start, end)
	}

cockroach/pkg/spanconfig/spanconfigstore/store.go

Lines 116 to 118 in d7e9824

	func (s *Store) NeedsSplit(ctx context.Context, start, end roachpb.RKey) (bool, error) {
	splits, err := s.ComputeSplitKey(ctx, start, end)
	if err != nil {

cockroach/pkg/spanconfig/spanconfigstore/store.go

Lines 129 to 131 in d7e9824

	return s.mu.spanConfigStore.computeSplitKey(start, end)
	}

cockroach/pkg/spanconfig/spanconfigstore/span_store.go

Lines 117 to 119 in d7e9824

	iter, query := s.btree.MakeIter(), makeQueryEntry(sp)
	for iter.FirstOverlap(query); iter.Valid(); iter.NextOverlap(query) {
	interned := iter.Cur().spanConfigPairInterned

https://github.com/cockroachdb/cockroach/blob/d7e9824b4cd6ebf7a8548156f2a772ae6648257d/bazel-out/k8-opt/bin/pkg/spanconfig/spanconfigstore/entry_interval_btree.go#L1070-L1072
https://github.com/cockroachdb/cockroach/blob/d7e9824b4cd6ebf7a8548156f2a772ae6648257d/bazel-out/k8-opt/bin/pkg/spanconfig/spanconfigstore/entry_interval_btree.go#L1108-L1110
GOROOT/src/runtime/signal_unix.go#L838-L840
GOROOT/src/runtime/panic.go#L259-L261
GOROOT/src/runtime/panic.go#L883-L885

GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 470
pkg/kv/kvserver/pkg/kv/kvserver/queue.go in pkg/kv/kvserver.(*baseQueue).Async.func1 at line 612
pkg/kv/kvserver/pkg/kv/kvserver/queue.go in pkg/kv/kvserver.(*baseQueue).MaybeAddAsync.func1 at line 625
pkg/kv/kvserver/pkg/kv/kvserver/queue.go in pkg/kv/kvserver.baseQueueHelper.MaybeAdd at line 573
pkg/kv/kvserver/pkg/kv/kvserver/queue.go in pkg/kv/kvserver.(*baseQueue).maybeAdd at line 672
pkg/spanconfig/spanconfigkvsubscriber/kvsubscriber.go in pkg/spanconfig/spanconfigkvsubscriber.(*KVSubscriber).NeedsSplit at line 343
pkg/spanconfig/spanconfigstore/store.go in pkg/spanconfig/spanconfigstore.(*Store).NeedsSplit at line 117
pkg/spanconfig/spanconfigstore/store.go in pkg/spanconfig/spanconfigstore.(*Store).ComputeSplitKey at line 130
pkg/spanconfig/spanconfigstore/span_store.go in pkg/spanconfig/spanconfigstore.(*spanConfigStore).computeSplitKey at line 118
bazel-out/k8-opt/bin/pkg/spanconfig/spanconfigstore/entry_interval_btree.go in pkg/spanconfig/spanconfigstore.(*iterator).FirstOverlap at line 1071
bazel-out/k8-opt/bin/pkg/spanconfig/spanconfigstore/entry_interval_btree.go in pkg/spanconfig/spanconfigstore.(*iterator).findNextOverlap at line 1109
GOROOT/src/runtime/signal_unix.go in runtime.sigpanic at line 839
GOROOT/src/runtime/panic.go in runtime.panicmem at line 260
GOROOT/src/runtime/panic.go in runtime.gopanic at line 884

Tags

Tag	Value
Command	start-single-node
Environment	v23.1.12
Go Version	go1.19.13
Platform	linux amd64
Distribution	CCL
Cockroach Release	v23.1.12
Cockroach SHA	d7e9824
# of CPUs	8
# of Goroutines	365

Jira issue: CRDB-36499