Bug: Potential SEGFAULT in LSM Store "replace" (object bucket)

[etiennedi]

[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x7cf59c]

goroutine 661 [running]:
github.com/semi-technologies/weaviate/adapters/repos/db/lsmkv.(*Store).Bucket(...)
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/lsmkv/store.go:42
github.com/semi-technologies/weaviate/adapters/repos/db.(*Shard).vectorByIndexID(0x4001214700, 0xee5310, 0x40001ba040, 0x0, 0x4000106b98, 0x1f188, 0x40b86235e0, 0x50, 0x40b8623400)
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/shard_read.go:145 +0x6c
github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw.(*shardedLockCache).get(0x40002d8cb0, 0xee5310, 0x40001ba040, 0x0, 0x4000106be8, 0x69ae24, 0x4000106c08, 0x844144d9, 0xb68ae0190c392fa8)
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw/vector_cache.go:67 +0x118
github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw.(*hnsw).reassignNeighborsOf(0x40001e07e0, 0x406e93bc80, 0x485f2eb5a394, 0x40012121e0)
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw/delete.go:185 +0x138
github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw.(*hnsw).CleanUpTombstonedNodes(0x40001e07e0, 0x4000106e20, 0x0)
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw/delete.go:124 +0x48
github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw.(*hnsw).registerTombstoneCleanup.func1(0x40001e07e0)
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw/startup.go:134 +0x98
created by github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw.(*hnsw).registerTombstoneCleanup
        /go/src/github.com/semi-technologies/weaviate/adapters/repos/db/vector/hnsw/startup.go:127 +0x50

Notes

• Based on the stack above it seems this is related to tombstone cleanup which means that frequent updates/deletes may be a way to reliably reproduce this
• It could be that the problem and required solution is exactly the same as that used in Bug: SEGFAULT when querying while importing #1837 where we saw this for LSM type "Map"
• Assuming the above assumption is correct, then it would be very likely that a similar bug also sits in the LSM type "Set", but we simply haven't discovered it yet, as right now we seem to have experience this for 2 out of 3 types

[vrukshal]

I want to work on this bug. can anyone guide me on how to find the code to work on?

[etiennedi]

Hey, @vrukshal. Thanks for your interest, we're already working on this one internally. Let me make that clear by adding assignees :-)