runtime.boundserror in line 48, in (*memUvarintReader).ReadUvarint

[agnivade]

We just received such a crash in Sentry. Bleve version used is 2.3.0

runtime.boundsError: runtime error: index out of range [2626] with length 2626
  File "github.com/blevesearch/zapx/v15@v15.3.2/memuvarint.go", line 48, in (*memUvarintReader).ReadUvarint
  File "github.com/blevesearch/zapx/v15@v15.3.2/intDecoder.go", line 103, in (*chunkedIntDecoder).readUvarint
  File "github.com/blevesearch/zapx/v15@v15.3.2/posting.go", line 381, in (*PostingsIterator).skipFreqNormReadHasLocs
  File "github.com/blevesearch/zapx/v15@v15.3.2/posting.go", line 732, in (*PostingsIterator).currChunkNext
  File "github.com/blevesearch/zapx/v15@v15.3.2/posting.go", line 594, in (*PostingsIterator).nextDocNumAtOrAfter
  File "github.com/blevesearch/zapx/v15@v15.3.2/posting.go", line 476, in (*PostingsIterator).nextAtOrAfter
  File "github.com/blevesearch/zapx/v15@v15.3.2/posting.go", line 465, in (*PostingsIterator).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/index/scorch/snapshot_index_tfr.go", line 79, in (*IndexSnapshotTermFieldReader).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_term.go", line 88, in (*TermSearcher).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_conjunction.go", line 216, in (*ConjunctionSearcher).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_conjunction.go", line 216, in (*ConjunctionSearcher).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_boolean.go", line 164, in (*BooleanSearcher).advanceNextMust
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_boolean.go", line 315, in (*BooleanSearcher).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_boolean.go", line 395, in (*BooleanSearcher).Advance
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_conjunction.go", line 251, in (*ConjunctionSearcher).advanceChild
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_conjunction.go", line 188, in (*ConjunctionSearcher).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_boolean.go", line 164, in (*BooleanSearcher).advanceNextMust
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/searcher/search_boolean.go", line 315, in (*BooleanSearcher).Next
  File "github.com/blevesearch/bleve/v2@v2.3.0/search/collector/topn.go", line 226, in (*TopNCollector).Collect
  File "github.com/blevesearch/bleve/v2@v2.3.0/index_impl.go", line 531, in (*indexImpl).SearchInContext
  File "github.com/blevesearch/bleve/v2@v2.3.0/index_impl.go", line 368, in (*indexImpl).Search
  File "github.com/mattermost/mattermost-server/v6/services/searchengine/bleveengine/search.go", line 215, in (*BleveEngine).SearchPosts
  File "github.com/mattermost/mattermost-server/v6/store/searchlayer/post_layer.go", line 150, in SearchPostStore.searchPostsForUserByEngine
  File "github.com/mattermost/mattermost-server/v6/store/searchlayer/post_layer.go", line 176, in SearchPostStore.SearchPostsForUser
  File "github.com/mattermost/mattermost-server/v6/store/timerlayer/timerlayer.go", line 5295, in (*TimerLayerPostStore).SearchPostsForUser
  File "github.com/mattermost/mattermost-server/v6/app/post.go", line 1322, in (*App).SearchPostsForUser
  File "github.com/mattermost/mattermost-server/v6/api4/post.go", line 602, in searchPosts
  File "github.com/mattermost/mattermost-server/v6/api4/post.go", line 555, in searchPostsInTeam
  File "github.com/mattermost/mattermost-server/v6/web/handlers.go", line 339, in Handler.ServeHTTP
  File "github.com/mattermost/gziphandler@v0.0.1/gzip.go", line 337, in GzipHandlerWithOpts.func1.1
  File "net/http/server.go", line 2049, in HandlerFunc.ServeHTTP
  File "github.com/gorilla/mux@v1.8.0/mux.go", line 210, in (*Router).ServeHTTP
  File "net/http/server.go", line 2049, in HandlerFunc.ServeHTTP
  File "net/http/server.go", line 2867, in serverHandler.ServeHTTP
  File "net/http/server.go", line 1932, in (*conn).serve

I hope the stack trace is enough to identify the cause here. Unfortunately, I don't have anything more than this since this just comes from Sentry.

[abhinavdangeti]

@agnivade A unit test trying to reproduce the sequence of steps that led you to this crash would be of great help here. If not, at least some context into how you're using bleve might prove helpful.

[agnivade]

@abhinavdangeti - Unfortunately, I don't have a unit test because this is just a crash reported in Sentry. I can help with the second answer though. Mattermost uses Bleve as a search layer to search users, channels, posts. You can see our code here: https://github.com/mattermost/mattermost-server/tree/master/services/searchengine/bleveengine

The entry point to this crash starts from this file: github.com/mattermost/mattermost-server/v6/services/searchengine/bleveengine/search.go.

Hopefully that helps.

[abhinavdangeti]

Made some fixes within blevesearch/zapx and this change upgrades the version(s) of zapx used by bleve ..
#1655

[agnivade]

Thanks!