Optimize memory usage of table.TableIndex

[jarifibrahim]

Badger keeps the index used by each table in memory

badger/table/table.go

Lines 94 to 102 in 62b7a10

	type Table struct {
	sync.Mutex
	fd *os.File // Own fd.
	tableSize int // Initialized in OpenTable, using fd.Stat().
	bfLock sync.Mutex
	blockIndex []*pb.BlockOffset
	ref int32 // For file garbage collection. Atomic.

As the number of SST grows, the number of in-memory index blocks grows. The memory usage grows linearly with the number of tables. We should be cache the index blocks in ristretto.

This issue was also seen on dgraph-io/dgraph#5361 (comment)

// HEAP //
File: dgraph
Type: inuse_space
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 44487.54MB, 99.36% of 44774.33MB total
Dropped 234 nodes (cum <= 223.87MB)
Showing top 10 nodes out of 60
      flat  flat%   sum%        cum   cum%
20127.60MB 44.95% 44.95% 20127.60MB 44.95%  github.com/dgraph-io/ristretto/z.(*Bloom).Size
15454.99MB 34.52% 79.47% 23157.80MB 51.72%  github.com/dgraph-io/badger/v2/pb.(*TableIndex).Unmarshal
 7702.81MB 17.20% 96.67%  7702.81MB 17.20%  github.com/dgraph-io/badger/v2/pb.(*BlockOffset).Unmarshal