use RWMutex replace Mutex in segment

[hengfeiyang]

This will be slight, but it would cause problem when load many many segments, need a mechanism to release unused cache.

[mschoch]

I have 2 concerns with this approach:

1. We reuse the buffer, but we keep re-decomprressing the same bytes over and over again. For example, if I have match 1000 documents and they all are in the same chunk in the same segment. To simply load the _id field of each, we decompress that same compressed stored doc chunk 1000 times. Do you agree this will happen and that it is bad?
2. We now have additional lock contention, and all stored field access will have a bottleneck on this single shared buffer.

I don't see how this can work either.

[mschoch]

To me, compression chunks of stored documents is great for compression, but terrible for our existing search API, and without changes to that, it cannot perform well.

[hengfeiyang]

We can reduce the scale of lock, when load data we can initial many mutex equal to thunk num, but we don't cache uncompress data. We still cache uncompress data when first use it.

After then, we split the lock contention to 1/N (n=thunkNum)

Fake code:

decompressedStoredFieldChunks map[uint64]cacheData

type cacheData struct{
    data []byte
    m      sync.RWMutex
}

[mschoch]

I guess I misunderstood what you were doing here.

Don't you still think this is going to cache too much data?

[hengfeiyang]

It going will be to add a cache manager to keep memory usage at a proper size.

[mschoch]

It going will be to add a cache manager to keep memory size at a proper usage.

You're saying in order for this to work well, we need some other component that doesn't exist yet?

[hengfeiyang]

Yes, you are right, let me think more, try to find other solution.

[mschoch]

I don't think we ever actually succeed in updating items inside the map. Although you mutate storedFieldDecompressed I believe this is a copy, not the item the map. See this example I think is analagous: https://go.dev/play/p/s0UrS7nD_fg

[hengfeiyang]

Yeah, you are right, i changed it. 34f03d6