Correct way to use otter to count stuff

[ribtoks]

Hi

Thank you again for developing otter!

Sorry, if this looks too trivial of a question, but I have some concerns with regards to otters architecture for the purpose of keeping counters (concerns how the architecture dictates the choice of APIs, not the concerns about architecture "problems").

Let's imagine I have a whatever-typed key (e.g. integer) and an uint32 counter (e.g. otter.Cache[int, *uint32] - pointer for atomic calls?). My sources of confusion are:

• do I understand correctly that whenever I Get() the value and increment it using atomic.AddUint32() from stdlib, it sort of breaks inner logic of expiration? (sort of an "invisible" write)
• I'm a bit confused with regards to batching read/write operations and read buffer "overflow". Does that mean that, when other code is writing and reading counts separately, it's possible for readers to observe not-entirely-final state of the counters due to batched writes? (yes, I saw that "buffers are drained after write" - one of the conditions - but I'm not sure how it affects my question)
• is the Compute() way - the only correct approach for this?

[maypok86]

Hello,

• Yes, if you are using ExpiryWriting or ExpiryCreating, the expiration time will not be updated when using Get. If you need to update it on every Get, you can use ExpiryAccessing or implement your own ExpiryCalculator. Essentially, you can think of Otter as a regular map on steroids. In this case, you're just getting a pointer from the map, but not updating the pointer (only the value it points to).
• No, key-value pairs are updated immediately (unlike in Ristretto), meaning the hash table guarantees synchronization of operations with entries. Batching is only needed to save time and avoid updating eviction/expiration policies because W-TinyLFU (eviction policy) and Timer Wheel (expiration policy) require an exclusive lock for the entire cache. Batching might cause issues during write operations (e.g., write operations might be processed out of order), but Otter solves this with a state machine in each node and other checks.
• Maybe :), I don't fully understand your use case.

Perhaps this will help. The company I work for uses a push-based metrics collector (Graphite), and we used to frequently have problems with the speed of sending counters (counters are updated very often, and trying to send data over the network every time is too expensive). In the end, we decided to build an aggregator for these counters that would update them locally and periodically send them to the server.

How can something like this be implemented? A map + mutex seems like bad solution because a counter is created only once (on first use), and all subsequent operations with it are reads. You could use sync.Map, but it has problems with constant, meaningless allocations. Otter allows you to make such an aggregator faster and without additional allocations. For example,

type Aggregator struct {
	counters otter.Cache[string, *int64]
	// counters sync.Map
}

// ...

func (a *Aggregator) Add(key string, value int64) {
    // create
    counter, _ := a.counters.ComputeIfAbsent(key, func() (newValue *int64, cancel bool) {
        return new(int64), false
    })
    atomic.AddInt64(counter, value)
}

// Alternative implementation with sync.Map (commented out):
// func (a *Aggregator) Add(key string, value int64) {
//    counter, _ := a.counters.LoadOrStore(key, new(int64))
//    atomic.AddInt64(counter.(*int64), value)
// }

[ribtoks]

Thank you very much for the explanations!