-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make counters work with CDC #6013
Comments
I think that representing counter as a map<shard_id, int64> would be best. This way we would be able to replicate counter value to another Scylla instance. This map should be non-frozen and each counter shard leader should write only updates to its own shard. |
Is it possible to reliably log the actual delta (difference between counter values caused by an update)?
|
Let's first explore other options. |
Seems doable to me - we can try augmenting counter update mutations. The cdc-deltas would be propagated alongside corresponding counter-shards-mutations, as reliably as the latter do it now..? ATM I'm investigating how C* solved it (they have something). In the meantime we can think on how to do it perfectly. |
I don't think its a good idea to log the delta - AFAIK thats how counters in Cassandra started and they switched to the current method. Counters AFIAK do enforce RMW for their operation - which means you have a pre-image fetched and a post-image computed as well (since this is what we write at the end to the sstables/memtables). Given that counters enforce a RMW for their operation - I think we should log in their case the full value always (even for the change) as a map<share,value> instead of only the delta (we do have a CQL extension to allow writing this as well). |
It seems that we can move forward with map<shard_id, value> idea, no? |
Just some indirect evidence.
Given that C*'s CDC has the logic of commitlog, we can speculate that the local counter-shards are being logged. So the map approach looks like the way to go. |
I'm still unsure if it plays well with existing code...
a. b. Injected? Computing counter-shards on leader is integrated with applying this change to DB. Docs even say that this happens under a lock. That means we'll have to fetch What can we do?
|
A table with counters cannot have CDC enabled:
This happens for a purely technical reason: counter columns are territorial animals and don't allow anyone else into CF, apart from PK and CK (and other counters). If we try to enable CDC for a table-with-counters, CDC tries to create "ordinary" columns like
ttl
,cdc$deleted*
which are in conflict with counter column(s). The CDC column type (here:counter_type
) is inferred from the base table, hence the error.I have a quickpatch with sum-of-local-counter-shards being cast to bigint, but this would not be reliable - please imagine a situation where two counter-leaders concurrently update their own counter-shards - what CDC should both of them log? Should they reconcile their CDC writes? I don't know, my proposals are:
I'm not sure that clients want to be bothered with "counter shards". I'm even less sure about reconciliation of CDC logs for counter writes.
Preimage/postimage? even more issues may arise.
Please, comment @avikivity @slivne @haaawk @elcallio @kbr- @gleb-cloudius
The text was updated successfully, but these errors were encountered: