New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Commitlog handling #1710
Comments
It seems to me that this can and should be implemented in an external library, not part of the core gocql. Each CQL database implementation (Cassandra, Scylla, ...) has a different CDC implementation. For example Scylla has https://github.com/scylladb/scylla-cdc-go. |
@martin-sucha yes, I'm well aware. The exact Cassandra CDC implementation is on developers shoulders, that's why we have things like debezium. There's literally no golang libs for CommitLog codding, and the exact CommitLog operation is heavily dependent on the number of available replicas and the client implementation itself - we can't stream and replay what's haven't been sync'ed yet, across all the hosts. In that regard there's nothing available for golang, yet. |
My aim is to stream Cassandra CDC to Knative Eventing CloudEvent channel with a custom Apache Arrow marshaling and processing (flatbuffers IPC easier to optimize). And I'd like to do that with zero additional hops and with zero-alloc golang. Had considered rust, as well. |
How is gocql related to that? |
@martin-sucha I'd like to do that in golang, reusing the existing gocql client with CommitLog sync bindings. |
Please answer these questions before submitting your issue. Thanks!
What version of Cassandra are you using?
4.1
What version of Gocql are you using?
1.5.2
What version of Go are you using?
1.20
What did you do?
Streaming cassandra commitlogs with common Java reader/replayer, similar to cassandra-commitlog-extract.
What did you expect to see?
It would be really nice if gocql could parse Cassandra Commitlogs, so people could use golang for Cassandra CDC streaming.
What did you see instead?
That there's no Commitlog support in gocql.
The text was updated successfully, but these errors were encountered: