Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cassandra 5 vector support #1676

Merged
merged 1 commit into from
Jul 15, 2024
Merged

Cassandra 5 vector support #1676

merged 1 commit into from
Jul 15, 2024

Conversation

rukai
Copy link
Member

@rukai rukai commented Jun 25, 2024

This PR fixes a bug causing shotover to error out when processing cassandra custom types.
In doing so we enable support for cassandra 5's new vector types.

The problem is that the cassandra-protocol crate will return an Error when it tries to parse a Custom cassandra value, this makes sense from the perspective of the crate since it does not know how to handle this type. If the user wants to use the custom type they should use a driver that knows about this type.
However from the shotover perspective its reasonable for the client and cassandra to use a custom type that shotover knows nothing about, in this case we should just pass on the bytes as is.

To achieve this we add an enum CustomOrStandardType which allows us to store either a Custom type (not supported by cassandra-protocol) or any other standard type supported by cassandra-protocol.
Long term I consider this a hack, but unless we have a push to improve cassandra performance I think we'll be living with this hack forever.

As demonstrated here the java driver only supports 4 data types for use with vectors. So even though cassandra returns results for the other data types that could in theory be interpreted by a driver, we dont test those since there's no way to do that through the java driver. Shotover doesn't parse these anyway so in reality the coverage isnt important.

Copy link

codspeed-hq bot commented Jun 25, 2024

CodSpeed Performance Report

Merging #1676 will not alter performance

Comparing rukai:vector_support (723447f) with main (aa0c320)

Summary

✅ 39 untouched benchmarks

@rukai rukai force-pushed the vector_support branch 2 times, most recently from 01f8a43 to b8fb024 Compare June 26, 2024 02:19
@rukai rukai force-pushed the vector_support branch 6 times, most recently from 41cdd46 to f56b357 Compare July 12, 2024 00:12
@rukai rukai marked this pull request as ready for review July 12, 2024 00:15
@rukai rukai merged commit ce4a153 into shotover:main Jul 15, 2024
41 checks passed
@rukai rukai mentioned this pull request Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants