Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for User Defined Types (Issue #98, #105) #113

Merged
merged 2 commits into from
Nov 23, 2020

Conversation

cvybhu
Copy link
Contributor

@cvybhu cvybhu commented Nov 17, 2020

This PR implements support for using User Defined Types that can be used like this:

// CREATE TYPE IF NOT EXISTS ks.my_type (int_val int, text_val text)

// Define custom struct that matches User Defined Type created earlier
// wrapping field in Option will gracefully handle null field values
#[derive(Debug, IntoUserType, FromUserType)]
struct MyType {
    int_val: i32,
    text_val: Option<String>,
}

let to_insert = MyType {
    int_val: 17,
    text_val: Some("Some string".to_string()),
};

// It can be inserted like a normal value
session
    .query(
        "INSERT INTO ks.udt_tab (k, my) VALUES (5, ?)",
        &(try_values!(to_insert)?),
    )
    .await?;

// And read like any normal value
if let Some(rows) = session.query("SELECT my FROM ks.udt_tab", &[]).await? {
    for row in rows.into_typed::<(MyType,)>() {
        let (my_val,) = row?;
        println!("{:?}", my_val)
    }
}

Fixes #105
Fixes #98

The main challenge introduced by UDT is that creating a Value from user struct can fail if the resulting value would be bigger than 2GiB. This PR deals with this by implementing TryIntoValue for user structs with corresponding try_values!.
Maybe the values! API could be kept if we changed public query APIs to take ValueIn instead of Value where ValueIn would be something like:

pub enum ValueIn {
    Value(value::Value),
    Invalid
}

and we would return an error when sending a request with Invalid values.
But that would introduce more complexity in query code, maybe we could wait with such refactor until implementing named values as they are gonna require a redesign anyway?

Another issue is that currently serializing struct into value is not very efficient - fields get recursively converted into values with one allocation for every field. This could be avoided by adding a serialization trait that would write each type into BufMut as binary Value. But adding such trait would make current Value struct not needed - we could just have Bytes and value would be just Bytes created using serialization trait.

This PR implements UDTs using current Value api, I'm not sure if full redesign of Value api should be included in this, maybe we could create a separate issue for this? Also there might be additional issues that are gonna come to light when implementing the rest of CQL types, so maybe we could wait to get the full picture and then start refactoring.

EDIT:
I have moved refactoring Value api into #114

@cvybhu cvybhu mentioned this pull request Nov 17, 2020
Copy link
Collaborator

@piodul piodul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, left some small nits. It's sad that serializing values involves so much copies and allocations now, but I agree that improving this situation is a task for another PR.

fn try_into_value(self) -> Result<Value, ValueTooBig>;
}

pub struct ValueTooBig {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: let's change it to a unit struct:

pub struct ValueTooBig;

This way you don't have to type curly braces after the type name when you want to instantiate it:

let err = ValueTooBig;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 👍
Cool I didn't know that was possible

};

TokenStream::from(generated)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, please put refactors such as moving parts of code into a separate method into separate commits. It makes reviewing easier.

@cvybhu
Copy link
Contributor Author

cvybhu commented Nov 18, 2020

Okay done

@cvybhu
Copy link
Contributor Author

cvybhu commented Nov 22, 2020

@piodul After we merge this I'm going to open a pull request for Value refactor (#114), which builds on code from this pr

Copy link
Collaborator

@piodul piodul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. LGTM.

@piodul piodul merged commit ceafc38 into scylladb:main Nov 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support user-defined types (UDT) Procedural macros for UDT serialization
2 participants