Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CALCITE-4510] RexLiteral can produce wrong digest for some user defined types #2356

Merged
merged 1 commit into from Apr 30, 2021

Conversation

liyafan82
Copy link
Contributor

We find weird literals for some user defined non-nullable types. Some investigation shows that the problem lies in the RexLiteral#toJavaString method.

In particular, it checks the type string suffix with an 8-character string:

if (!fullTypeString.endsWith("NOT NULL")) {

However, it trims the last 9 characters from the end of the string:

sb.append(fullTypeString, 0, fullTypeString.length() - 9);

@liyafan82 liyafan82 force-pushed the fly_0223_dg branch 4 times, most recently from 61396dd to 3ce0f1f Compare February 26, 2021 10:01
@liyafan82 liyafan82 changed the title [CALCITE-4510] Weird digests for literals with some user defined types [CALCITE-4510] RexLiteral can produce wrong digest for some user defined types Feb 26, 2021
@rubenada
Copy link
Contributor

LGTM

/**
* Suffix for the digests of non-nullable types.
*/
String NON_NULLABLE_TYPE_DIGEST_SUFFIX = " NOT NULL";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Frankly speaking, I do not like how this leaks to the public API.
Could you declare the constant in a Calcite-private location to prevent it accidental use in the client code?

Copy link
Contributor

@danny0405 danny0405 Feb 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me too, i prefer the string constant " NOT NULL" directly because it is straight-forward.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand @vlsi 's concern. I guess it would be better to define this constant e.g. in RelDataTypeImpl rather than RelDataType.
@danny0405 I think it is cleaner to "centralize" this constant in one single place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vlsi 's concern sounds reasonable, so I have moved the literal to class RelDataTypeImpl according to @rubenada 's suggestion.

IMO, extracting a constant to a static final field makes the code more structured and maintainable, and it also helps avoiding some bugs and typos (like the one in this issue).

@vlsi
Copy link
Contributor

vlsi commented Feb 26, 2021

Can you please add the test to verify the change?

@liyafan82
Copy link
Contributor Author

Can you please add the test to verify the change?

Sure. I have added one to emulate the user defined type that has revealed the problem.

@rubenada
Copy link
Contributor

rubenada commented Mar 1, 2021

Sure. I have added one to emulate the user defined type that has revealed the problem.

@liyafan82 have you pushed your latest changes? I cannot see the test

@liyafan82
Copy link
Contributor Author

Sure. I have added one to emulate the user defined type that has revealed the problem.

@liyafan82 have you pushed your latest changes? I cannot see the test

@rubenada Pushed just now. Please forgive me for my carelessness.

* Suffix for the digests of non-nullable types.
*/
public static final String NON_NULLABLE_TYPE_DIGEST_SUFFIX = " NOT NULL";

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any special reason that the variable NON_NULLABLE_TYPE_DIGEST_SUFFIX belongs to this class and can we give it a shorter name ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any special reason that the variable NON_NULLABLE_TYPE_DIGEST_SUFFIX belongs to this class and can we give it a shorter name ?

Sure. I have refactored the code to give it a shorter name.

I think the reason to place it in this class is that, it defines the common suffix for all Calcite internal types, and this class is the common base class for all Calcite internal types.

One alternative that I can think of is to put it into the RelDataType interface, and annotate it as an internal API.
However, I personally prefer to place it in class RelDataTypeImpl, as in this PR.

WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine with the change.

@liyafan82
Copy link
Contributor Author

Many thanks to all reviewers for the feedback.
If there are no objections, I want to merge this PR in a few days.

@liyafan82
Copy link
Contributor Author

Merging. Thanks.

@liyafan82 liyafan82 merged commit 350802b into apache:master Apr 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LGTM-will-merge-soon Overall PR looks OK. Only minor things left.
Projects
None yet
4 participants