Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pulsar SQL support for Decimal data type #15153

Merged
merged 6 commits into from Apr 21, 2022
Merged

Pulsar SQL support for Decimal data type #15153

merged 6 commits into from Apr 21, 2022

Conversation

shibd
Copy link
Member

@shibd shibd commented Apr 13, 2022 •
edited

Motivation

Current Pulsar SQL does not support the Decimal data type, Trino does specify the Decimal as one of the supported data types. Avro also has specifications to support it.

Modifications

  • Pulsar SQL support for Decimal data type(AVRO Schema).
  • Add unit test.
  • In the current implementation, since JsonSchema is generated by Avro, there may exist LogicalTypes.Decimal. So Mapping decimalType with varcharType in JsonSchema.

Documentation

Need to update docs?

  • no-need-doc

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Apr 13, 2022
@RobertIndie RobertIndie added type/feature The PR added a new feature or issue requested a new feature area/sql Pulsar SQL related features labels Apr 13, 2022
@RobertIndie RobertIndie added this to the 2.11.0 milestone Apr 13, 2022
@shibd
Copy link
Member Author

shibd commented Apr 13, 2022

/pulsarbot run-failure-checks

@shibd shibd changed the title Pulsar SQL support for Decimal data type(AVRO Schema) WIP: Pulsar SQL support for Decimal data type(AVRO Schema) Apr 13, 2022
@shibd shibd changed the title WIP: Pulsar SQL support for Decimal data type(AVRO Schema) Pulsar SQL support for Decimal data type(AVRO Schema) Apr 13, 2022
@shibd
Copy link
Member Author

shibd commented Apr 14, 2022

/pulsarbot run-failure-checks

@shibd shibd changed the title Pulsar SQL support for Decimal data type(AVRO Schema) Pulsar SQL support for Decimal data type Apr 14, 2022
@shibd
Copy link
Member Author

shibd commented Apr 14, 2022

@mattisonchao @codelipenghui @gaoran10 @Technoboy- PTAL, Thanks.

@lhotari
Copy link
Member

lhotari commented Apr 14, 2022

/pulsarbot run-failure-checks

1 similar comment
@shibd
Copy link
Member Author

shibd commented Apr 15, 2022

/pulsarbot run-failure-checks

codelipenghui
codelipenghui reviewed Apr 18, 2022
View reviewed changes
Copy link
Contributor

@codelipenghui codelipenghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shibd I noticed we have a test for BigDecimal https://github.com/apache/pulsar/blob/master/tests/integration/src/test/java/org/apache/pulsar/tests/integration/schema/Schemas.java#L99, could you please help check why the test works?

@shibd
Copy link
Member Author

shibd commented Apr 18, 2022

@shibd I noticed we have a test for BigDecimal https://github.com/apache/pulsar/blob/master/tests/integration/src/test/java/org/apache/pulsar/tests/integration/schema/Schemas.java#L99, could you please help check why the test works?

@codelipenghui Hi, This test is test pulsar schema. It has nothing to do with pulsar SQL.

@codelipenghui
Copy link
Contributor

@codelipenghui Hi, This test is test pulsar schema. It has nothing to do with pulsar SQL.

Oh, I see. Thanks for the clarification.

@codelipenghui codelipenghui merged commit 6b004ed into apache:master Apr 21, 2022
codelipenghui pushed a commit that referenced this pull request Apr 28, 2022
@shibd @codelipenghui
Pulsar SQL support for Decimal data type (#15153)
642159c 
(cherry picked from commit 6b004ed)
codelipenghui pushed a commit that referenced this pull request Apr 29, 2022
@shibd @codelipenghui
Pulsar SQL support for Decimal data type (#15153)
91a9e0b 
(cherry picked from commit 6b004ed)
@codelipenghui codelipenghui added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Apr 29, 2022
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request May 9, 2022
@shibd @nicoloboschi
Pulsar SQL support for Decimal data type (apache#15153)
2482228 
(cherry picked from commit 6b004ed)
(cherry picked from commit 642159c)
codelipenghui pushed a commit that referenced this pull request May 20, 2022
@shibd @codelipenghui
Pulsar SQL support for Decimal data type (#15153)
28970b7 
(cherry picked from commit 6b004ed)
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label May 20, 2022
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request May 23, 2022
@shibd @nicoloboschi
Pulsar SQL support for Decimal data type (apache#15153)
5ae8b92 
(cherry picked from commit 6b004ed)
(cherry picked from commit 28970b7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@codelipenghui codelipenghui codelipenghui approved these changes

@RobertIndie RobertIndie RobertIndie approved these changes

@hangc0276 hangc0276 Awaiting requested review from hangc0276

@gaoran10 gaoran10 Awaiting requested review from gaoran10

Assignees

@shibd shibd

Labels
area/sql Pulsar SQL related features cherry-picked/branch-2.8 Archived: 2.8 is end of life cherry-picked/branch-2.9 Archived: 2.9 is end of life cherry-picked/branch-2.10 doc-not-needed Your PR changes do not impact docs release/2.9.3 release/2.10.1 type/feature The PR added a new feature or issue requested a new feature
Projects
None yet
Milestone
2.11.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@shibd @lhotari @codelipenghui @RobertIndie