Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug][Spark-Translation] Fix a bug of Spark Translation when has a timestamp/date type filed in source #4226

Merged
merged 8 commits into from
Mar 9, 2023

Conversation

rewerma
Copy link
Contributor

@rewerma rewerma commented Feb 26, 2023

Purpose of this pull request

Check list

TyrantLucifer
TyrantLucifer previously approved these changes Feb 26, 2023
Copy link
Member

@TyrantLucifer TyrantLucifer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TyrantLucifer
TyrantLucifer previously approved these changes Feb 26, 2023
Copy link
Member

@TyrantLucifer TyrantLucifer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good job. cc @Hisoka-X @EricJoy2048

TaoZex
TaoZex previously approved these changes Feb 26, 2023
Copy link
Contributor

@TaoZex TaoZex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@Hisoka-X
Copy link
Member

Hi, can you add e2e to test this case? Thanks.

@Hisoka-X
Copy link
Member

Or modify e2e already existed.

Hi, can you add e2e to test this case? Thanks.

@TyrantLucifer
Copy link
Member

Hi, can you add e2e to test this case? Thanks.

This case I had help him to test in local env. The main problem is that when we invoke createDataFrame method manully spark will convert data timestamp to java.sql.Date java.sql.Timestamp. If user use transfrom-v2 in spark engine it will have some problems.

@rewerma
Copy link
Contributor Author

rewerma commented Feb 27, 2023

Or modify e2e already existed.

Hi, can you add e2e to test this case? Thanks.

ok

@TyrantLucifer TyrantLucifer added this to the 2.3.1 milestone Feb 27, 2023
Copy link
Member

@TyrantLucifer TyrantLucifer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TyrantLucifer TyrantLucifer merged commit 5aa7cdb into apache:dev Mar 9, 2023
@rewerma rewerma deleted the hotfix/spark-time-type branch March 25, 2023 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants