Skip to content

JDBC sources: Fix unexcepcted long Integer value failure - #3846

Merged
Andrii Leonets (DoNotPanicUA) merged 4 commits into
masterfrom
3840-fix-mysql
Jun 4, 2021
Merged

JDBC sources: Fix unexcepcted long Integer value failure#3846
Andrii Leonets (DoNotPanicUA) merged 4 commits into
masterfrom
3840-fix-mysql

Conversation

@DoNotPanicUA

@DoNotPanicUA Andrii Leonets (DoNotPanicUA) commented Jun 3, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor

Issue: #3840

What

Fix an error when JDBC source produces Integer value bigger than max Java Integer value.

How

Implement handler which tries to convert into Integer first. My assumption is that at most cases the value will be in the range.
If it fails, the handler tries to convert into the Long type. In all other negative cases it will replace value by null.

Pre-merge Checklist

  • Run integration tests
  • Publish Docker images

Recommended reading order

  1. JdbcUtils.java

@DoNotPanicUA
implement additional handler for the Integer source values in order t…
bfd05f6 
…o cover the case with too large value (possible for some sources)
@DoNotPanicUA
format
a218b46
@DoNotPanicUA Andrii Leonets (DoNotPanicUA) linked an issue Jun 3, 2021 that may be closed by this pull request
Source MySQL: unsigned numeric values larger than signed integer size break connector #3840
Closed
@DoNotPanicUA
Andrii Leonets (DoNotPanicUA) marked this pull request as ready for review June 3, 2021 18:49
sherifnada
Sherif A. Nada (sherifnada) approved these changes Jun 3, 2021
View reviewed changes

@sherifnada Sherif A. Nada (sherifnada) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- please publish and merge. Also, can you add this to the data type tests?

BTW see the definition-of-done checklist here to help clarify what next steps would be: #3864

@sherifnada

Sherif A. Nada (sherifnada) commented Jun 3, 2021

Copy link
Copy Markdown
Contributor

Andrii Leonets (@DoNotPanicUA) also can you update the PR title to match best practices for PR titles?

@DoNotPanicUA Andrii Leonets (DoNotPanicUA) changed the title 3840 fix mysql JDBC sources: Fix unexcepcted long Integer value failure Jun 4, 2021
@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-mysql

🕑 source-mysql https://github.com/airbytehq/airbyte/actions/runs/905415719
✅ source-mysql https://github.com/airbytehq/airbyte/actions/runs/905415719

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-jdbc

🕑 source-jdbc https://github.com/airbytehq/airbyte/actions/runs/905437299
✅ source-jdbc https://github.com/airbytehq/airbyte/actions/runs/905437299

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-mssql

🕑 source-mssql https://github.com/airbytehq/airbyte/actions/runs/905438603
✅ source-mssql https://github.com/airbytehq/airbyte/actions/runs/905438603

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-oracle

🕑 source-oracle https://github.com/airbytehq/airbyte/actions/runs/905439792
✅ source-oracle https://github.com/airbytehq/airbyte/actions/runs/905439792

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-postgres

🕑 source-postgres https://github.com/airbytehq/airbyte/actions/runs/905440971
✅ source-postgres https://github.com/airbytehq/airbyte/actions/runs/905440971

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-redshift

🕑 source-redshift https://github.com/airbytehq/airbyte/actions/runs/905442071
✅ source-redshift https://github.com/airbytehq/airbyte/actions/runs/905442071

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=source-clickhouse

🕑 source-clickhouse https://github.com/airbytehq/airbyte/actions/runs/905443231
✅ source-clickhouse https://github.com/airbytehq/airbyte/actions/runs/905443231

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-snowflake

🕑 destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/905488975
✅ destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/905488975

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-jdbc

🕑 destination-jdbc https://github.com/airbytehq/airbyte/actions/runs/905491481
✅ destination-jdbc https://github.com/airbytehq/airbyte/actions/runs/905491481

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-mysql

🕑 destination-mysql https://github.com/airbytehq/airbyte/actions/runs/905493539
✅ destination-mysql https://github.com/airbytehq/airbyte/actions/runs/905493539

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-mssql

🕑 destination-mssql https://github.com/airbytehq/airbyte/actions/runs/905494132
✅ destination-mssql https://github.com/airbytehq/airbyte/actions/runs/905494132

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-postgres

🕑 destination-postgres https://github.com/airbytehq/airbyte/actions/runs/905495886
✅ destination-postgres https://github.com/airbytehq/airbyte/actions/runs/905495886

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-redshift

🕑 destination-redshift https://github.com/airbytehq/airbyte/actions/runs/905497914
✅ destination-redshift https://github.com/airbytehq/airbyte/actions/runs/905497914

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/test connector=destination-snowflake

🕑 destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/905498232
✅ destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/905498232

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021

Copy link
Copy Markdown
Contributor Author

LGTM -- please publish and merge. Also, can you add this to the data type tests?

This case is added to the PR with tests. Corresponding commit.

@DoNotPanicUA

Andrii Leonets (DoNotPanicUA) commented Jun 4, 2021 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor Author

/publish connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/905800611
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/905800611

@DoNotPanicUA
Andrii Leonets (DoNotPanicUA) deleted the 3840-fix-mysql branch June 4, 2021 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@subodh1810 Subodh Chaturvedi (subodh1810) Awaiting requested review from subodh1810
@tuliren Liren Tu (tuliren) Awaiting requested review from tuliren
3 more reviewers
@sherifnada Sherif A. Nada (sherifnada) sherifnada approved these changes
@masonwheeler masonwheeler masonwheeler approved these changes
@yaroslav-hrytsaienko Yaroslav Hrytsaienko (yaroslav-hrytsaienko) yaroslav-hrytsaienko approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@DoNotPanicUA Andrii Leonets (DoNotPanicUA)

Labels

connectors/destination/snowflake connectors/destinations-warehouse

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Source MySQL: unsigned numeric values larger than signed integer size break connector

5 participants

@DoNotPanicUA @sherifnada @masonwheeler @yaroslav-hrytsaienko @karinakuz