Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix JdbcSource handling of tables with same names in different schemas #1724

Merged
merged 3 commits into from
Jan 20, 2021
Merged

Fix JdbcSource handling of tables with same names in different schemas #1724

merged 3 commits into from
Jan 20, 2021

Conversation

cgardens
Copy link
Contributor

  • Closes Unable to exit onboarding - Stuck on "Set up connection" in  #1684

  • Previously the JdbcSource was combining the columns of any tables with the same name across different schemas into a single stream in the catalog.

  • This was caught because in those tables there were columns of the same name with different types which triggered a precondition to check for this.

  • The fix makes sure we group by both schema name and table name.

  • Adds test to the standard jdbc tests to catch this case.

  • This test does NOT run for mysql as, mysql has no concept of schemas.

Checklist

  • bump versions for all jdbc sources (will do once PR is approved but before merging)

@cgardens
Fix JdbcSource handling of tables with same names in different schemas
ec78ae4 
* Previously the JdbcSource was combining the columns of any tables with the same name across different schemas into a single stream in the catalog.

* This was caught because in those tables there were columns of the same name with different types which triggered a precondition to check for this.

* The fix makes sure we group by both schema name and table name.

* Adds test to the standard jdbc tests to catch this case.

* This test does NOT run for mysql as, mysql has no concept of schemas.
@cgardens cgardens force-pushed the cgardens/handle_tables_with_same_names_and_multiple_schemas branch from e90be05 to ec78ae4 Compare January 19, 2021 23:15
@cgardens cgardens mentioned this pull request Jan 19, 2021
Closed
@cgardens
Copy link
Contributor Author

cgardens commented Jan 20, 2021 •
edited by github-actions bot
Loading

/test connector=destination-postgres

🕑 destination-postgres https://github.com/airbytehq/airbyte/actions/runs/497529387
✅ destination-postgres https://github.com/airbytehq/airbyte/actions/runs/497529387

@cgardens
Copy link
Contributor Author

cgardens commented Jan 20, 2021 •
edited by github-actions bot
Loading

/test connector=source-postgres

🕑 source-postgres https://github.com/airbytehq/airbyte/actions/runs/497529558
✅ source-postgres https://github.com/airbytehq/airbyte/actions/runs/497529558

@cgardens
Copy link
Contributor Author

cgardens commented Jan 20, 2021 •
edited by github-actions bot
Loading

/test connector=source-mssql

🕑 source-mssql https://github.com/airbytehq/airbyte/actions/runs/497529672
✅ source-mssql https://github.com/airbytehq/airbyte/actions/runs/497529672

@cgardens
Copy link
Contributor Author

cgardens commented Jan 20, 2021 •
edited by github-actions bot
Loading

/test connector=source-mysql

🕑 source-mysql https://github.com/airbytehq/airbyte/actions/runs/497529826
✅ source-mysql https://github.com/airbytehq/airbyte/actions/runs/497529826

@cgardens
Copy link
Contributor Author

cgardens commented Jan 20, 2021 •
edited by github-actions bot
Loading

/test connector=source-redshift

🕑 source-redshift https://github.com/airbytehq/airbyte/actions/runs/497529879
✅ source-redshift https://github.com/airbytehq/airbyte/actions/runs/497529879

@cgardens cgardens merged commit 3670545 into master Jan 20, 2021
@cgardens cgardens deleted the cgardens/handle_tables_with_same_names_and_multiple_schemas branch January 20, 2021 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michel-tricot michel-tricot michel-tricot approved these changes

@sherifnada sherifnada Awaiting requested review from sherifnada

@jrhizor jrhizor Awaiting requested review from jrhizor

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unable to exit onboarding - Stuck on "Set up connection" in
2 participants
@cgardens @michel-tricot