Migrate ssh exception -> config error exception #19094

akashkulk · 2022-11-07T21:08:14Z

Couple of exception handling changes:

Migrate ssh exception -> config error exception
Unnest in IntegrationRunner to determine whether an exception is a config error. In the codebase, many exceptions are caught and re-packaged as RuntimeException, so we have to be grab the root exception to check if it is a config-error

Migrate ssh exception -> config error exception

github-actions · 2022-11-07T21:09:22Z

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

Run integration tests
Bump connector version
Add changelog
Publish the new version

⚠ Sources (30)

Connector	Version	Changelog	Publish
`source-alloydb`	`1.0.17`	✅	✅
`source-alloydb-strict-encrypt`	`1.0.17`	✅	⚠ (not in seed)
`source-bigquery`	`0.2.3`	✅	✅
`source-clickhouse`	`0.1.14`	✅	✅
`source-clickhouse-strict-encrypt`	`0.1.14`	✅	⚠ (not in seed)
`source-cockroachdb`	`0.1.18`	✅	✅
`source-cockroachdb-strict-encrypt`	`0.1.18`	✅	⚠ (not in seed)
`source-db2`	`0.1.16`	✅	✅
`source-db2-strict-encrypt`	`0.1.16`	✅	⚠ (not in seed)
`source-e2e-test`	`2.1.3`	✅	✅
`source-e2e-test-cloud`	`2.1.1`	⚠ (doc not found)	⚠ (not in seed)
`source-elasticsearch`	`0.1.0`	✅	✅
`source-jdbc`	`0.3.5`	⚠ (doc not found)	⚠ (not in seed)
`source-kafka`	`0.2.2`	✅	✅
`source-mongodb-strict-encrypt`	`0.1.19`	⚠ (doc not found)	⚠ (not in seed)
`source-mongodb-v2`	`0.1.19`	✅	✅
`source-mssql`	`0.4.25`	✅	✅
`source-mssql-strict-encrypt`	`0.4.25`	✅	⚠ (not in seed)
`source-mysql`	`1.0.11`	✅	✅
`source-mysql-strict-encrypt`	`1.0.11`	✅	⚠ (not in seed)
`source-oracle`	`0.3.21`	✅	✅
`source-oracle-strict-encrypt`	`0.3.21`	✅	⚠ (not in seed)
`source-postgres`	`1.0.22`	✅	✅
`source-postgres-strict-encrypt`	`1.0.22`	✅	⚠ (not in seed)
`source-redshift`	`0.3.15`	✅	✅
`source-relational-db`	`0.3.1`	⚠ (doc not found)	⚠ (not in seed)
`source-scaffold-java-jdbc`	`0.1.0`	⚠ (doc not found)	⚠ (not in seed)
`source-sftp`	`0.1.2`	✅	✅
`source-snowflake`	`0.1.24`	✅	✅
`source-tidb`	`0.2.1`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

❌ Destinations (42)

Connector	Version	Changelog	Publish
`destination-azure-blob-storage`	`0.1.6`	⚠ (doc not found)	✅
`destination-bigquery`	`1.2.5`	✅	✅
`destination-bigquery-denormalized`	`1.2.5`	⚠ (doc not found)	✅
`destination-cassandra`	`0.1.4`	❌ (changelog missing)	✅
`destination-clickhouse`	`0.2.0`	✅	✅
`destination-clickhouse-strict-encrypt`	`0.2.0`	✅	⚠ (not in seed)
`destination-csv`	`0.2.10`	⚠ (doc not found)	✅
`destination-databricks`	`0.3.1`	✅	✅
`destination-dev-null`	`0.2.7`	⚠ (doc not found)	⚠ (not in seed)
`destination-dynamodb`	`0.1.7`	✅	✅
`destination-e2e-test`	`0.2.4`	✅	✅
`destination-elasticsearch`	`0.1.6`	✅	✅
`destination-elasticsearch-strict-encrypt`	`0.1.6`	✅	⚠ (not in seed)
`destination-gcs`	`0.2.12`	✅	✅
`destination-jdbc`	`0.3.14`	⚠ (doc not found)	⚠ (not in seed)
`destination-kafka`	`0.1.10`	❌ (changelog missing)	✅
`destination-keen`	`0.2.4`	❌ (changelog missing)	✅
`destination-kinesis`	`0.1.5`	❌ (changelog missing)	✅
`destination-local-json`	`0.2.11`	✅	✅
`destination-mariadb-columnstore`	`0.1.7`	✅	✅
`destination-mongodb`	`0.1.8`	✅	✅
`destination-mongodb-strict-encrypt`	`0.1.8`	✅	⚠ (not in seed)
`destination-mqtt`	`0.1.3`	✅	✅
`destination-mssql`	`0.1.22`	✅	✅
`destination-mssql-strict-encrypt`	`0.1.22`	✅	⚠ (not in seed)
`destination-mysql`	`0.1.20`	✅	✅
`destination-mysql-strict-encrypt`	❌ `0.1.21` (mismatch: `0.1.20`)	✅	⚠ (not in seed)
`destination-oracle`	`0.1.19`	✅	✅
`destination-oracle-strict-encrypt`	`0.1.19`	✅	⚠ (not in seed)
`destination-postgres`	`0.3.26`	✅	✅
`destination-postgres-strict-encrypt`	`0.3.26`	✅	⚠ (not in seed)
`destination-pubsub`	`0.1.6`	❌ (changelog missing)	✅
`destination-pulsar`	`0.1.3`	❌ (changelog missing)	✅
`destination-r2`	`0.1.0`	✅	✅
`destination-redis`	`0.1.4`	✅	✅
`destination-redshift`	`0.3.51`	✅	✅
`destination-rockset`	`0.1.4`	❌ (changelog missing)	✅
`destination-s3`	`0.3.17`	✅	✅
`destination-scylla`	`0.1.3`	✅	✅
`destination-snowflake`	`0.4.38`	✅	✅
`destination-tidb`	`0.1.0`	✅	✅
`destination-yugabytedb`	`0.1.0`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the seed file (e.g. `source_definitions.yaml`), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
	❌ diff seed version	The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the `/publish` command to publish the latest version.

akashkulk · 2022-11-07T23:57:02Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3415095158
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3415095158
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-07T23:57:12Z

/test connector=connectors/source-postgres-strict-encrypt

🕑 connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3415095351
✅ connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3415095351
No Python unittests run

Build Passed

Test summary info:

All Passed

rodireich · 2022-11-08T00:04:36Z

...tegrations/bases/base-java/src/main/java/io/airbyte/integrations/base/IntegrationRunner.java

+      if (isConfigError(current)) {
+        return current;
+      } else {
+        current = current.getCause();


Like peeling an onion..

May be able to simply use google commons Throwable library to do something like this

https://github.com/spotify/helios/blob/0b2855955dc6f3268e772252d84861082a516b6e/helios-services/src/main/java/com/spotify/helios/agent/HealthCheckerFactory.java#L153-L159

Good find! Unfortunately, Throwables.getRootCause gets the root exception. Here, we actually want the root exception that is a possible configexception. The nuance here is that we could catch some SQLException, repackage it as a ConfigErrorException which then is repackaged up the stack as a RuntimeException.

We actually want the ConfigErrorException, so I think we have to manually unnest this

I see, then can we add a javadoc comment that indicates this looks for the first instance of a ConfigErrorException? Also curious what happens if the Exception does not contain a ConfigErrorException, it seems would it be returning null, right?

Ahhh good catch :(
Will just return the original exception if there is no config error. Also updated the comments

ryankfu

LGTM, thanks for making sure we're not the ones last min causing a NPE

akashkulk and others added 2 commits November 7, 2022 13:05

Update SshTunnel.java

91d19b6

Migrate ssh exception -> config error exception

Merge branch 'master' into new_config_err

139b152

Unnest exception

e496bf3

akashkulk requested a review from rodireich November 7, 2022 23:55

Merge branch 'master' into new_config_err

c216bc4

akashkulk marked this pull request as ready for review November 7, 2022 23:57

akashkulk requested a review from a team as a code owner November 7, 2022 23:57

rodireich approved these changes Nov 8, 2022

View reviewed changes

akashkulk requested a review from ryankfu November 8, 2022 00:40

Address PR comments

9f28f16

ryankfu approved these changes Nov 8, 2022

View reviewed changes

Add test case

9071287

akashkulk merged commit 4fe8468 into master Nov 8, 2022

akashkulk deleted the new_config_err branch November 8, 2022 15:12

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate ssh exception -> config error exception #19094

Migrate ssh exception -> config error exception #19094

akashkulk commented Nov 7, 2022 •

edited

Loading

github-actions bot commented Nov 7, 2022 •

edited

Loading

⚠ Sources (30)

❌ Destinations (42)

Actionable Items

akashkulk commented Nov 7, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 7, 2022 •

edited by github-actions bot

Loading

rodireich Nov 8, 2022

ryankfu Nov 8, 2022 •

edited

Loading

akashkulk Nov 8, 2022

ryankfu Nov 8, 2022

akashkulk Nov 8, 2022

ryankfu left a comment

Migrate ssh exception -> config error exception #19094

Migrate ssh exception -> config error exception #19094

Conversation

akashkulk commented Nov 7, 2022 • edited Loading

github-actions bot commented Nov 7, 2022 • edited Loading

Affected Connector Report

⚠ Sources (30)

❌ Destinations (42)

Actionable Items

akashkulk commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

akashkulk commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

rodireich Nov 8, 2022

Choose a reason for hiding this comment

ryankfu Nov 8, 2022 • edited Loading

Choose a reason for hiding this comment

akashkulk Nov 8, 2022

Choose a reason for hiding this comment

ryankfu Nov 8, 2022

Choose a reason for hiding this comment

akashkulk Nov 8, 2022

Choose a reason for hiding this comment

ryankfu left a comment

Choose a reason for hiding this comment

akashkulk commented Nov 7, 2022 •

edited

Loading

github-actions bot commented Nov 7, 2022 •

edited

Loading

akashkulk commented Nov 7, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 7, 2022 •

edited by github-actions bot

Loading

ryankfu Nov 8, 2022 •

edited

Loading