BigQuery 0.3.10-0.3.12 not migrating existing connections with non-US datasets

[sayowayah]

Enviroment

• Airbyte version: 0.29.4-alpha
• OS Version / Instance: Ubuntu 20.04.2
• Deployment: Docker
• Source Connector and version: (if applicable example Salesforce 0.2.3)
• Destination Connector and version: BigQuery 0.3.10-0.3.12
• Severity: High
• Step where error happened: Sync job

Current Behavior

I have a bunch of existing connections using non-US BigQuery datasets that were created pre-BigQuery connector 0.3.10 (when dataset location was introduced). When upgrading the connector to 0.3.10-0.3.12, my existing connections all fail with a 404 Not Found: <my dataset name> error. Even after adjusting the Destination settings to the correct region, the syncs still fail with the same error. Note that if I create a new connection with the same source, same destination, and same namespace settings as my existing connections, it works; therefore I suspect this is a migration issue where existing connections are not properly grabbing the dataset location set in the Destination settings.

Expected Behavior

I'd expect my existing connections to continue working with a connector upgrade.

[sherifnada]

@sayowayah thanks for the heads up! are you still blocked on this issue?

[sayowayah]

@sherifnada I'm personally not blocked at the moment because I downgraded to a working version. But I'm unable to upgrade to the latest connector versions

[DoNotPanicUA]

Hi @sayowayah,
is this problem still relevant for you? Have you tried the latest versions?
Current version of the BigQuery Destination is 0.6.5.

[sayowayah]

Hey! I actually just bit the bucket a few weeks ago and re-setup all my connections, so it's no longer an issue for me. Thanks