Skip to content

Edgao/component upsert tests/connectors - #68186

Merged
Edward Gao (edgao) merged 29 commits into
masterfrom
edgao/component_upsert_tests/connectors
Oct 29, 2025
Merged

Edward Gao (edgao) merged 29 commits into
masterfrom
edgao/component_upsert_tests/connectors

Conversation

@edgao

@edgao Edward Gao (edgao) commented Oct 17, 2025 •

Copy link
Copy Markdown
Contributor

What

Connectors half of part 2 of https://github.com/airbytehq/airbyte-internal-issues/issues/14865. Branched from #68185.

How

Mostly just minor tweaks. Crucially, make snowflake's readTable method read timestamptz/array/object/variant in the way that the expected records expect.

Technically: we could make the snowflake tests declare a different set of expected records? But that feels really annoying, given that I want to change the input records at some point https://github.com/airbytehq/airbyte-internal-issues/issues/14910

(the readTable changes also just seem pretty reasonable, i.e. read an iso8601 timestamp instead of whatever snowflake does by default; read unstructured types into a map/list/etc. instead of leaving them as string)

Review guide

As usual, review each commit in sequence.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@github-actions

Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
    • /bump-bulk-cdk-version type=patch changelog='foo' - Bump the Bulk CDK's version. type can be major/minor/patch.
  • Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Oct 18, 2025 •

Copy link
Copy Markdown
Contributor

destination-snowflake Connector Test Results

 27 files   27 suites   27m 45s ⏱️
330 tests 294 ✅ 36 💤 0 ❌
335 runs  298 ✅ 36 💤 1 ❌

Results for commit 846d001.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Oct 18, 2025 •

Copy link
Copy Markdown
Contributor

destination-clickhouse Connector Test Results

 14 files   14 suites   23m 5s ⏱️
283 tests 234 ✅ 49 💤 0 ❌
313 runs  264 ✅ 49 💤 0 ❌

Results for commit 846d001.

♻️ This comment has been updated with latest results.

@edgao
Edward Gao (edgao) force-pushed the edgao/component_upsert_tests/cdk branch from 34fc6e7 to 8d94d4a Compare October 20, 2025 16:55

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

# Conflicts:
#	airbyte-cdk/bulk/core/load/src/testFixtures/kotlin/io/airbyte/cdk/load/component/TableOperationsFixtures.kt
#	airbyte-cdk/bulk/core/load/src/testFixtures/kotlin/io/airbyte/cdk/load/component/TableOperationsSuite.kt
#	airbyte-cdk/bulk/core/load/src/testFixtures/kotlin/io/airbyte/cdk/load/component/TableOperationsTestHarness.kt
#	airbyte-integrations/connectors/destination-clickhouse/gradle.properties
#	airbyte-integrations/connectors/destination-snowflake/gradle.properties
#	airbyte-integrations/connectors/destination-snowflake/src/main/kotlin/io/airbyte/integrations/destination/snowflake/client/SnowflakeAirbyteClient.kt
Base automatically changed from edgao/component_upsert_tests/cdk to master October 28, 2025 23:24
# ClickHouse

A fresh implementation of ClickHouse leveraging our new CDK.
A fresh implementation of ClickHouse leveraging our new CDK.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.We] Try to avoid using first-person plural like 'our'.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint-fix] reported by reviewdog 🐶

* All sync modes supported
* Data will be typed and written to columns matching the defined schema (Direct Load)
* Performance improvements
* Actively maintained and developed by Airbyte

@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

Deploy preview for airbyte-docs ready!

✅ Preview
https://airbyte-docs-i66k518au-airbyte-growth.vercel.app

Built with commit 846d001.
This pull request is being automatically deployed with vercel-action

@edgao
Edward Gao (edgao) merged commit 3f704ca into master Oct 29, 2025
39 of 40 checks passed
@edgao
Edward Gao (edgao) deleted the edgao/component_upsert_tests/connectors branch October 29, 2025 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants