17644 Update Destination data type test to use the new data types #19281
Conversation
Affected Connector ReportNOTE
|
| Connector | Version | Changelog | Publish |
|---|
- See "Actionable Items" below for how to resolve warnings and errors.
❌ Destinations (47)
| Connector | Version | Changelog | Publish |
|---|---|---|---|
destination-aws-datalake |
0.1.1 |
✅ | ✅ |
destination-azure-blob-storage |
0.1.6 |
⚠ (doc not found) |
✅ |
destination-bigquery |
1.2.7 |
✅ | ✅ |
destination-bigquery-denormalized |
1.2.7 |
⚠ (doc not found) |
✅ |
destination-cassandra |
0.1.4 |
❌ (changelog missing) |
✅ |
destination-clickhouse |
0.2.0 |
✅ | ✅ |
destination-clickhouse-strict-encrypt |
0.2.0 |
✅ | ⚠ (not in seed) |
destination-csv |
0.2.10 |
⚠ (doc not found) |
✅ |
destination-databricks |
0.3.1 |
✅ | ✅ |
destination-dev-null |
0.2.7 |
⚠ (doc not found) |
⚠ (not in seed) |
destination-doris |
0.1.0 |
❌ (changelog missing) |
✅ |
destination-dynamodb |
0.1.7 |
✅ | ✅ |
destination-e2e-test |
0.2.4 |
✅ | ✅ |
destination-elasticsearch |
0.1.6 |
✅ | ✅ |
destination-elasticsearch-strict-encrypt |
0.1.6 |
✅ | ⚠ (not in seed) |
destination-gcs |
0.2.12 |
✅ | ✅ |
destination-iceberg |
0.1.0 |
❌ (changelog missing) |
✅ |
destination-jdbc |
0.3.14 |
⚠ (doc not found) |
⚠ (not in seed) |
destination-kafka |
0.1.10 |
❌ (changelog missing) |
✅ |
destination-keen |
0.2.4 |
❌ (changelog missing) |
✅ |
destination-kinesis |
0.1.5 |
❌ (changelog missing) |
✅ |
destination-local-json |
0.2.11 |
✅ | ✅ |
destination-mariadb-columnstore |
0.1.7 |
✅ | ✅ |
destination-mongodb |
0.1.9 |
✅ | ✅ |
destination-mongodb-strict-encrypt |
0.1.9 |
✅ | ⚠ (not in seed) |
destination-mqtt |
0.1.3 |
✅ | ✅ |
destination-mssql |
0.1.22 |
✅ | ✅ |
destination-mssql-strict-encrypt |
0.1.22 |
✅ | ⚠ (not in seed) |
destination-mysql |
0.1.20 |
✅ | ✅ |
destination-mysql-strict-encrypt |
❌ 0.1.21(mismatch: 0.1.20) |
✅ | ⚠ (not in seed) |
destination-oracle |
0.1.19 |
✅ | ✅ |
destination-oracle-strict-encrypt |
0.1.19 |
✅ | ⚠ (not in seed) |
destination-postgres |
0.3.26 |
✅ | ✅ |
destination-postgres-strict-encrypt |
0.3.26 |
✅ | ⚠ (not in seed) |
destination-pubsub |
0.1.6 |
❌ (changelog missing) |
✅ |
destination-pulsar |
0.1.3 |
❌ (changelog missing) |
✅ |
destination-r2 |
0.1.0 |
✅ | ✅ |
destination-redis |
0.1.4 |
✅ | ✅ |
destination-redpanda |
0.1.0 |
❌ (changelog missing) |
✅ |
destination-redshift |
0.3.51 |
✅ | ✅ |
destination-rockset |
0.1.4 |
❌ (changelog missing) |
✅ |
destination-s3 |
0.3.17 |
✅ | ✅ |
destination-s3-glue |
0.1.0 |
❌ (changelog missing) |
✅ |
destination-scylla |
0.1.3 |
✅ | ✅ |
destination-snowflake |
0.4.40 |
✅ | ✅ |
destination-tidb |
0.1.0 |
✅ | ✅ |
destination-yugabytedb |
0.1.0 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
✅ Other Modules (0)
Actionable Items
(click to expand)
| Category | Status | Actionable Item |
|---|---|---|
| Version | ❌ mismatch |
The version of the connector is different from its normal variant. Please bump the version of the connector. |
| ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
|
| Changelog | ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
| ❌ changelog missing |
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
| Publish | ⚠ not in seed |
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
| ❌ diff seed version |
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
|
/test connector=connectors/destination-aws-datalake, destination-azure-blob-storage, destination-bigquery
|
|
/test connector=connectors/destination-aws-datalake
Build FailedTest summary info: |
|
/test connector=connectors/destination-azure-blob-storage
Build PassedTest summary info: |
|
/test connector=connectors/destination-bigquery
Build PassedTest summary info: |
|
/test connector=connectors/destination-cassandra
Build PassedTest summary info: |
|
/test connector=connectors/destination-clickhouse
Build PassedTest summary info: |
|
/test connector=connectors/destination-csv
Build PassedTest summary info: |
|
/test connector=connectors/destination-bigquery-denormalized
Build PassedTest summary info: |
|
/test connector=connectors/destination-clickhouse-strict-encrypt
Build PassedTest summary info: |
|
/test connector=connectors/destination-databricks
Build PassedTest summary info: |
|
/test connector=connectors/destination-dev-null
Build FailedTest summary info: |
|
/test connector=connectors/destination-dynamodb
Build PassedTest summary info: |
|
/test connector=connectors/destination-e2e-test
Build FailedTest summary info: |
|
/test connector=connectors/destination-elasticsearch
Build PassedTest summary info: |
|
/test connector=connectors/destination-elasticsearch-strict-encrypt
Build PassedTest summary info: |
|
/test connector=connectors/destination-gcs
Build PassedTest summary info: |
|
/test connector=connectors/destination-jdbc
Build PassedTest summary info: |
|
/test connector=connectors/destination-kafka
Build PassedTest summary info: |
|
/test connector=connectors/destination-keen
Build PassedTest summary info: |
|
/test connector=connectors/destination-kinesis
Build PassedTest summary info: |
|
/test connector=connectors/destination-local-json
Build PassedTest summary info: |
|
/test connector=connectors/destination-mariadb-columnstore
Build PassedTest summary info: |
|
/test connector=connectors/destination-mongodb
Build PassedTest summary info: |
|
/test connector=connectors/destination-mongodb-strict-encrypt
Build PassedTest summary info: |
|
/test connector=connectors/destination-mqtt
Build PassedTest summary info: |
|
/test connector=connectors/destination-mssql
Build PassedTest summary info: |
|
/test connector=connectors/destination-mssql-strict-encrypt
Build PassedTest summary info: |
|
/test connector=connectors/destination-mysql
Build FailedTest summary info: |
…_tests_new_data_types
...ion-test/src/main/java/io/airbyte/integrations/standardtest/destination/ProtocolVersion.java
Outdated
Show resolved
Hide resolved
...ses/standard-destination-test/src/main/resources/v0/data_type_array_object_test_catalog.json
Outdated
Show resolved
Hide resolved
| "type": ["array"], | ||
| "items": { | ||
| "type": ["object"], | ||
| "properties": { |
There was a problem hiding this comment.
let's add a property_binary_data to the new-style test cases, since that's a new data type.
There was a problem hiding this comment.
sorry, I meant that all the test files should include binary data (i.e. also data_type_array_test, data_type_basic_test, etc)
Yeah good point. Will work on this. Assuming this will require two new fields in |
| "$ref": "WellKnownTypes.json#definitions/TimestampWithTimezone" | ||
| }, | ||
| "property_timestamp_without_timezone": { | ||
| "$ref": "WellKnownTypes.json#definitions/TimestampWithoutTimezone" |
There was a problem hiding this comment.
I see we also have TimeWithTimezone and TimeWithoutTimezone. Do we cover it?
There was a problem hiding this comment.
Adding it. Thanks for noticing
|
/test connector=connectors/destination-postgres
Build PassedTest summary info: |
…_tests_new_data_types
|
This PR only changes tests and does not affect production, so it is safe to merge. |
grishick
deleted the
suhomud/17644_update_destination_data_type_tests_new_data_types
branch
…9281) This PR only changes tests and does not affect production, so it is safe to merge. * 17644 Destination acceptance test: protocol support * 17644 data type array test catalog update * 17644 correct array string type * add v0 and v1 versions * added binary_data type * Added infinity, nan support. Added test time dataset. Updated ref link * format fix
* master: (74 commits) Fix support icon in sidebar fix: add BuildPulse report for helm ac tests (#19785) fix: yaml syntax (#19775) 🪟 🐛 Fix custom connection creation endpoint (#19702) Source facebook marketing: check "breakdowns" combinations (#19645) fix typo: notify instead of sync (#19737) find_non_rate_limited_PAT (#19736) Source Google Ads: fix schema for "campaigns" stream (#19700) 🎉 Source Asana: migrate to new SAT, added base HTTP errors handling (#19561) fix order not to randomly fail backward compatibility check (#19377) Bump helm chart version reference to 0.42.0 (#19706) fix: add extraEnv block (#19703) Bump Airbyte version from 0.40.21 to 0.40.22 (#19687) Bump helm chart version reference to 0.41.3 (#19685) Add connector builder server to airbyte proxy, kube overlays, and helm charts (#19554) dbt Cloud integration doc (#19619) 🪟 🎉 Display service token validation errors in the UI (#19578) 17644 Update Destination data type test to use the new data types (#19281) Docs: fix broken connector builder UI docs links (#19631) Bump Airbyte version from 0.40.20 to 0.40.21 (#19634) ...
What
Prerequisite : see
Add support of new protocol versions for
DestinationAcceptanceTest. With this change we can handle different protocol versions in destination integration tests.How
Added new method
getProtocolVersion()toDestinationAcceptanceTestwhich can be overridden in child classes.original method example:
overridden:
The ProtocolVersion is used to prefix dataset files
Recommended reading order
DestinationAcceptanceTest- new getProtocolVersion method, prefixing files name.ProtocolVersion- enum with supported protocol version and file prefixArgumentProviderUtilprefix file name method, reflection magic (This one is used to get protocol version in Argument Providers, since during it's execution test Instance is not instantiated. PS: Feel free to argue this approach since this result can be achieved in a tests itself without modifying Argument Providers )DataArgumentsProviderDataTypeTestArgumentProviderNamespaceTestCaseProvidermoved fromDestinationAcceptanceTest.NumberDataTypeTestArgumentProviderresources/v0folder