🐛 📝 DAT: verify that a destination is able to write any ISO8601-compliant date string

[bazarnov]

What

#5712 - Destination acceptance tests: verify that a destination is able to write any ISO8601-compliant date string

How

1. Verify snowflake destination is passing acceptance tests before beginning by running ./gradlew :airbyte-integrations:connectors:destination-snowflake:integrationTest (you'll need to place the config in lastpass in the secrets/ directory for this to work)
2. Identify all the acceptable ISO 8601 json schema date and date-time date string formats (i.e; what are all the acceptable string patterns for each of these formats?). It is probably helpful to list them in a text file in the directory linked above, just for reference e.g: just write out all the formats in a formats_reference.txt file.
3. Add one stream to the edge_cases_catalog which has two string fields whose format fields are set to date and date-time respectively.
4. For each format you listed in step 2, add a few records in the records file. Now the destination acceptance test (DAT) will try to insert them into every destination.
5. Verify that the snowflake tests are now failing.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions
• Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here

[sherifnada]

discussed offline this morning: since almost all destinations are failing this test we'll keep this branch alive while we merge fixes for the destinations

[CLAassistant]

All committers have signed the CLA.

[yurii-bidiuk]

The following integration tests are failed on the normalization phase with error like: invalid datetime '1/1/2021':

BigQueryDestinationAcceptanceTest.testSyncWithNormalization()
MySQLDestinationAcceptanceTest.testSyncWithNormalization()
PostgresDestinationAcceptanceTest.testSyncWithNormalization()
RedshiftInsertDestinationAcceptanceTest.testSyncWithNormalization()
SnowflakeS3CopyDestinationAcceptanceTest.testSyncWithNormalization()
SnowflakeInsertDestinationAcceptanceTest.testSyncWithNormalization()
SnowflakeGcsCopyDestinationAcceptanceTest.testSyncWithNormalization()

MSSQLDestinatuionAcceptanceTest.testSyncWithNormalization -> normalization returns null for values when value couldn't be normalized

These two tests are failed with the message like: Invalid format: "20210101T0930" is malformed at "30". It should be fixed on the Java destination side (io.airbyte.integrations.destination.bigquery.BigQueryUtils#transformJsonDateTimeToBigDataFormat):

BigQueryDenormalizedDestinationAcceptanceTest.testSyncNormalizedWithoutNormalization()
BigQueryDenormalizedDestinationAcceptanceTest.testSync()