CAT: Ensure we only generate iso8601 with hypothesis in backwards compatibility test

[bnchrch]

What

Ok so we have one GLOBAL problem of our formats do not match ALL valid ISO dates.

But lets ignore that.

The subproblem here is backwards compatibility tests are failing for the following reasons

1. Hypothesis generates date-times that represent all valid RFC3339 values (an ISO subset) (screenshot 1)
2. Our format for date-time in our specs are very strict about having 3 miliseconds (screenshot 2)

This causes failing backwards compatibility tests (slack thread)

Solution

Only generate date-times that represent our valid version

blocks #26535

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Octavia Squidington III seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

• Run integration tests
• Bump connector or module version
• Add changelog
• Publish the new version

✅ Sources (1)

Connector	Version	Changelog	Publish
source-railz	0.1.1	✅	✅

• See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector	Version	Changelog	Publish

• See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the cloud or oss registry, so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that you have added a metadata.yaml file and the expected registries are enabled.

[bnchrch]

@evantahler moving our slack convo here

TLDR: That was a good callout that had me reevaluate my approach to this test

The issue here is essentially hypothesis-jsonschema can generate properties that define format: date-time and pattern: <REGEX> but not both.

In the case of both it blows up because many of the date-time strings do not match our pattern regex's as they are often very strict

e.g. "^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z$",

I think our best solution to this is in the case of format: date-time + pattern is to

1. Only generate the value based on the regex pattern
2. Rely on the jsonschema validation that uses both the new schema and a different jsonschema library to validate that it is indeed a date-time field..

How we do that is by removing format: date-time anywhere its defined alongside a pattern from the previous schema before using it to generate prop tests. This forces hypothesis to use the regex pattern as its basis for generating values.

[sherifnada]

makes sense to me. Approving since this is blocking a roadmap item for GL

[davydov-d]

hey @bnchrch it looks like the problem still persists, could you please take a look? https://github.com/airbytehq/airbyte/actions/runs/5266559038/jobs/9520679064

[bnchrch]

Just closing the loop here as my reply yesterday was on the latest PR
#26535 (comment)

Essentially it looks like the problem is resolved!