Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the underlying component schema to use check instead of checker #19910

Merged
merged 5 commits into from
Dec 1, 2022

Conversation

brianjlai
Copy link
Contributor

@brianjlai brianjlai commented Nov 30, 2022

What

The manifest component schema that is generated dynamically based on the low-code data model and used to validate manifests expects checker but it should really be validating that check is present.

How

In the class ConcreteDeclarativeSource, the dataclass field is called checker which then formats the manifest by that name. This is causing an issue for the form based connector builder UI. For some reason in a manifest we take the contents of check and place them under checker. We should just do away with this entirely and format everything under checker. This PR does that and also regenerates the static schema as well.

Given that all manifests were already defining the check, this isn't a breaking change, we're just doing away with remapping it to checker which seems unnecessary.

Recommended reading order

  1. manifest_declarative_source.py

@octavia-squidington-iv octavia-squidington-iv added the CDK Connector Development Kit label Nov 30, 2022
"SimpleRetriever": {
"allOf": [
{
"$ref": "#/definitions/Retriever"
},
{
"type": "object",
"required": ["requester", "record_selector"],
"required": ["requester", "record_selector", "config"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why this PR is adding config as a required field to several of these objects? I think ideally this would not be required, since we are not planning to set this in the connector builder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because we haven't regenerated the schema in at least a few weeks and there were some component classes that had the config object added as a field. So that gets reflected in the auto-generated schema. I did some spot checks and this is accurate to the current fields of the SimpleRetriever class.

Sadly this is just another case of the downsides of the autogenerated schema since we're coupled to the underlying implementation which passes things like config under the hood

@brianjlai brianjlai marked this pull request as ready for review November 30, 2022 17:13
@brianjlai brianjlai requested a review from a team as a code owner November 30, 2022 17:13
@brianjlai
Copy link
Contributor Author

brianjlai commented Dec 1, 2022

/publish-cdk dry-run=true

🕑 https://github.com/airbytehq/airbyte/actions/runs/3588545489
https://github.com/airbytehq/airbyte/actions/runs/3588545489

@brianjlai
Copy link
Contributor Author

brianjlai commented Dec 1, 2022

/publish-cdk dry-run=false

🕑 https://github.com/airbytehq/airbyte/actions/runs/3588613924
https://github.com/airbytehq/airbyte/actions/runs/3588613924

@brianjlai brianjlai merged commit b1c50b8 into master Dec 1, 2022
@brianjlai brianjlai deleted the brian/rename_checker_manifest_schema branch December 1, 2022 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants