Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support refreshes to the metadata. #37472

Merged
merged 2 commits into from Apr 29, 2024

Conversation

benmoriceau
Copy link
Contributor

What

Add the support refreshes to the metadata.

How

Add a new fields in ConnectorMetadataDefinitionV0.yaml. 2 test files (valid and invalid) have been added. The other files are auto generated.

Review guide

User Impact

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@benmoriceau benmoriceau requested a review from a team as a code owner April 22, 2024 16:39
Copy link

vercel bot commented Apr 22, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 26, 2024 4:58pm

@@ -113,3 +113,5 @@ properties:
"$ref": AirbyteInternal.yaml
remoteRegistries:
"$ref": RemoteRegistries.yaml
supportsRefreshes:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want this to appear in the registry you will have to add it to ConnectorRegistrySourceDefinition.yaml and ConnectorRegistryDestinationDefinition.yaml as well with defaults

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, I only added it to the destination since it is not needed for the source

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some converter that are needed as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, basically any metadata fields will be copied over to the registry so long they are defined in both

Copy link
Contributor

@bnchrch bnchrch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for diving in! Just a few minor comments

@@ -113,3 +113,5 @@ properties:
"$ref": AirbyteInternal.yaml
remoteRegistries:
"$ref": RemoteRegistries.yaml
supportsRefreshes:
type: boolean
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a sane default?

Suggested change
type: boolean
type: boolean
default: false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -38,18 +34,8 @@ class Config:


class JobType(BaseModel):
__root__: Literal[
"get_spec",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a feeling CI will get mad since it looks like these were not run through the formatter.

I think you can run

airbyte-ci format fix python

unless something has changed in the last 3 months.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, It didn't changed any file:
image

@@ -113,3 +113,5 @@ properties:
"$ref": AirbyteInternal.yaml
remoteRegistries:
"$ref": RemoteRegistries.yaml
supportsRefreshes:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want this to appear in the registry you will have to add it to ConnectorRegistrySourceDefinition.yaml and ConnectorRegistryDestinationDefinition.yaml as well with defaults

Copy link
Contributor

@bnchrch bnchrch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@benmoriceau benmoriceau merged commit bc7b4cb into master Apr 29, 2024
35 checks passed
@benmoriceau benmoriceau deleted the bmoric/rollout-refresh-metadata branch April 29, 2024 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants