Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[low-code cdk] decouple parsing the yaml manifest from the declarative source implementation #19095

Merged
merged 3 commits into from
Nov 8, 2022

Conversation

brianjlai
Copy link
Contributor

What

The current way that YamlDeclarativeSource is structured and implemented couples parsing and resolving a YAML file and the implementation to run connector commands like read, spec, and check.

YAML parsing can really just function as a subclass of the underlying declarative implementation made up of the ConnectorDefinition manifest file. And this will allow us to start using the ManifestDeclarativeSource from the connector builder UI.

How

This is mostly moving the implementation to a new superclass that YamlDeclarativeSource will then inherit from. And YamlDeclarativeSource only needs to override the constructor to parse a YAML file and the remaining flow should be the same. The PR also looks large, but I reorganized the tests so a bulk of our testing and edge cases is against the already parsed manifest and only behavior relevant to yaml parsing is in the existing tests.

Recommended reading order

  1. manifest_declarative_source.py
  2. yaml_declarative_source.py
  3. test_manifest_declarative_source.py
  4. test_yaml_declarative_source.py

@github-actions github-actions bot added the CDK Connector Development Kit label Nov 7, 2022
@brianjlai brianjlai marked this pull request as ready for review November 7, 2022 21:40
@brianjlai brianjlai requested a review from a team as a code owner November 7, 2022 21:40
@brianjlai
Copy link
Contributor Author

test no file change from a different package. will still publish

@brianjlai
Copy link
Contributor Author

brianjlai commented Nov 8, 2022

/publish-cdk dry-run=true

🕑 https://github.com/airbytehq/airbyte/actions/runs/3422632047
https://github.com/airbytehq/airbyte/actions/runs/3422632047

@brianjlai
Copy link
Contributor Author

brianjlai commented Nov 8, 2022

/publish-cdk dry-run=false

🕑 https://github.com/airbytehq/airbyte/actions/runs/3422977643
https://github.com/airbytehq/airbyte/actions/runs/3422977643

@brianjlai brianjlai merged commit 5c9e5d9 into master Nov 8, 2022
@brianjlai brianjlai deleted the brian/low_code_decouple_yaml_parse branch November 8, 2022 20:45
akashkulk pushed a commit that referenced this pull request Dec 2, 2022
…e source implementation (#19095)

* decouple parsing the yaml manifest from the declarative source implementation

* bump version and changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants