Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QA checks] tolerate connector without dockerfile #30467

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Sep 15, 2023

What

Now that it's OK that a connector does not have a Dockerfile our QA checks should tolerate it.

How

  • Expose a has_dockerfile property on the Connector utility class
  • Do not run dockerfile related checks when a Dockerfile does not exist

@alafanechere alafanechere marked this pull request as ready for review September 15, 2023 10:39
@alafanechere alafanechere force-pushed the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch from 2aef971 to 41ce5f5 Compare September 15, 2023 11:25
@alafanechere alafanechere changed the base branch from 09-14-change_our_python_connector_build_process_to_use_the_base_images to augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container September 15, 2023 14:01
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch 2 times, most recently from 001d411 to 43049f0 Compare September 15, 2023 14:12
@alafanechere alafanechere force-pushed the augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container branch from f9bea4b to b608d76 Compare September 15, 2023 15:08
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from 43049f0 to 2690638 Compare September 15, 2023 15:08
@alafanechere alafanechere requested a review from a team September 15, 2023 15:34
)


def get_qa_checks_to_run(connector: Connector) -> Tuple[Callable]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@alafanechere alafanechere force-pushed the augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container branch from b608d76 to 10448cc Compare September 17, 2023 15:16
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from 2690638 to 5938a15 Compare September 17, 2023 15:16
@alafanechere alafanechere force-pushed the augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container branch from 10448cc to 72b1de6 Compare September 18, 2023 10:40
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from 5938a15 to b83cb4d Compare September 18, 2023 10:40
@alafanechere alafanechere force-pushed the augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container branch from 72b1de6 to 3fffee5 Compare September 18, 2023 10:45
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from b83cb4d to 9029852 Compare September 18, 2023 10:45
@alafanechere alafanechere force-pushed the augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container branch from 3fffee5 to 0a60854 Compare September 18, 2023 14:16
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from 9029852 to e6bcea4 Compare September 18, 2023 14:16
@alafanechere alafanechere force-pushed the augustin/09-15-_airbyte-ci_test_connector_inside_their_built_container branch from 0a60854 to 94f6f82 Compare September 18, 2023 14:46
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch 2 times, most recently from d66bece to d067ece Compare September 21, 2023 11:05
return None

@property
def version(self) -> str:
def version(self) -> Optional[str]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this one optional? I get becuase self.version_in_dockerfile_label could now be None - but shouldn't we error out if we can't retrieve a version for the connector?

@bnchrch bnchrch force-pushed the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch from 96068a5 to 18693fc Compare September 28, 2023 15:16
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from d067ece to d8731ed Compare October 10, 2023 13:55
@airbyte-oss-build-runner
Copy link
Collaborator

source-faker test report (commit d8731ed979) - ✅

⏲️ Total pipeline duration: 01mn58s

Step Result
Connector package install
Build source-faker docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-faker
Connector version semver check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-faker test

@airbyte-oss-build-runner
Copy link
Collaborator

destination-duckdb test report (commit d8731ed979) - ✅

⏲️ Total pipeline duration: 02mn33s

Step Result
Connector package install
Build destination-duckdb docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for destination-duckdb
Connector version semver check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-duckdb test

@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch 2 times, most recently from 6456ed2 to 183eef5 Compare October 10, 2023 14:02
@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label Oct 10, 2023
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch 2 times, most recently from 74800ab to 6f77a7a Compare October 10, 2023 14:07
Base automatically changed from 09-14-change_our_python_connector_build_process_to_use_the_base_images to master October 10, 2023 17:15
@alafanechere alafanechere force-pushed the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch from 6f77a7a to 98c98e3 Compare October 10, 2023 18:10
@airbyte-oss-build-runner
Copy link
Collaborator

source-auth0 test report (commit 2c438bc195) - ✅

⏲️ Total pipeline duration: 02mn09s

Step Result
Connector package install
Build source-auth0 docker image for platform(s) linux/x86_64
Acceptance tests
Code format checks
Validate metadata for source-auth0
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-auth0 test

@alafanechere alafanechere enabled auto-merge (squash) October 11, 2023 07:06
@alafanechere alafanechere merged commit 846d9a6 into master Oct 11, 2023
30 checks passed
@alafanechere alafanechere deleted the augustin/09-15-make_QA_checks_tolerate_connector_without_dockerfile branch October 11, 2023 07:08
ariesgun pushed a commit to ariesgun/airbyte that referenced this pull request Oct 20, 2023
ariesgun pushed a commit to ariesgun/airbyte that referenced this pull request Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants