Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connector-acceptance-test: test it with airbyte-ci #29509

Merged
merged 13 commits into from
Aug 18, 2023

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Aug 17, 2023

What

Now that we daggerized CAT we can't test it with ./gradlew anymore.
This PR enables CAT testing in the CI using the airbyte-ci test command.

How

  • Remove connector-acceptance-test from tools/bin/ci_integration_test.sh. It means CAT will no longer be testable with /legacy-test
  • Refactor the airbyte-ci test command to make it capable of running any pytest suite for a poetry package
  • Update the airbyte-ci-test.yml following the refactor
  • Add a cat-tests.yml worfklow to run CAT unit tests on PR modifying it.

Summary:

  • How do I test CAT locally: using poetry directly or using airbyte-ci test airbyte-integrations/bases/connector-acceptance-test --test-directory=unit_tests
  • How can I test my CAT changes on a connector: the airbyte-ci connectors test command always build the local CAT version: if you run airbyte-ci connectors --name=source-faker test airbyte-ci will use the local CAT.
  • How can I publish a new CAT version: the story is not great: /legacy-publish connector=bases/connector-acceptance-test run-tests=false . We're working on porting the CAT publish workflow to airbyte-ci.

@alafanechere
Copy link
Contributor Author

alafanechere commented Aug 17, 2023

/legacy-test connector=connectors/source-openweather local_cdk=1

🕑 connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/5888048602
✅ connectors/source-openweather https://github.com/airbytehq/airbyte/actions/runs/5888048602
Python tests coverage:

Name                                      Stmts   Miss  Cover
-------------------------------------------------------------
source_openweather/source.py                 22      0   100%
source_openweather/extra_validations.py      22      0   100%
source_openweather/__init__.py                2      0   100%
source_openweather/streams.py                33      5    85%
-------------------------------------------------------------
TOTAL                                        79      5    94%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../app/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../app/connector_acceptance_test/tests/test_core.py:103: The previous and actual specifications are identical.
SKIPPED [1] ../app/connector_acceptance_test/tests/test_core.py:623: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../app/connector_acceptance_test/tests/test_core.py:720: This tests currently leads to too much failures. We need to fix the connectors at scale first.
============ 38 passed, 4 skipped, 44 warnings in 80.71s (0:01:20) =============

@alafanechere
Copy link
Contributor Author

alafanechere commented Aug 17, 2023

/legacy-test connector=bases/connector-acceptance-test local_cdk=1

🕑 bases/connector-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/5888146848
❌ bases/connector-acceptance-test https://github.com/airbytehq/airbyte/actions/runs/5888146848
🐛

Build Failed

Test summary info:

Could not find result summary

@alafanechere alafanechere changed the title remove connector-acceptance-test from /legacy-test connector-acceptance-test: test it with airbyte-ci Aug 17, 2023
@alafanechere alafanechere marked this pull request as ready for review August 17, 2023 08:59
@alafanechere alafanechere requested review from a team and maxi297 August 17, 2023 09:00
@alafanechere alafanechere enabled auto-merge (squash) August 17, 2023 15:22
@alafanechere alafanechere merged commit 2af100f into master Aug 18, 2023
20 checks passed
@alafanechere alafanechere deleted the augustin/fix-legacy-test branch August 18, 2023 00:59
harrytou pushed a commit to KYVENetwork/airbyte that referenced this pull request Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants