Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harmonize /test and /publish commands on github + Fix MySQL tests in CI #2531

Merged
merged 7 commits into from
Mar 23, 2021

Conversation

ChristopheDuong
Copy link
Contributor

@ChristopheDuong ChristopheDuong commented Mar 19, 2021

What

When we want to publish docker images, we can comment on github with

/publish connector=connectors/source-gitlab-singer

But if we want to run integration tests, it's:

/test connector=source-gitlab-singer

How

This PR makes it so that we can also run integration tests with:

/test connector=connectors/source-gitlab-singer

so it is easier to switch between a publish or a test comment

It is also now possible to run:

/test connector=bases/base-normalization

whereas, the following command will fail:

/test connector=base-normalization

with:

* What went wrong:
Project 'base-normalization' not found in project ':airbyte-integrations:connectors'.

This PR also fix compilation error of normalization image and tests for the MySQL sources when launched from CI

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Mar 22, 2021

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/675840938
❌ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/675840938

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Mar 22, 2021

/test connector=source-mysql

🕑 source-mysql https://github.com/airbytehq/airbyte/actions/runs/676283615
❌ source-mysql https://github.com/airbytehq/airbyte/actions/runs/676283615

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Mar 22, 2021

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/676308852
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/676308852

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Mar 22, 2021

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/676881192
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/676881192

@ChristopheDuong ChristopheDuong marked this pull request as ready for review March 22, 2021 17:27
@ChristopheDuong ChristopheDuong changed the title Harmonize /test and /publish commands on github Harmonize /test and /publish commands on github + Fix MySQL tests in CI Mar 22, 2021
@sherifnada
Copy link
Contributor

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Mar 22, 2021

/test connector=source-mysql ref=master

🕑 source-mysql https://github.com/airbytehq/airbyte/actions/runs/677428374
❌ source-mysql https://github.com/airbytehq/airbyte/actions/runs/677428374

@@ -11,8 +13,14 @@ if [[ "$connector" == "all" ]] ; then
echo "Running: ./gradlew --no-daemon --scan integrationTest"
./gradlew --no-daemon --scan integrationTest
else
selected_integration_test=$(echo "$all_integration_tests" | grep "^$connector$" || echo "")
integrationTestCommand=":airbyte-integrations:connectors:$connector:integrationTest"
if [[ "$connector" == *"connectors"* ]] || [[ "$connector" == *"bases"* ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this conditional? can't we just say you should always do it one way?

if i'm interpreting right it seems like you can do: connectors/source-postgres or source-postgres? i think it's the else that feels a little iffy, because i think it'll work for source-postgres but it won't work for base-normalization since :connectors:is hard coded.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition makes it so that it keeps both the old behavior and the new one.

You are right, the old behavior (which is source-postgres) won't work with base-normalization as you pointed out

However, I don't understand yet why the new behavior gets rejected when used with the ref=XXX arguments:
#2531 (comment)

but the old behavior does pass validation: #2531 (comment)

So since the new behavior doesn't fully replace the old one yet in terms of functionalities, I am leaving both options available...

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Mar 23, 2021

/test connector=connectors/source-mysql ref=master

❌ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/679058756

@ChristopheDuong ChristopheDuong merged commit 4bebeed into master Mar 23, 2021
@ChristopheDuong ChristopheDuong deleted the chris/test-github-ci branch March 23, 2021 09:47
@davinchia
Copy link
Contributor

I'm late to this; but this is very cool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants