Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove dockerfiles for certified python connnectors and use the base image #31377

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Oct 13, 2023

Closes #30243

What

  • Make all our certified connector use our latest python base image
  • Remove their Dockerfile
  • Update their documentation

How

Run airbyte-ci connectors --support-level=certified migrate-to-base-image 31377
(#30520)

Roll out plan

  1. Merge all connectors with 🟢 tests first: I'll remove the changes of failing connector tests from this branch and will merge
  2. Enable the new CI check which fail when no baseImage is declared on a certified connector

@vercel
Copy link

vercel bot commented Oct 13, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 16, 2023 7:53am

Copy link
Contributor Author

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 13, 2023

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
  • If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@airbyte-oss-build-runner
Copy link
Collaborator

source-slack test report (commit 50a636f4ae) - ✅

⏲️ Total pipeline duration: 80mn34s

Step Result
Build source-slack docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-slack
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-slack test

@airbyte-oss-build-runner
Copy link
Collaborator

source-salesforce test report (commit 50a636f4ae) - ❌

⏲️ Total pipeline duration: 113mn14s

Step Result
Build source-salesforce docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for source-salesforce
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

@airbyte-oss-build-runner
Copy link
Collaborator

source-file test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 01mn48s

Step Result
Build source-file docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for source-file
Connector version semver check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-file test

@airbyte-oss-build-runner
Copy link
Collaborator

source-google-sheets test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 03mn17s

Step Result
Build source-google-sheets docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-google-sheets
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-google-sheets test

@airbyte-oss-build-runner
Copy link
Collaborator

source-notion test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 03mn52s

Step Result
Build source-notion docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-notion
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

@airbyte-oss-build-runner
Copy link
Collaborator

source-surveymonkey test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 04mn34s

Step Result
Build source-surveymonkey docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-surveymonkey
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-surveymonkey test

@airbyte-oss-build-runner
Copy link
Collaborator

source-intercom test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 03mn04s

Step Result
Build source-intercom docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for source-intercom
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-intercom test

@airbyte-oss-build-runner
Copy link
Collaborator

destination-weaviate test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 01mn03s

Step Result
Build destination-weaviate docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for destination-weaviate
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-weaviate test

@airbyte-oss-build-runner
Copy link
Collaborator

destination-milvus test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 01mn24s

Step Result
Build destination-milvus docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for destination-milvus
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-milvus test

@airbyte-oss-build-runner
Copy link
Collaborator

source-zendesk-chat test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 02mn45s

Step Result
Build source-zendesk-chat docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-zendesk-chat
Connector version semver check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-zendesk-chat test

@airbyte-oss-build-runner
Copy link
Collaborator

source-sendgrid test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 04mn38s

Step Result
Build source-sendgrid docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-sendgrid
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-sendgrid test

@airbyte-oss-build-runner
Copy link
Collaborator

source-typeform test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 01mn38s

Step Result
Build source-typeform docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-typeform
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-typeform test

@airbyte-oss-build-runner
Copy link
Collaborator

source-airtable test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 02mn45s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-airtable test

@airbyte-oss-build-runner
Copy link
Collaborator

source-paypal-transaction test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 02mn08s

Step Result
Build source-paypal-transaction docker image for platform(s) linux/x86_64
Acceptance tests
Code format checks
Validate metadata for source-paypal-transaction
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-paypal-transaction test

@airbyte-oss-build-runner
Copy link
Collaborator

source-zendesk-support test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 08mn27s

Step Result
Build source-zendesk-support docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-zendesk-support
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-zendesk-support test

@airbyte-oss-build-runner
Copy link
Collaborator

source-chargebee test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 02mn34s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-chargebee test

@airbyte-oss-build-runner
Copy link
Collaborator

source-google-ads test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 11mn39s

Step Result
Build source-google-ads docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-google-ads
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-google-ads test

@airbyte-oss-build-runner
Copy link
Collaborator

source-bing-ads test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 22mn42s

Step Result
Build source-bing-ads docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-bing-ads
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-bing-ads test

@airbyte-oss-build-runner
Copy link
Collaborator

source-amazon-ads test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 02mn54s

Step Result
Build source-amazon-ads docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-amazon-ads
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-amazon-ads test

@airbyte-oss-build-runner
Copy link
Collaborator

source-google-analytics-v4 test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 45mn19s

Step Result
Build source-google-analytics-v4 docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-google-analytics-v4
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-google-analytics-v4 test

@airbyte-oss-build-runner
Copy link
Collaborator

destination-pinecone test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 01mn31s

Step Result
Build destination-pinecone docker image for platform(s) linux/x86_64
Unit tests
Integration tests
Acceptance tests
Code format checks
Validate metadata for destination-pinecone
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-pinecone test

@airbyte-oss-build-runner
Copy link
Collaborator

source-gitlab test report (commit 0030b79f43) - ❌

⏲️ Total pipeline duration: 10mn54s

Step Result
Build source-gitlab docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-gitlab
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-gitlab test

@airbyte-oss-build-runner
Copy link
Collaborator

source-slack test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 79mn04s

Step Result
Build source-slack docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-slack
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-slack test

@airbyte-oss-build-runner
Copy link
Collaborator

source-mixpanel test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 67mn03s

Step Result
Build source-mixpanel docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-mixpanel
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-mixpanel test

@airbyte-oss-build-runner
Copy link
Collaborator

source-google-analytics-data-api test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 86mn29s

Step Result
Build source-google-analytics-data-api docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-google-analytics-data-api
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-google-analytics-data-api test

@airbyte-oss-build-runner
Copy link
Collaborator

source-marketo test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 70mn29s

Step Result
Build source-marketo docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-marketo
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-marketo test

@airbyte-oss-build-runner
Copy link
Collaborator

source-harvest test report (commit 0030b79f43) - ✅

⏲️ Total pipeline duration: 37mn24s

Step Result
Build source-harvest docker image for platform(s) linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate metadata for source-harvest
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-harvest test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/destination/milvus connectors/destination/pinecone connectors/destination/weaviate connectors/source/airtable connectors/source/amazon-ads connectors/source/amplitude connectors/source/bing-ads connectors/source/chargebee connectors/source/facebook-marketing connectors/source/file connectors/source/file-secure connectors/source/freshdesk connectors/source/github connectors/source/gitlab connectors/source/google-ads connectors/source/google-analytics-data-api connectors/source/google-analytics-v4 connectors/source/google-search-console connectors/source/google-sheets connectors/source/greenhouse connectors/source/harvest connectors/source/hubspot connectors/source/instagram connectors/source/intercom connectors/source/iterable connectors/source/jira connectors/source/klaviyo connectors/source/linkedin-ads connectors/source/mailchimp connectors/source/marketo connectors/source/mixpanel connectors/source/monday connectors/source/notion connectors/source/openweather connectors/source/paypal-transaction connectors/source/pinterest connectors/source/recharge connectors/source/s3 connectors/source/salesforce connectors/source/sendgrid connectors/source/sentry connectors/source/shopify connectors/source/slack connectors/source/snapchat-marketing connectors/source/stripe connectors/source/surveymonkey connectors/source/tiktok-marketing connectors/source/twilio connectors/source/typeform connectors/source/zendesk-chat connectors/source/zendesk-support connectors/source/zendesk-talk
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove dockerfiles for certified connectors and build them using our new build process
3 participants