Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SAT: skip backward compatibility tests on spec/catalog if previous and actual spec/catalog are identical #20435

Merged
merged 5 commits into from
Dec 14, 2022
Merged

SAT: skip backward compatibility tests on spec/catalog if previous and actual spec/catalog are identical #20435

merged 5 commits into from
Dec 14, 2022

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Dec 13, 2022 •
edited
Loading

What

We don't need to run backward compatibility tests - that can be long-running - if the current and previous connector spec /catalog are identical.

Unblocking #20237 (comment)

How

Skip spec backward compatibility tests if the current and previous connector spec are identical.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 13, 2022 •
edited
Loading

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

❌ Sources (64)
Connector Version Changelog Publish
source-airtable 0.1.3 ✅ ✅
source-amazon-ads 0.1.25 ✅ ✅
source-amazon-seller-partner 0.2.29 ✅ ✅
source-amazon-sqs 0.1.0 ✅ ✅
source-amplitude 0.1.19 ✅ ✅
source-appsflyer 0.1.0 ✅ ✅
source-asana 0.1.5 ✅ ✅
source-azure-table 0.1.3 ✅ ✅
source-braintree 0.1.3 ✅ ✅
source-cart 0.2.0 ✅ ✅
source-chargebee 0.1.16 ✅ ✅
source-commercetools 0.1.0 ✅ ✅
source-confluence 0.1.1 ✅ ✅
source-datadog 0.1.0 ✅ ✅
source-delighted 0.2.0 ✅ ✅
source-drift 0.2.5 ✅ ✅
source-facebook-marketing 0.2.78 ✅ ✅
source-facebook-pages 0.1.6 ✅ ✅
source-freshcaller 0.1.0 ✅ ✅
source-freshsales 0.1.2 ✅ ✅
source-freshservice 0.1.1 ✅ ✅
source-github 0.3.8 ✅ ✅
source-gitlab 0.1.10 ✅ ✅
source-google-ads 0.2.5 ✅ ✅
source-google-search-console 0.1.18 ✅ ✅
source-greenhouse 0.3.0 ✅ ✅
source-harvest 0.1.11 ✅ ✅
source-instagram 1.0.0 ✅ ✅
source-iterable 0.1.22 ✅ ✅
source-klaviyo 0.1.10 ✅ ✅
source-lemlist 0.1.1 ✅ ✅
source-lever-hiring 0.1.3 ✅ ✅
source-linnworks 0.1.5 ✅ ✅
source-mailchimp 0.3.0 ✅ ✅
source-mailgun 0.1.0 ✅ ✅
source-monday 0.2.0 ✅ ✅
source-notion 0.1.10 ✅ ✅
source-okta 0.1.13 ✅ ✅
source-onesignal 0.1.2 ✅ ✅
source-openweather 0.1.6 ✅ ✅
source-outreach 0.1.2 ✅ ✅
source-pardot 0.1.0 ✅ ✅
source-paystack 0.1.1 ✅ ✅
source-pinterest 0.2.0 ✅ ✅
source-pipedrive 0.1.13 ✅ ✅
source-plaid 0.3.2 ❌
(changelog missing)		 ✅
source-posthog 0.1.7 ✅ ✅
source-prestashop 0.3.0 ✅ ✅
source-quickbooks-singer 0.1.5 ⚠
(doc not found)		 ✅
source-recharge 0.2.4 ✅ ✅
source-retently 0.1.2 ✅ ✅
source-salesforce 1.0.27 ✅ ✅
source-salesloft 0.1.3 ✅ ✅
source-sendgrid 0.2.16 ✅ ✅
source-sentry 0.1.7 ✅ ✅
source-strava 0.1.2 ✅ ✅
source-surveymonkey 0.1.13 ✅ ✅
source-tplcentral 0.1.1 ✅ ✅
source-twilio 0.1.14 ✅ ✅
source-weatherstack 0.1.0 ✅ ✅
source-youtube-analytics 0.1.3 ✅ ✅
source-zendesk-sunshine 0.1.1 ✅ ✅
source-zendesk-talk 0.1.5 ✅ ✅
source-zenloop 0.1.4 ✅ ✅
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)
Connector Version Changelog Publish
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)
Category Status Actionable Item
Version ❌
mismatch		 The version of the connector is different from its normal variant. Please bump the version of the connector.
⚠
doc not found		 The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog ⚠
doc not found		 The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
❌
changelog missing		 There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish ⚠
not in seed		 The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
❌
diff seed version		 The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@alafanechere alafanechere marked this pull request as ready for review December 13, 2022 15:56
@alafanechere alafanechere mentioned this pull request Dec 13, 2022
Merged
@alafanechere alafanechere requested review from bazarnov and a team December 13, 2022 15:58
davydov-d
davydov-d approved these changes Dec 13, 2022
View reviewed changes
Copy link
Collaborator

@davydov-d davydov-d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@alafanechere alafanechere changed the title SAT: skip backward compatibility tests on spec if previous and actual spec are identical SAT: skip backward compatibility tests on spec/catalog if previous and actual spec/catalog are identical Dec 13, 2022
@alafanechere
Copy link
Contributor Author

alafanechere commented Dec 13, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-facebook-marketing

🕑 connectors/source-facebook-marketing https://github.com/airbytehq/airbyte/actions/runs/3687365468
✅ connectors/source-facebook-marketing https://github.com/airbytehq/airbyte/actions/runs/3687365468
Python tests coverage:

Name                                                        Stmts   Miss  Cover
-------------------------------------------------------------------------------
source_facebook_marketing/streams/__init__.py                   2      0   100%
source_facebook_marketing/__init__.py                           2      0   100%
source_facebook_marketing/spec.py                              41      1    98%
source_facebook_marketing/api.py                               96     12    88%
source_facebook_marketing/streams/base_streams.py             138     27    80%
source_facebook_marketing/source.py                            53     18    66%
source_facebook_marketing/streams/common.py                    44     16    64%
source_facebook_marketing/utils.py                             24      9    62%
source_facebook_marketing/streams/streams.py                  119     46    61%
source_facebook_marketing/streams/base_insight_streams.py     145     64    56%
source_facebook_marketing/streams/async_job.py                217    136    37%
source_facebook_marketing/streams/async_job_manager.py         78     60    23%
-------------------------------------------------------------------------------
TOTAL                                                         959    389    59%
Name                                                        Stmts   Miss  Cover
-------------------------------------------------------------------------------
source_facebook_marketing/utils.py                             24      0   100%
source_facebook_marketing/streams/common.py                    44      0   100%
source_facebook_marketing/streams/async_job.py                217      0   100%
source_facebook_marketing/streams/__init__.py                   2      0   100%
source_facebook_marketing/spec.py                              41      0   100%
source_facebook_marketing/__init__.py                           2      0   100%
source_facebook_marketing/streams/async_job_manager.py         78      3    96%
source_facebook_marketing/api.py                               96      9    91%
source_facebook_marketing/source.py                            53      5    91%
source_facebook_marketing/streams/base_insight_streams.py     145     16    89%
source_facebook_marketing/streams/base_streams.py             138     30    78%
source_facebook_marketing/streams/streams.py                  119     27    77%
-------------------------------------------------------------------------------
TOTAL                                                         959     90    91%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       140      5    96%   87, 93, 238, 242-243
	 source_acceptance_test/conftest.py                     208     92    56%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-280, 288-301, 306-312, 319-330, 337-353
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1603    336    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:377: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_incremental.py:200: Skipping new incremental test based on acceptance-test-config.yml
============ 28 passed, 3 skipped, 31 warnings in 760.44s (0:12:40) ============

@alafanechere
Copy link
Contributor Author

alafanechere commented Dec 13, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-instagram

🕑 connectors/source-instagram https://github.com/airbytehq/airbyte/actions/runs/3687368968
✅ connectors/source-instagram https://github.com/airbytehq/airbyte/actions/runs/3687368968
Python tests coverage:

Name                           Stmts   Miss  Cover
--------------------------------------------------
source_instagram/__init__.py       2      0   100%
source_instagram/api.py           59      5    92%
source_instagram/source.py        31     11    65%
source_instagram/streams.py      203     93    54%
source_instagram/common.py        36     22    39%
--------------------------------------------------
TOTAL                            331    131    60%
Name                           Stmts   Miss  Cover
--------------------------------------------------
source_instagram/source.py        31      0   100%
source_instagram/__init__.py       2      0   100%
source_instagram/common.py        36      3    92%
source_instagram/streams.py      203     19    91%
source_instagram/api.py           59      7    88%
--------------------------------------------------
TOTAL                            331     29    91%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       140      5    96%   87, 93, 238, 242-243
	 source_acceptance_test/conftest.py                     208     92    56%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-280, 288-301, 306-312, 319-330, 337-353
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1603    336    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:377: The previous and actual discovered catalogs are identical.
============ 27 passed, 2 skipped, 29 warnings in 401.81s (0:06:41) ============

bazarnov
bazarnov approved these changes Dec 13, 2022
View reviewed changes
Copy link
Collaborator

@bazarnov bazarnov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what there should be from the start! Thanks!

@alafanechere
Copy link
Contributor Author

alafanechere commented Dec 14, 2022 •
edited by github-actions bot
Loading

/publish connector=bases/source-acceptance-test auto-bump-version=false

🕑 Publishing the following connectors:
bases/source-acceptance-test
https://github.com/airbytehq/airbyte/actions/runs/3692786512


Connector Did it publish? Were definitions generated?
bases/source-acceptance-test ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@alafanechere alafanechere merged commit 4017c0a into master Dec 14, 2022
@alafanechere alafanechere deleted the augustin/sat/improve-backward-compat branch December 14, 2022 09:12
@github-actions github-actions bot mentioned this pull request Dec 15, 2022
Closed
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Dec 18, 2022
Merged
@alafanechere alafanechere mentioned this pull request Dec 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sherifnada sherifnada sherifnada approved these changes

@davydov-d davydov-d davydov-d approved these changes

@erohmensing erohmensing erohmensing approved these changes

@bazarnov bazarnov bazarnov approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@alafanechere @sherifnada @davydov-d @erohmensing @bazarnov