Source Amazon Ads: get rid of fail_on_extra_columns: false in SAT
#25913
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
octavia-squidington-iii
added
area/connectors
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
|
/test connector=connectors/source-amazon-ads
Build FailedTest summary info: |
|
/test connector=connectors/source-amazon-ads
Build PassedTest summary info: |
lazebnyi
approved these changes
May 10, 2023
| @@ -36,6 +36,9 @@ def schema_extra(cls, schema: Dict[str, Any], model: Type["BaseModel"]) -> None: | |||
| if "type" in prop: | |||
| if allow_none: | |||
| prop["type"] = ["null", prop["type"]] | |||
| if prop["type"] == "array" and prop["items"]: | |||
There was a problem hiding this comment.
let's add unit tests to check this functionality.
|
@roman-yermilov-gl can you please lay out what changes are made and why, in the PR description? Will remove my request until then so it doesn't show up in my list, but feel free to rerequest |
roman-yermilov-gl
changed the title
fail_on_extra_columns to false
roman-yermilov-gl
changed the title
fail_on_extra_columns to falsefail_on_extra_columns: false in SAT
|
/test connector=connectors/source-amazon-ads
Build PassedTest summary info: |
|
/publish connector=connectors/source-amazon-ads
if you have connectors that successfully published but failed definition generation, follow step 4 here |
roman-yermilov-gl
force-pushed
the
ryermilov/source-amazon-ads-fix-schema
branch
from
1fca5bf
to
0448776
Compare
|
/publish connector=connectors/source-amazon-ads
if you have connectors that successfully published but failed definition generation, follow step 4 here |
roman-yermilov-gl
force-pushed
the
ryermilov/source-amazon-ads-fix-schema
branch
from
0448776
to
3bb4548
Compare
|
/publish connector=connectors/source-amazon-ads
if you have connectors that successfully published but failed definition generation, follow step 4 here |
bnchrch
pushed a commit
that referenced
this pull request
May 16, 2023
…25913) * Source Amazon Ads: small schema fixes * Source Amazon Ads: update changelog * Source Amazon Ads: update unittest * Source Amazon Ads: unittest additional property is boolean * Source Amazon Ads: bump version * auto-bump connector version --------- Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
bnchrch
added a commit
that referenced
this pull request
May 16, 2023
* Remove Definition File References from Python files (#25590) * Remove check_images_exist.sh * Update definitions.py * Update build_report.py * Update tools/bin/ci_integration_workflow_launcher.py * Update tools/bin/ci_check_dependency.py * tools/bin/scan_all_spec_airbyte_secret.py * Remove qa engine references * Revert "Remove check_images_exist.sh" This reverts commit 7675162. * Improve get url function * Add test * remove scan_all_spec_airbyte_secret.py * add additional test * Remove check_images_exist.sh (#25593) * Remove Definition File References from Java files (LocalDefinitionProvider), shell scripts and docs (#25592) * Remove CombinedConnectorCatalogGenerator.java * Update local definition provider * Update local def test * Add spec mask downloader * Make downloader work * Delete generators and add tests * REMOVE THE YAML FILES * Roughly update docs * Update shell scripts * Remove unused * Add connector metadata file doc * Apply suggestions from code review Co-authored-by: Augustin <augustin@airbyte.io> * Additional PR comments * Run format tasks --------- Co-authored-by: Augustin <augustin@airbyte.io> * Remove unused import * bundle registry * Ignore future updates * Update registry * new registry * Readd maskeddatainterceptor * Automated Change * Remove icon validation * Automated Change * Automated Change * Source Amazon Ads: get rid of `fail_on_extra_columns: false` in SAT (#25913) * Source Amazon Ads: small schema fixes * Source Amazon Ads: update changelog * Source Amazon Ads: update unittest * Source Amazon Ads: unittest additional property is boolean * Source Amazon Ads: bump version * auto-bump connector version --------- Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * connectors-ci: make spec-cache / metadata bucket and creds not required for pre-release (#26119) * Automated Change --------- Co-authored-by: Augustin <augustin@airbyte.io> Co-authored-by: bnchrch <bnchrch@users.noreply.github.com> Co-authored-by: Roman Yermilov [GL] <86300758+roman-yermilov-gl@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
marcosmarxm
pushed a commit
to natalia-miinto/airbyte
that referenced
this pull request
Jun 8, 2023
…irbytehq#25913) * Source Amazon Ads: small schema fixes * Source Amazon Ads: update changelog * Source Amazon Ads: update unittest * Source Amazon Ads: unittest additional property is boolean * Source Amazon Ads: bump version * auto-bump connector version --------- Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
marcosmarxm
pushed a commit
to natalia-miinto/airbyte
that referenced
this pull request
Jun 8, 2023
* Remove Definition File References from Python files (airbytehq#25590) * Remove check_images_exist.sh * Update definitions.py * Update build_report.py * Update tools/bin/ci_integration_workflow_launcher.py * Update tools/bin/ci_check_dependency.py * tools/bin/scan_all_spec_airbyte_secret.py * Remove qa engine references * Revert "Remove check_images_exist.sh" This reverts commit 7675162. * Improve get url function * Add test * remove scan_all_spec_airbyte_secret.py * add additional test * Remove check_images_exist.sh (airbytehq#25593) * Remove Definition File References from Java files (LocalDefinitionProvider), shell scripts and docs (airbytehq#25592) * Remove CombinedConnectorCatalogGenerator.java * Update local definition provider * Update local def test * Add spec mask downloader * Make downloader work * Delete generators and add tests * REMOVE THE YAML FILES * Roughly update docs * Update shell scripts * Remove unused * Add connector metadata file doc * Apply suggestions from code review Co-authored-by: Augustin <augustin@airbyte.io> * Additional PR comments * Run format tasks --------- Co-authored-by: Augustin <augustin@airbyte.io> * Remove unused import * bundle registry * Ignore future updates * Update registry * new registry * Readd maskeddatainterceptor * Automated Change * Remove icon validation * Automated Change * Automated Change * Source Amazon Ads: get rid of `fail_on_extra_columns: false` in SAT (airbytehq#25913) * Source Amazon Ads: small schema fixes * Source Amazon Ads: update changelog * Source Amazon Ads: update unittest * Source Amazon Ads: unittest additional property is boolean * Source Amazon Ads: bump version * auto-bump connector version --------- Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * connectors-ci: make spec-cache / metadata bucket and creds not required for pre-release (airbytehq#26119) * Automated Change --------- Co-authored-by: Augustin <augustin@airbyte.io> Co-authored-by: bnchrch <bnchrch@users.noreply.github.com> Co-authored-by: Roman Yermilov [GL] <86300758+roman-yermilov-gl@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Amazon Ads passes CAT without declaring
fail_on_extra_columns: falseHow
The following descriptions of streams that pass undeclared columns come from results of the failed connector acceptance test:
Additional properties are not allowed ('<column>', 'column' were unexpected)logs to the connector's spec.fail_on_extra_columns: falsefrom theacceptance-test-config.ymlfile.acceptance-test-config.ymland open a PR./testcommand