Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add call_details stream #49

Merged
merged 8 commits into from
Apr 22, 2022
Merged

Add call_details stream #49

merged 8 commits into from
Apr 22, 2022

Conversation

bryantgray
Copy link
Contributor

Description of change

  • Remove segments from core streams
    • campaign_budgets is the only stream affected; removed the segment campaign_id
    • Any campaign_budgets not linked to a campaign will now be returned
  • Add call_details core stream

QA steps

  • automated tests passing
  • manual qa steps passing

Risks

  • Users who had selected the campaign_budgets stream will no longer receive campaign_id

Rollback steps

  • revert this branch

Copy link
Contributor

@kspeer825 kspeer825 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment request for the tests, very minor

tests/test_google_ads_automatic_fields.py Outdated Show resolved Hide resolved
tests/test_google_ads_bookmarks.py Outdated Show resolved Hide resolved
Copy link
Contributor

@kspeer825 kspeer825 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are good 👍

@dsprayberry dsprayberry merged commit 06c7f3e into main Apr 22, 2022
@dsprayberry dsprayberry deleted the add-core-streams branch April 22, 2022 16:10
@dsprayberry dsprayberry mentioned this pull request Apr 22, 2022
2 tasks
lideke added a commit to sendinblue/tap-google-ads that referenced this pull request Aug 2, 2022
* Bump to v1.0.0, update changelog (singer-io#45)

* Qa/future testing (singer-io#48)

* documented reasons for untested streams

* list high level scenarios for manual qa checks

* remove stitch specific pr template

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Add call_details stream (singer-io#49)

* Add call_view core stream, filter non-attribute fields from core streams

* Change `call_view` stream name to `call_details`

* Make pylint happy

* Fix integration tests

* Remove campaign_id from foreign keys, split call_details foreign keys

* Update tests to exclude call_details as needed

* Add context around excluding call_details from tests

* Remove outdated TODO related to addition of call_details stream.

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53)

* Add core LABELS streams and campaign.labels fields to relevant reports

* Update test exclusions to exclude new core streams.

* Update discover to include campaign_label in reports list

* Update foreign_key expected metadata to include attributed resource foreign_keys

* Update tests to re-include campaign_labels and labels as we now have test data.

* Handles case where state does not have currently_syncing (singer-io#54)

* Handles case where state does not have currently_syncing

* tap-tester test added

* takeout unused imports in test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Version bump and changelog entry (singer-io#52)

* Version bump and changelog entry

* Update to include PR 53.

* Add PR 54 to changelog

* Implement Automatic Keys (singer-io#55)

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Update setup.py with docs explaining required but unused dep

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Cleanup (singer-io#56)

* Remove useless function

* Rename "REPORTS" to "STREAMS" for accuracy / readability

* Version bump for PRs 56 and 55 (singer-io#57)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* [Feature] Add more core streams (singer-io#58)

* WIP add new core streams

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* WIP remove IPDB and try except

* Fix report_definition typo and add handling for user_interest_id field

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Remove ipdb -_-

* Update setup.py with docs explaining required but unused dep

* WIP w/ failing tests and attributed_resource foreign_keys

* WIP with failing tests; remove extraneous attributed_resoruce automatic fields

* Create UserInterestStream class to handle its edge case

* Add transform_keys to UserInterestStream class; start test updates

* Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys

* update auto fields test to account for compound pks

* Fix campaigns typo

* Remove exclusion of feed and feed items from sync canary test

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: atribed <agorka@talend.com>

* V1.3.0 (singer-io#59)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* Version bump for 1.3.0

Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Qa/fix build notification (singer-io#65)

* fix slack notif for build

* remove click_performance_report from tests

* remove click_performance_report from sync canary test

* run only streams that are untested in canary sync

* just skip canaray test

* put back the assert in the skipped test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Crest master (singer-io#66)

* Tdl 19235 handle uncaught exceptions (singer-io#61)

* Added backoff for 5xx, 429 and ReadTimeout errors.

* Resolved pylint error.

* Updated comments in the unittest cases.

* Updated error handling.

* TDL-18749 Implement interruptible full table streams. (singer-io#60)

* Implemented interruptible full table streams.

* Resolved pylint error

* Resolved error in full table sync test case.

* Updated config.yml to pass cci

* Updated query building logic.

* Updated integration test case.

* Resolved review comments.

* Resolved comments.

* Implemeted logic to skip the duplicate records.

* Resolved unittest case error.

* Resolved pylint error

* Resolved integration test case error

* Added empty filter param for call_details and campaign_label stream.

* Added unit test cases for should_sync method.

* Revert "Implemeted logic to skip the duplicate records."

This reverts commit cd06e11.

* Added logger message for debugging purpose

* Updated integration test case.

* Replaced .format with f string.

* Updated comment in integration test.

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* Bump version (singer-io#67)

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* TDL-18524 updated readme and added sample config (singer-io#51)

* updated readme and added sample config

* updated endpoints

* add new streams

* resolved PR comments

* fixed a typo

* added the

* Add timeout parameter to gas.search (singer-io#64)

* Add timeout parameter to search gas.search

* Increase timeout to 15 minutes for safety.

* Add get_request_timeout function, add config to make_request as needed

* Update on_giveup to raise specific exception text for timeoutexception class; add unit test

* Make Pylint happy take 1

* Update make_request signature in unittests.

* Another Unittest update

* More unittest fixes for signature

* Add default config param value for ease of implementation in future tests

* Fix pylint dangerous-defaul-value

* Fix stupid error

* Version bump and changelog (singer-io#70)

* TDL-19486 Add limit clause to core stream queries (singer-io#68)

* Initial commit for add page limit.

* Added limit parameter in sync method of ReportStream class.

* Fixed issue for call_details stream.

* Resolved unit test case error.

* Added test cases.

* Updated code comments.

* Fixed keyerror issue.

* Updated default query limit.

* Updated pagination test case.

* Modify config name to be more explicit.

* Update comment for accuracy.

* Update property name in base and propogate name change to tests

* Exclude feed from streams_to_test because of lack of data

* Committing stuff from main that should have already been in

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Version Bump and Changelog Update (singer-io#72)

* Reintroduce unintentionally removed files. (singer-io#73)

* add schemaless

* fix schemaless config

* update login_customer_ids

* fix missing key from dict

* add google ads api version in config
add log to show the version

* use version from config

* use version global

Co-authored-by: bryantgray <bgray@talend.com>
Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>
Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: atribed <agorka@talend.com>
Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com>
Co-authored-by: KrishnanG <kgurusamy@talend.com>
Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com>
Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
lideke added a commit to sendinblue/tap-google-ads that referenced this pull request Aug 2, 2022
* Bump to v1.0.0, update changelog (singer-io#45)

* Qa/future testing (singer-io#48)

* documented reasons for untested streams

* list high level scenarios for manual qa checks

* remove stitch specific pr template

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Add call_details stream (singer-io#49)

* Add call_view core stream, filter non-attribute fields from core streams

* Change `call_view` stream name to `call_details`

* Make pylint happy

* Fix integration tests

* Remove campaign_id from foreign keys, split call_details foreign keys

* Update tests to exclude call_details as needed

* Add context around excluding call_details from tests

* Remove outdated TODO related to addition of call_details stream.

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53)

* Add core LABELS streams and campaign.labels fields to relevant reports

* Update test exclusions to exclude new core streams.

* Update discover to include campaign_label in reports list

* Update foreign_key expected metadata to include attributed resource foreign_keys

* Update tests to re-include campaign_labels and labels as we now have test data.

* Handles case where state does not have currently_syncing (singer-io#54)

* Handles case where state does not have currently_syncing

* tap-tester test added

* takeout unused imports in test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Version bump and changelog entry (singer-io#52)

* Version bump and changelog entry

* Update to include PR 53.

* Add PR 54 to changelog

* Implement Automatic Keys (singer-io#55)

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Update setup.py with docs explaining required but unused dep

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Cleanup (singer-io#56)

* Remove useless function

* Rename "REPORTS" to "STREAMS" for accuracy / readability

* Version bump for PRs 56 and 55 (singer-io#57)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* [Feature] Add more core streams (singer-io#58)

* WIP add new core streams

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* WIP remove IPDB and try except

* Fix report_definition typo and add handling for user_interest_id field

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Remove ipdb -_-

* Update setup.py with docs explaining required but unused dep

* WIP w/ failing tests and attributed_resource foreign_keys

* WIP with failing tests; remove extraneous attributed_resoruce automatic fields

* Create UserInterestStream class to handle its edge case

* Add transform_keys to UserInterestStream class; start test updates

* Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys

* update auto fields test to account for compound pks

* Fix campaigns typo

* Remove exclusion of feed and feed items from sync canary test

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: atribed <agorka@talend.com>

* V1.3.0 (singer-io#59)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* Version bump for 1.3.0

Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Qa/fix build notification (singer-io#65)

* fix slack notif for build

* remove click_performance_report from tests

* remove click_performance_report from sync canary test

* run only streams that are untested in canary sync

* just skip canaray test

* put back the assert in the skipped test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Crest master (singer-io#66)

* Tdl 19235 handle uncaught exceptions (singer-io#61)

* Added backoff for 5xx, 429 and ReadTimeout errors.

* Resolved pylint error.

* Updated comments in the unittest cases.

* Updated error handling.

* TDL-18749 Implement interruptible full table streams. (singer-io#60)

* Implemented interruptible full table streams.

* Resolved pylint error

* Resolved error in full table sync test case.

* Updated config.yml to pass cci

* Updated query building logic.

* Updated integration test case.

* Resolved review comments.

* Resolved comments.

* Implemeted logic to skip the duplicate records.

* Resolved unittest case error.

* Resolved pylint error

* Resolved integration test case error

* Added empty filter param for call_details and campaign_label stream.

* Added unit test cases for should_sync method.

* Revert "Implemeted logic to skip the duplicate records."

This reverts commit cd06e11.

* Added logger message for debugging purpose

* Updated integration test case.

* Replaced .format with f string.

* Updated comment in integration test.

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* Bump version (singer-io#67)

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* TDL-18524 updated readme and added sample config (singer-io#51)

* updated readme and added sample config

* updated endpoints

* add new streams

* resolved PR comments

* fixed a typo

* added the

* Add timeout parameter to gas.search (singer-io#64)

* Add timeout parameter to search gas.search

* Increase timeout to 15 minutes for safety.

* Add get_request_timeout function, add config to make_request as needed

* Update on_giveup to raise specific exception text for timeoutexception class; add unit test

* Make Pylint happy take 1

* Update make_request signature in unittests.

* Another Unittest update

* More unittest fixes for signature

* Add default config param value for ease of implementation in future tests

* Fix pylint dangerous-defaul-value

* Fix stupid error

* Version bump and changelog (singer-io#70)

* TDL-19486 Add limit clause to core stream queries (singer-io#68)

* Initial commit for add page limit.

* Added limit parameter in sync method of ReportStream class.

* Fixed issue for call_details stream.

* Resolved unit test case error.

* Added test cases.

* Updated code comments.

* Fixed keyerror issue.

* Updated default query limit.

* Updated pagination test case.

* Modify config name to be more explicit.

* Update comment for accuracy.

* Update property name in base and propogate name change to tests

* Exclude feed from streams_to_test because of lack of data

* Committing stuff from main that should have already been in

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Version Bump and Changelog Update (singer-io#72)

* Reintroduce unintentionally removed files. (singer-io#73)

* add schemaless

* fix schemaless config

* update login_customer_ids

* fix missing key from dict

* add google ads api version in config
add log to show the version

* use version from config

* use version global

Co-authored-by: bryantgray <bgray@talend.com>
Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>
Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: atribed <agorka@talend.com>
Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com>
Co-authored-by: KrishnanG <kgurusamy@talend.com>
Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com>
Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
lideke added a commit to sendinblue/tap-google-ads that referenced this pull request Mar 22, 2023
* Bump to v1.0.0, update changelog (singer-io#45)

* Qa/future testing (singer-io#48)

* documented reasons for untested streams

* list high level scenarios for manual qa checks

* remove stitch specific pr template

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Add call_details stream (singer-io#49)

* Add call_view core stream, filter non-attribute fields from core streams

* Change `call_view` stream name to `call_details`

* Make pylint happy

* Fix integration tests

* Remove campaign_id from foreign keys, split call_details foreign keys

* Update tests to exclude call_details as needed

* Add context around excluding call_details from tests

* Remove outdated TODO related to addition of call_details stream.

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53)

* Add core LABELS streams and campaign.labels fields to relevant reports

* Update test exclusions to exclude new core streams.

* Update discover to include campaign_label in reports list

* Update foreign_key expected metadata to include attributed resource foreign_keys

* Update tests to re-include campaign_labels and labels as we now have test data.

* Handles case where state does not have currently_syncing (singer-io#54)

* Handles case where state does not have currently_syncing

* tap-tester test added

* takeout unused imports in test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Version bump and changelog entry (singer-io#52)

* Version bump and changelog entry

* Update to include PR 53.

* Add PR 54 to changelog

* Implement Automatic Keys (singer-io#55)

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Update setup.py with docs explaining required but unused dep

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Cleanup (singer-io#56)

* Remove useless function

* Rename "REPORTS" to "STREAMS" for accuracy / readability

* Version bump for PRs 56 and 55 (singer-io#57)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* [Feature] Add more core streams (singer-io#58)

* WIP add new core streams

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* WIP remove IPDB and try except

* Fix report_definition typo and add handling for user_interest_id field

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Remove ipdb -_-

* Update setup.py with docs explaining required but unused dep

* WIP w/ failing tests and attributed_resource foreign_keys

* WIP with failing tests; remove extraneous attributed_resoruce automatic fields

* Create UserInterestStream class to handle its edge case

* Add transform_keys to UserInterestStream class; start test updates

* Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys

* update auto fields test to account for compound pks

* Fix campaigns typo

* Remove exclusion of feed and feed items from sync canary test

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: atribed <agorka@talend.com>

* V1.3.0 (singer-io#59)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* Version bump for 1.3.0

Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Qa/fix build notification (singer-io#65)

* fix slack notif for build

* remove click_performance_report from tests

* remove click_performance_report from sync canary test

* run only streams that are untested in canary sync

* just skip canaray test

* put back the assert in the skipped test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Crest master (singer-io#66)

* Tdl 19235 handle uncaught exceptions (singer-io#61)

* Added backoff for 5xx, 429 and ReadTimeout errors.

* Resolved pylint error.

* Updated comments in the unittest cases.

* Updated error handling.

* TDL-18749 Implement interruptible full table streams. (singer-io#60)

* Implemented interruptible full table streams.

* Resolved pylint error

* Resolved error in full table sync test case.

* Updated config.yml to pass cci

* Updated query building logic.

* Updated integration test case.

* Resolved review comments.

* Resolved comments.

* Implemeted logic to skip the duplicate records.

* Resolved unittest case error.

* Resolved pylint error

* Resolved integration test case error

* Added empty filter param for call_details and campaign_label stream.

* Added unit test cases for should_sync method.

* Revert "Implemeted logic to skip the duplicate records."

This reverts commit cd06e11.

* Added logger message for debugging purpose

* Updated integration test case.

* Replaced .format with f string.

* Updated comment in integration test.

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* Bump version (singer-io#67)

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* TDL-18524 updated readme and added sample config (singer-io#51)

* updated readme and added sample config

* updated endpoints

* add new streams

* resolved PR comments

* fixed a typo

* added the

* Add timeout parameter to gas.search (singer-io#64)

* Add timeout parameter to search gas.search

* Increase timeout to 15 minutes for safety.

* Add get_request_timeout function, add config to make_request as needed

* Update on_giveup to raise specific exception text for timeoutexception class; add unit test

* Make Pylint happy take 1

* Update make_request signature in unittests.

* Another Unittest update

* More unittest fixes for signature

* Add default config param value for ease of implementation in future tests

* Fix pylint dangerous-defaul-value

* Fix stupid error

* Version bump and changelog (singer-io#70)

* TDL-19486 Add limit clause to core stream queries (singer-io#68)

* Initial commit for add page limit.

* Added limit parameter in sync method of ReportStream class.

* Fixed issue for call_details stream.

* Resolved unit test case error.

* Added test cases.

* Updated code comments.

* Fixed keyerror issue.

* Updated default query limit.

* Updated pagination test case.

* Modify config name to be more explicit.

* Update comment for accuracy.

* Update property name in base and propogate name change to tests

* Exclude feed from streams_to_test because of lack of data

* Committing stuff from main that should have already been in

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Version Bump and Changelog Update (singer-io#72)

* Reintroduce unintentionally removed files. (singer-io#73)

* add schemaless

* fix schemaless config

* update login_customer_ids

* fix missing key from dict

* add google ads api version in config
add log to show the version

* use version from config

* use version global

Co-authored-by: bryantgray <bgray@talend.com>
Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>
Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: atribed <agorka@talend.com>
Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com>
Co-authored-by: KrishnanG <kgurusamy@talend.com>
Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com>
Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
lideke added a commit to sendinblue/tap-google-ads that referenced this pull request Oct 2, 2023
* Bump to v1.0.0, update changelog (singer-io#45)

* Qa/future testing (singer-io#48)

* documented reasons for untested streams

* list high level scenarios for manual qa checks

* remove stitch specific pr template

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Add call_details stream (singer-io#49)

* Add call_view core stream, filter non-attribute fields from core streams

* Change `call_view` stream name to `call_details`

* Make pylint happy

* Fix integration tests

* Remove campaign_id from foreign keys, split call_details foreign keys

* Update tests to exclude call_details as needed

* Add context around excluding call_details from tests

* Remove outdated TODO related to addition of call_details stream.

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53)

* Add core LABELS streams and campaign.labels fields to relevant reports

* Update test exclusions to exclude new core streams.

* Update discover to include campaign_label in reports list

* Update foreign_key expected metadata to include attributed resource foreign_keys

* Update tests to re-include campaign_labels and labels as we now have test data.

* Handles case where state does not have currently_syncing (singer-io#54)

* Handles case where state does not have currently_syncing

* tap-tester test added

* takeout unused imports in test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Version bump and changelog entry (singer-io#52)

* Version bump and changelog entry

* Update to include PR 53.

* Add PR 54 to changelog

* Implement Automatic Keys (singer-io#55)

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Update setup.py with docs explaining required but unused dep

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Cleanup (singer-io#56)

* Remove useless function

* Rename "REPORTS" to "STREAMS" for accuracy / readability

* Version bump for PRs 56 and 55 (singer-io#57)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* [Feature] Add more core streams (singer-io#58)

* WIP add new core streams

* Set geographic_view.location_type as automatic to account for reporting discrepancies

* Update Automatic Fields test happy / error paths

* Update base to include automatic_keys metadata for use in tests

* WIP Add Automatic Report Fields

* Pass automatic_keys to BaseStream; use automatic_keys for inclusion

* Fix bad field name in streams; Start updating tests

* Rename function to match base.py

* Add closing brace -_-

* Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report

* Rename report_field_parts to split_report_field

* Update transform_keys to raise ad_group_ad.ad fields

* Update happy path streams_to_test to exclude streams with no data

* WIP remove IPDB and try except

* Fix report_definition typo and add handling for user_interest_id field

* Remove field name change for ad_performance_report stream

* Accept Andy's Suggestion

Co-authored-by: Andy Lu <andy@stitchdata.com>

* Reverting Andy's change because it affects core streams

* Explicitly install grpcio-status to avoid from_call attribute errors

* Remove ipdb -_-

* Update setup.py with docs explaining required but unused dep

* WIP w/ failing tests and attributed_resource foreign_keys

* WIP with failing tests; remove extraneous attributed_resoruce automatic fields

* Create UserInterestStream class to handle its edge case

* Add transform_keys to UserInterestStream class; start test updates

* Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys

* update auto fields test to account for compound pks

* Fix campaigns typo

* Remove exclusion of feed and feed items from sync canary test

Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: atribed <agorka@talend.com>

* V1.3.0 (singer-io#59)

* Version bump for PRs 56 and 55

* Update to exclude ad_group_ad change for ad_performance_report

* Version bump for 1.3.0

Co-authored-by: Arthur Gorka <azgorka@gmail.com>

* Qa/fix build notification (singer-io#65)

* fix slack notif for build

* remove click_performance_report from tests

* remove click_performance_report from sync canary test

* run only streams that are untested in canary sync

* just skip canaray test

* put back the assert in the skipped test

Co-authored-by: kspeer <kspeer@stitchdata.com>

* Crest master (singer-io#66)

* Tdl 19235 handle uncaught exceptions (singer-io#61)

* Added backoff for 5xx, 429 and ReadTimeout errors.

* Resolved pylint error.

* Updated comments in the unittest cases.

* Updated error handling.

* TDL-18749 Implement interruptible full table streams. (singer-io#60)

* Implemented interruptible full table streams.

* Resolved pylint error

* Resolved error in full table sync test case.

* Updated config.yml to pass cci

* Updated query building logic.

* Updated integration test case.

* Resolved review comments.

* Resolved comments.

* Implemeted logic to skip the duplicate records.

* Resolved unittest case error.

* Resolved pylint error

* Resolved integration test case error

* Added empty filter param for call_details and campaign_label stream.

* Added unit test cases for should_sync method.

* Revert "Implemeted logic to skip the duplicate records."

This reverts commit cd06e11.

* Added logger message for debugging purpose

* Updated integration test case.

* Replaced .format with f string.

* Updated comment in integration test.

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* Bump version (singer-io#67)

Co-authored-by: KrishnanG <kgurusamy@talend.com>

* TDL-18524 updated readme and added sample config (singer-io#51)

* updated readme and added sample config

* updated endpoints

* add new streams

* resolved PR comments

* fixed a typo

* added the

* Add timeout parameter to gas.search (singer-io#64)

* Add timeout parameter to search gas.search

* Increase timeout to 15 minutes for safety.

* Add get_request_timeout function, add config to make_request as needed

* Update on_giveup to raise specific exception text for timeoutexception class; add unit test

* Make Pylint happy take 1

* Update make_request signature in unittests.

* Another Unittest update

* More unittest fixes for signature

* Add default config param value for ease of implementation in future tests

* Fix pylint dangerous-defaul-value

* Fix stupid error

* Version bump and changelog (singer-io#70)

* TDL-19486 Add limit clause to core stream queries (singer-io#68)

* Initial commit for add page limit.

* Added limit parameter in sync method of ReportStream class.

* Fixed issue for call_details stream.

* Resolved unit test case error.

* Added test cases.

* Updated code comments.

* Fixed keyerror issue.

* Updated default query limit.

* Updated pagination test case.

* Modify config name to be more explicit.

* Update comment for accuracy.

* Update property name in base and propogate name change to tests

* Exclude feed from streams_to_test because of lack of data

* Committing stuff from main that should have already been in

Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>

* Version Bump and Changelog Update (singer-io#72)

* Reintroduce unintentionally removed files. (singer-io#73)

* add schemaless

* fix schemaless config

* update login_customer_ids

* fix missing key from dict

* add google ads api version in config
add log to show the version

* use version from config

* use version global

Co-authored-by: bryantgray <bgray@talend.com>
Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com>
Co-authored-by: kspeer <kspeer@stitchdata.com>
Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com>
Co-authored-by: Andy Lu <andy@stitchdata.com>
Co-authored-by: Arthur Gorka <azgorka@gmail.com>
Co-authored-by: atribed <agorka@talend.com>
Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com>
Co-authored-by: KrishnanG <kgurusamy@talend.com>
Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com>
Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com>
Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants