[Feature] Add more core streams #58
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…re/more-core-streams
…reorder test metadata; add campaign.id to campaign_audience_performance_report
Co-authored-by: Andy Lu <andy@stitchdata.com>
atribed
reviewed
May 3, 2022
…erestStream transform_keys
cosimon
approved these changes
May 5, 2022
lideke
added a commit
to sendinblue/tap-google-ads
that referenced
this pull request
Aug 2, 2022
* Bump to v1.0.0, update changelog (singer-io#45) * Qa/future testing (singer-io#48) * documented reasons for untested streams * list high level scenarios for manual qa checks * remove stitch specific pr template Co-authored-by: kspeer <kspeer@stitchdata.com> * Add call_details stream (singer-io#49) * Add call_view core stream, filter non-attribute fields from core streams * Change `call_view` stream name to `call_details` * Make pylint happy * Fix integration tests * Remove campaign_id from foreign keys, split call_details foreign keys * Update tests to exclude call_details as needed * Add context around excluding call_details from tests * Remove outdated TODO related to addition of call_details stream. Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53) * Add core LABELS streams and campaign.labels fields to relevant reports * Update test exclusions to exclude new core streams. * Update discover to include campaign_label in reports list * Update foreign_key expected metadata to include attributed resource foreign_keys * Update tests to re-include campaign_labels and labels as we now have test data. * Handles case where state does not have currently_syncing (singer-io#54) * Handles case where state does not have currently_syncing * tap-tester test added * takeout unused imports in test Co-authored-by: kspeer <kspeer@stitchdata.com> * Version bump and changelog entry (singer-io#52) * Version bump and changelog entry * Update to include PR 53. * Add PR 54 to changelog * Implement Automatic Keys (singer-io#55) * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Update setup.py with docs explaining required but unused dep Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Cleanup (singer-io#56) * Remove useless function * Rename "REPORTS" to "STREAMS" for accuracy / readability * Version bump for PRs 56 and 55 (singer-io#57) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * [Feature] Add more core streams (singer-io#58) * WIP add new core streams * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * WIP remove IPDB and try except * Fix report_definition typo and add handling for user_interest_id field * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Remove ipdb -_- * Update setup.py with docs explaining required but unused dep * WIP w/ failing tests and attributed_resource foreign_keys * WIP with failing tests; remove extraneous attributed_resoruce automatic fields * Create UserInterestStream class to handle its edge case * Add transform_keys to UserInterestStream class; start test updates * Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys * update auto fields test to account for compound pks * Fix campaigns typo * Remove exclusion of feed and feed items from sync canary test Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: atribed <agorka@talend.com> * V1.3.0 (singer-io#59) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * Version bump for 1.3.0 Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Qa/fix build notification (singer-io#65) * fix slack notif for build * remove click_performance_report from tests * remove click_performance_report from sync canary test * run only streams that are untested in canary sync * just skip canaray test * put back the assert in the skipped test Co-authored-by: kspeer <kspeer@stitchdata.com> * Crest master (singer-io#66) * Tdl 19235 handle uncaught exceptions (singer-io#61) * Added backoff for 5xx, 429 and ReadTimeout errors. * Resolved pylint error. * Updated comments in the unittest cases. * Updated error handling. * TDL-18749 Implement interruptible full table streams. (singer-io#60) * Implemented interruptible full table streams. * Resolved pylint error * Resolved error in full table sync test case. * Updated config.yml to pass cci * Updated query building logic. * Updated integration test case. * Resolved review comments. * Resolved comments. * Implemeted logic to skip the duplicate records. * Resolved unittest case error. * Resolved pylint error * Resolved integration test case error * Added empty filter param for call_details and campaign_label stream. * Added unit test cases for should_sync method. * Revert "Implemeted logic to skip the duplicate records." This reverts commit cd06e11. * Added logger message for debugging purpose * Updated integration test case. * Replaced .format with f string. * Updated comment in integration test. Co-authored-by: KrishnanG <kgurusamy@talend.com> * Bump version (singer-io#67) Co-authored-by: KrishnanG <kgurusamy@talend.com> * TDL-18524 updated readme and added sample config (singer-io#51) * updated readme and added sample config * updated endpoints * add new streams * resolved PR comments * fixed a typo * added the * Add timeout parameter to gas.search (singer-io#64) * Add timeout parameter to search gas.search * Increase timeout to 15 minutes for safety. * Add get_request_timeout function, add config to make_request as needed * Update on_giveup to raise specific exception text for timeoutexception class; add unit test * Make Pylint happy take 1 * Update make_request signature in unittests. * Another Unittest update * More unittest fixes for signature * Add default config param value for ease of implementation in future tests * Fix pylint dangerous-defaul-value * Fix stupid error * Version bump and changelog (singer-io#70) * TDL-19486 Add limit clause to core stream queries (singer-io#68) * Initial commit for add page limit. * Added limit parameter in sync method of ReportStream class. * Fixed issue for call_details stream. * Resolved unit test case error. * Added test cases. * Updated code comments. * Fixed keyerror issue. * Updated default query limit. * Updated pagination test case. * Modify config name to be more explicit. * Update comment for accuracy. * Update property name in base and propogate name change to tests * Exclude feed from streams_to_test because of lack of data * Committing stuff from main that should have already been in Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Version Bump and Changelog Update (singer-io#72) * Reintroduce unintentionally removed files. (singer-io#73) * add schemaless * fix schemaless config * update login_customer_ids * fix missing key from dict * add google ads api version in config add log to show the version * use version from config * use version global Co-authored-by: bryantgray <bgray@talend.com> Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: atribed <agorka@talend.com> Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com> Co-authored-by: KrishnanG <kgurusamy@talend.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com> Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com> Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
lideke
added a commit
to sendinblue/tap-google-ads
that referenced
this pull request
Aug 2, 2022
* Bump to v1.0.0, update changelog (singer-io#45) * Qa/future testing (singer-io#48) * documented reasons for untested streams * list high level scenarios for manual qa checks * remove stitch specific pr template Co-authored-by: kspeer <kspeer@stitchdata.com> * Add call_details stream (singer-io#49) * Add call_view core stream, filter non-attribute fields from core streams * Change `call_view` stream name to `call_details` * Make pylint happy * Fix integration tests * Remove campaign_id from foreign keys, split call_details foreign keys * Update tests to exclude call_details as needed * Add context around excluding call_details from tests * Remove outdated TODO related to addition of call_details stream. Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53) * Add core LABELS streams and campaign.labels fields to relevant reports * Update test exclusions to exclude new core streams. * Update discover to include campaign_label in reports list * Update foreign_key expected metadata to include attributed resource foreign_keys * Update tests to re-include campaign_labels and labels as we now have test data. * Handles case where state does not have currently_syncing (singer-io#54) * Handles case where state does not have currently_syncing * tap-tester test added * takeout unused imports in test Co-authored-by: kspeer <kspeer@stitchdata.com> * Version bump and changelog entry (singer-io#52) * Version bump and changelog entry * Update to include PR 53. * Add PR 54 to changelog * Implement Automatic Keys (singer-io#55) * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Update setup.py with docs explaining required but unused dep Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Cleanup (singer-io#56) * Remove useless function * Rename "REPORTS" to "STREAMS" for accuracy / readability * Version bump for PRs 56 and 55 (singer-io#57) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * [Feature] Add more core streams (singer-io#58) * WIP add new core streams * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * WIP remove IPDB and try except * Fix report_definition typo and add handling for user_interest_id field * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Remove ipdb -_- * Update setup.py with docs explaining required but unused dep * WIP w/ failing tests and attributed_resource foreign_keys * WIP with failing tests; remove extraneous attributed_resoruce automatic fields * Create UserInterestStream class to handle its edge case * Add transform_keys to UserInterestStream class; start test updates * Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys * update auto fields test to account for compound pks * Fix campaigns typo * Remove exclusion of feed and feed items from sync canary test Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: atribed <agorka@talend.com> * V1.3.0 (singer-io#59) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * Version bump for 1.3.0 Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Qa/fix build notification (singer-io#65) * fix slack notif for build * remove click_performance_report from tests * remove click_performance_report from sync canary test * run only streams that are untested in canary sync * just skip canaray test * put back the assert in the skipped test Co-authored-by: kspeer <kspeer@stitchdata.com> * Crest master (singer-io#66) * Tdl 19235 handle uncaught exceptions (singer-io#61) * Added backoff for 5xx, 429 and ReadTimeout errors. * Resolved pylint error. * Updated comments in the unittest cases. * Updated error handling. * TDL-18749 Implement interruptible full table streams. (singer-io#60) * Implemented interruptible full table streams. * Resolved pylint error * Resolved error in full table sync test case. * Updated config.yml to pass cci * Updated query building logic. * Updated integration test case. * Resolved review comments. * Resolved comments. * Implemeted logic to skip the duplicate records. * Resolved unittest case error. * Resolved pylint error * Resolved integration test case error * Added empty filter param for call_details and campaign_label stream. * Added unit test cases for should_sync method. * Revert "Implemeted logic to skip the duplicate records." This reverts commit cd06e11. * Added logger message for debugging purpose * Updated integration test case. * Replaced .format with f string. * Updated comment in integration test. Co-authored-by: KrishnanG <kgurusamy@talend.com> * Bump version (singer-io#67) Co-authored-by: KrishnanG <kgurusamy@talend.com> * TDL-18524 updated readme and added sample config (singer-io#51) * updated readme and added sample config * updated endpoints * add new streams * resolved PR comments * fixed a typo * added the * Add timeout parameter to gas.search (singer-io#64) * Add timeout parameter to search gas.search * Increase timeout to 15 minutes for safety. * Add get_request_timeout function, add config to make_request as needed * Update on_giveup to raise specific exception text for timeoutexception class; add unit test * Make Pylint happy take 1 * Update make_request signature in unittests. * Another Unittest update * More unittest fixes for signature * Add default config param value for ease of implementation in future tests * Fix pylint dangerous-defaul-value * Fix stupid error * Version bump and changelog (singer-io#70) * TDL-19486 Add limit clause to core stream queries (singer-io#68) * Initial commit for add page limit. * Added limit parameter in sync method of ReportStream class. * Fixed issue for call_details stream. * Resolved unit test case error. * Added test cases. * Updated code comments. * Fixed keyerror issue. * Updated default query limit. * Updated pagination test case. * Modify config name to be more explicit. * Update comment for accuracy. * Update property name in base and propogate name change to tests * Exclude feed from streams_to_test because of lack of data * Committing stuff from main that should have already been in Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Version Bump and Changelog Update (singer-io#72) * Reintroduce unintentionally removed files. (singer-io#73) * add schemaless * fix schemaless config * update login_customer_ids * fix missing key from dict * add google ads api version in config add log to show the version * use version from config * use version global Co-authored-by: bryantgray <bgray@talend.com> Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: atribed <agorka@talend.com> Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com> Co-authored-by: KrishnanG <kgurusamy@talend.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com> Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com> Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
lideke
added a commit
to sendinblue/tap-google-ads
that referenced
this pull request
Mar 22, 2023
* Bump to v1.0.0, update changelog (singer-io#45) * Qa/future testing (singer-io#48) * documented reasons for untested streams * list high level scenarios for manual qa checks * remove stitch specific pr template Co-authored-by: kspeer <kspeer@stitchdata.com> * Add call_details stream (singer-io#49) * Add call_view core stream, filter non-attribute fields from core streams * Change `call_view` stream name to `call_details` * Make pylint happy * Fix integration tests * Remove campaign_id from foreign keys, split call_details foreign keys * Update tests to exclude call_details as needed * Add context around excluding call_details from tests * Remove outdated TODO related to addition of call_details stream. Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53) * Add core LABELS streams and campaign.labels fields to relevant reports * Update test exclusions to exclude new core streams. * Update discover to include campaign_label in reports list * Update foreign_key expected metadata to include attributed resource foreign_keys * Update tests to re-include campaign_labels and labels as we now have test data. * Handles case where state does not have currently_syncing (singer-io#54) * Handles case where state does not have currently_syncing * tap-tester test added * takeout unused imports in test Co-authored-by: kspeer <kspeer@stitchdata.com> * Version bump and changelog entry (singer-io#52) * Version bump and changelog entry * Update to include PR 53. * Add PR 54 to changelog * Implement Automatic Keys (singer-io#55) * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Update setup.py with docs explaining required but unused dep Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Cleanup (singer-io#56) * Remove useless function * Rename "REPORTS" to "STREAMS" for accuracy / readability * Version bump for PRs 56 and 55 (singer-io#57) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * [Feature] Add more core streams (singer-io#58) * WIP add new core streams * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * WIP remove IPDB and try except * Fix report_definition typo and add handling for user_interest_id field * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Remove ipdb -_- * Update setup.py with docs explaining required but unused dep * WIP w/ failing tests and attributed_resource foreign_keys * WIP with failing tests; remove extraneous attributed_resoruce automatic fields * Create UserInterestStream class to handle its edge case * Add transform_keys to UserInterestStream class; start test updates * Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys * update auto fields test to account for compound pks * Fix campaigns typo * Remove exclusion of feed and feed items from sync canary test Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: atribed <agorka@talend.com> * V1.3.0 (singer-io#59) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * Version bump for 1.3.0 Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Qa/fix build notification (singer-io#65) * fix slack notif for build * remove click_performance_report from tests * remove click_performance_report from sync canary test * run only streams that are untested in canary sync * just skip canaray test * put back the assert in the skipped test Co-authored-by: kspeer <kspeer@stitchdata.com> * Crest master (singer-io#66) * Tdl 19235 handle uncaught exceptions (singer-io#61) * Added backoff for 5xx, 429 and ReadTimeout errors. * Resolved pylint error. * Updated comments in the unittest cases. * Updated error handling. * TDL-18749 Implement interruptible full table streams. (singer-io#60) * Implemented interruptible full table streams. * Resolved pylint error * Resolved error in full table sync test case. * Updated config.yml to pass cci * Updated query building logic. * Updated integration test case. * Resolved review comments. * Resolved comments. * Implemeted logic to skip the duplicate records. * Resolved unittest case error. * Resolved pylint error * Resolved integration test case error * Added empty filter param for call_details and campaign_label stream. * Added unit test cases for should_sync method. * Revert "Implemeted logic to skip the duplicate records." This reverts commit cd06e11. * Added logger message for debugging purpose * Updated integration test case. * Replaced .format with f string. * Updated comment in integration test. Co-authored-by: KrishnanG <kgurusamy@talend.com> * Bump version (singer-io#67) Co-authored-by: KrishnanG <kgurusamy@talend.com> * TDL-18524 updated readme and added sample config (singer-io#51) * updated readme and added sample config * updated endpoints * add new streams * resolved PR comments * fixed a typo * added the * Add timeout parameter to gas.search (singer-io#64) * Add timeout parameter to search gas.search * Increase timeout to 15 minutes for safety. * Add get_request_timeout function, add config to make_request as needed * Update on_giveup to raise specific exception text for timeoutexception class; add unit test * Make Pylint happy take 1 * Update make_request signature in unittests. * Another Unittest update * More unittest fixes for signature * Add default config param value for ease of implementation in future tests * Fix pylint dangerous-defaul-value * Fix stupid error * Version bump and changelog (singer-io#70) * TDL-19486 Add limit clause to core stream queries (singer-io#68) * Initial commit for add page limit. * Added limit parameter in sync method of ReportStream class. * Fixed issue for call_details stream. * Resolved unit test case error. * Added test cases. * Updated code comments. * Fixed keyerror issue. * Updated default query limit. * Updated pagination test case. * Modify config name to be more explicit. * Update comment for accuracy. * Update property name in base and propogate name change to tests * Exclude feed from streams_to_test because of lack of data * Committing stuff from main that should have already been in Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Version Bump and Changelog Update (singer-io#72) * Reintroduce unintentionally removed files. (singer-io#73) * add schemaless * fix schemaless config * update login_customer_ids * fix missing key from dict * add google ads api version in config add log to show the version * use version from config * use version global Co-authored-by: bryantgray <bgray@talend.com> Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: atribed <agorka@talend.com> Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com> Co-authored-by: KrishnanG <kgurusamy@talend.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com> Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com> Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
lideke
added a commit
to sendinblue/tap-google-ads
that referenced
this pull request
Oct 2, 2023
* Bump to v1.0.0, update changelog (singer-io#45) * Qa/future testing (singer-io#48) * documented reasons for untested streams * list high level scenarios for manual qa checks * remove stitch specific pr template Co-authored-by: kspeer <kspeer@stitchdata.com> * Add call_details stream (singer-io#49) * Add call_view core stream, filter non-attribute fields from core streams * Change `call_view` stream name to `call_details` * Make pylint happy * Fix integration tests * Remove campaign_id from foreign keys, split call_details foreign keys * Update tests to exclude call_details as needed * Add context around excluding call_details from tests * Remove outdated TODO related to addition of call_details stream. Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Add core LABELS streams and campaign.labels fields to relevant reports (singer-io#53) * Add core LABELS streams and campaign.labels fields to relevant reports * Update test exclusions to exclude new core streams. * Update discover to include campaign_label in reports list * Update foreign_key expected metadata to include attributed resource foreign_keys * Update tests to re-include campaign_labels and labels as we now have test data. * Handles case where state does not have currently_syncing (singer-io#54) * Handles case where state does not have currently_syncing * tap-tester test added * takeout unused imports in test Co-authored-by: kspeer <kspeer@stitchdata.com> * Version bump and changelog entry (singer-io#52) * Version bump and changelog entry * Update to include PR 53. * Add PR 54 to changelog * Implement Automatic Keys (singer-io#55) * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Update setup.py with docs explaining required but unused dep Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Cleanup (singer-io#56) * Remove useless function * Rename "REPORTS" to "STREAMS" for accuracy / readability * Version bump for PRs 56 and 55 (singer-io#57) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * [Feature] Add more core streams (singer-io#58) * WIP add new core streams * Set geographic_view.location_type as automatic to account for reporting discrepancies * Update Automatic Fields test happy / error paths * Update base to include automatic_keys metadata for use in tests * WIP Add Automatic Report Fields * Pass automatic_keys to BaseStream; use automatic_keys for inclusion * Fix bad field name in streams; Start updating tests * Rename function to match base.py * Add closing brace -_- * Fix tests; rename ad_group_criterion_criterion_id in automatic_keys; reorder test metadata; add campaign.id to campaign_audience_performance_report * Rename report_field_parts to split_report_field * Update transform_keys to raise ad_group_ad.ad fields * Update happy path streams_to_test to exclude streams with no data * WIP remove IPDB and try except * Fix report_definition typo and add handling for user_interest_id field * Remove field name change for ad_performance_report stream * Accept Andy's Suggestion Co-authored-by: Andy Lu <andy@stitchdata.com> * Reverting Andy's change because it affects core streams * Explicitly install grpcio-status to avoid from_call attribute errors * Remove ipdb -_- * Update setup.py with docs explaining required but unused dep * WIP w/ failing tests and attributed_resource foreign_keys * WIP with failing tests; remove extraneous attributed_resoruce automatic fields * Create UserInterestStream class to handle its edge case * Add transform_keys to UserInterestStream class; start test updates * Rename obj to json_message; rename variables accordingly; fix UserInterestStream transform_keys * update auto fields test to account for compound pks * Fix campaigns typo * Remove exclusion of feed and feed items from sync canary test Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: atribed <agorka@talend.com> * V1.3.0 (singer-io#59) * Version bump for PRs 56 and 55 * Update to exclude ad_group_ad change for ad_performance_report * Version bump for 1.3.0 Co-authored-by: Arthur Gorka <azgorka@gmail.com> * Qa/fix build notification (singer-io#65) * fix slack notif for build * remove click_performance_report from tests * remove click_performance_report from sync canary test * run only streams that are untested in canary sync * just skip canaray test * put back the assert in the skipped test Co-authored-by: kspeer <kspeer@stitchdata.com> * Crest master (singer-io#66) * Tdl 19235 handle uncaught exceptions (singer-io#61) * Added backoff for 5xx, 429 and ReadTimeout errors. * Resolved pylint error. * Updated comments in the unittest cases. * Updated error handling. * TDL-18749 Implement interruptible full table streams. (singer-io#60) * Implemented interruptible full table streams. * Resolved pylint error * Resolved error in full table sync test case. * Updated config.yml to pass cci * Updated query building logic. * Updated integration test case. * Resolved review comments. * Resolved comments. * Implemeted logic to skip the duplicate records. * Resolved unittest case error. * Resolved pylint error * Resolved integration test case error * Added empty filter param for call_details and campaign_label stream. * Added unit test cases for should_sync method. * Revert "Implemeted logic to skip the duplicate records." This reverts commit cd06e11. * Added logger message for debugging purpose * Updated integration test case. * Replaced .format with f string. * Updated comment in integration test. Co-authored-by: KrishnanG <kgurusamy@talend.com> * Bump version (singer-io#67) Co-authored-by: KrishnanG <kgurusamy@talend.com> * TDL-18524 updated readme and added sample config (singer-io#51) * updated readme and added sample config * updated endpoints * add new streams * resolved PR comments * fixed a typo * added the * Add timeout parameter to gas.search (singer-io#64) * Add timeout parameter to search gas.search * Increase timeout to 15 minutes for safety. * Add get_request_timeout function, add config to make_request as needed * Update on_giveup to raise specific exception text for timeoutexception class; add unit test * Make Pylint happy take 1 * Update make_request signature in unittests. * Another Unittest update * More unittest fixes for signature * Add default config param value for ease of implementation in future tests * Fix pylint dangerous-defaul-value * Fix stupid error * Version bump and changelog (singer-io#70) * TDL-19486 Add limit clause to core stream queries (singer-io#68) * Initial commit for add page limit. * Added limit parameter in sync method of ReportStream class. * Fixed issue for call_details stream. * Resolved unit test case error. * Added test cases. * Updated code comments. * Fixed keyerror issue. * Updated default query limit. * Updated pagination test case. * Modify config name to be more explicit. * Update comment for accuracy. * Update property name in base and propogate name change to tests * Exclude feed from streams_to_test because of lack of data * Committing stuff from main that should have already been in Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> * Version Bump and Changelog Update (singer-io#72) * Reintroduce unintentionally removed files. (singer-io#73) * add schemaless * fix schemaless config * update login_customer_ids * fix missing key from dict * add google ads api version in config add log to show the version * use version from config * use version global Co-authored-by: bryantgray <bgray@talend.com> Co-authored-by: Kyle Speer <54034650+kspeer825@users.noreply.github.com> Co-authored-by: kspeer <kspeer@stitchdata.com> Co-authored-by: dsprayberry <28106103+dsprayberry@users.noreply.github.com> Co-authored-by: Andy Lu <andy@stitchdata.com> Co-authored-by: Arthur Gorka <azgorka@gmail.com> Co-authored-by: atribed <agorka@talend.com> Co-authored-by: Prijen Khokhani <88327452+prijendev@users.noreply.github.com> Co-authored-by: KrishnanG <kgurusamy@talend.com> Co-authored-by: KrisPersonal <66801357+KrisPersonal@users.noreply.github.com> Co-authored-by: namrata270998 <75604662+namrata270998@users.noreply.github.com> Co-authored-by: bbaltagi-dtsl <bilal.baltagi@sendinblue.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
Adds new core streams:
Create new class for UserInterestStream to handle specific transformations.
Rename obj to json_message for all transform_keys functions; rename corresponding variables appropriately.
Update tests accordingly.
QA steps
Risks
Rollback steps