Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 Source Google Analytics: Support chunked syncs to avoid sampling #2151

Merged
merged 5 commits into from
Mar 3, 2021
Merged

🎉 Source Google Analytics: Support chunked syncs to avoid sampling #2151

merged 5 commits into from
Mar 3, 2021

Conversation

vitaliizazmic
Copy link
Contributor

How

  • Created fork of pipelinewise-tap-google-analytics
  • Added reading of streams by day in fork
  • Installing singer tap from fork instead PyPi

Pre-merge Checklist

  • Run integration tests
  • Publish Docker images

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Feb 22, 2021 •
edited by github-actions bot
Loading

/test connector=source-googleanalytics-singer

🕑 source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/589712679
✅ source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/589712679

@sherifnada
Copy link
Contributor

@vitaliizazmic can you create a PR from https://github.com/vitaliizazmic/pipelinewise-tap-google-analytics to airbyte's fork of tap-google-analytics so we can review that change? I'll get to it tomorrow morning my time.

sherifnada
sherifnada approved these changes Feb 25, 2021
View reviewed changes
Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- can you bump the connector version and publish the connector by running /publish connectors=connector/source-googleanalytics-singer then merge this PR?

airbyte-integrations/connectors/source-googleanalytics-singer/setup.py Outdated
@@ -34,7 +34,7 @@
# two sets of dependencies: 1) for main 2) for standard test deps. 2 does not have all of the dependencies of 1, which is we cannot use install_requires.
extras_require={
"main": [
"pipelinewise-tap-google-analytics==1.1.1",
"pipelinewise-tap-google-analytics @ https://github.com/vitaliizazmic/pipelinewise-tap-google-analytics/tarball/master#egg=pipelinewise-tap-google-analytics-1.2.1",
Copy link
Contributor

@sherifnada sherifnada Feb 25, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vitaliizazmic make sure to update this link to the Airbyte version and merge the tap-google-analytics PR

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Feb 26, 2021 •
edited by github-actions bot
Loading

/test connector=source-googleanalytics-singer

🕑 source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/603158604
✅ source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/603158604

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Feb 26, 2021 •
edited by github-actions bot
Loading

/publish connector=connectors/source-googleanalytics-singer

🕑 connectors/source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/603285990
❌ connectors/source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/603285990

@vitaliizazmic
Bump version to 0.1.9
599f940
@vitaliizazmic
Merge remote-tracking branch 'upstream/master' into vitalii/2062_ga_c…
b3c4a82 
…hunked_sync

# Conflicts:
#	airbyte-config/init/src/main/resources/config/STANDARD_SOURCE_DEFINITION/39f092a6-8c87-4f6f-a8d9-5cef45b7dbe1.json
#	airbyte-config/init/src/main/resources/seed/source_definitions.yaml
#	airbyte-integrations/connectors/source-googleanalytics-singer/Dockerfile
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Feb 26, 2021 •
edited by github-actions bot
Loading

/publish connector=connectors/source-googleanalytics-singer

🕑 connectors/source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/603405307
❌ connectors/source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/603405307

@sherifnada sherifnada linked an issue Mar 1, 2021 that may be closed by this pull request
🎉 Source Google Analytics: Support chunked syncs to avoid sampling #2062
Closed
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Mar 3, 2021 •
edited by github-actions bot
Loading

/publish connector=connectors/source-googleanalytics-singer

🕑 connectors/source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/618217548
✅ connectors/source-googleanalytics-singer https://github.com/airbytehq/airbyte/actions/runs/618217548

@sherifnada sherifnada changed the title GA source #2062 - chunks sync 🎉 Source Google Analytics: Support chunked syncs to avoid sampling Mar 3, 2021
@vitaliizazmic vitaliizazmic merged commit 9d0d560 into master Mar 3, 2021
@vitaliizazmic vitaliizazmic deleted the vitalii/2062_ga_chunked_sync branch March 3, 2021 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sherifnada sherifnada sherifnada approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

🎉 Source Google Analytics: Support chunked syncs to avoid sampling
2 participants
@vitaliizazmic @sherifnada