Source Google Analytics Data API: implement advanced reports

[davydov-d]

What

#11689

How

Implement support for Cohort reports and Pivot reports according to https://developers.google.com/analytics/devguides/reporting/data/v1

[davydov-d]

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4692406047

[davydov-d]

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4692450643
❌ connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4692450643
🐛 https://gradle.com/s/sbcbubhwyodqq

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream traff...
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
FAILED test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
====== 4 failed, 35 passed, 1 skipped, 102 warnings in 3683.96s (1:01:23) ======

> Task :airbyte-integrations:connectors:source-google-analytics-data-api:connectorAcceptanceTest FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings
27 actionable tasks: 9 executed, 18 up-to-date

Publishing build scan...
https://gradle.com/s/sbcbubhwyodqq

S3 cache writes: 1, elapsed: 685ms, sent to cache: 455 B

[davydov-d]

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4697044496

[davydov-d]

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4697148144
✅ connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4697148144
Python tests coverage:

Name                                                Stmts   Miss  Cover
-----------------------------------------------------------------------
source_google_analytics_data_api/__init__.py            2      0   100%
source_google_analytics_data_api/authenticator.py      44      2    95%
source_google_analytics_data_api/utils.py              26      2    92%
source_google_analytics_data_api/source.py            225     28    88%
source_google_analytics_data_api/api_quota.py          94     12    87%
-----------------------------------------------------------------------
TOTAL                                                 391     44    89%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
=========== 39 passed, 1 skipped, 102 warnings in 202.85s (0:03:22) ============

[lazebnyi]

Why you delete dateRange from payload if you said that is required for cohort request. And this mixin is parent only for cohort report.

[davydov-d]

These are different date ranges. The one in the root of the payload should be unspecified because there is another one inside the "cohortSpec" object (see schema for details).

This mixin is one of the base classes for the dynamically created report if there's a "cohortSpec" object in the report definition. It can be either a regular report or a pivot report.

[clnoll]

Thanks @davydov-d! Just a few questions for you.

[arsenlosenko]

Hi @clnoll, @davydov-d is currently on PTO, so I'll dive into this PR and answer your questions regarding it a bit later.

[arsenlosenko]

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4788055918
✅ connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4788055918
Python tests coverage:

Name                                                Stmts   Miss  Cover
-----------------------------------------------------------------------
source_google_analytics_data_api/__init__.py            2      0   100%
source_google_analytics_data_api/authenticator.py      44      2    95%
source_google_analytics_data_api/utils.py              26      2    92%
source_google_analytics_data_api/source.py            225     28    88%
source_google_analytics_data_api/api_quota.py          94     12    87%
-----------------------------------------------------------------------
TOTAL                                                 391     44    89%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
=========== 39 passed, 1 skipped, 102 warnings in 200.28s (0:03:20) ============

[clnoll]

LGTM!

[arsenlosenko]

/publish connector=connectors/source-google-analytics-data-api

🕑 Publishing the following connectors:
connectors/source-google-analytics-data-api
https://github.com/airbytehq/airbyte/actions/runs/4799276846

Connector	Version	Did it publish?	Were definitions generated?
connectors/source-google-analytics-data-api	0.2.0	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️