Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[source-tiktok] Remove audience reports hourly streams #20598

Merged
merged 6 commits into from Dec 19, 2022
Merged

[source-tiktok] Remove audience reports hourly streams #20598

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
27c85d1
get rid of audience report streams using hourly dimension
brianjlai Dec 16, 2022
473d0b7
bump version and docs changelog
brianjlai Dec 16, 2022
b8a9262
fix tests
brianjlai Dec 16, 2022
f9ed877
update backwards compatibility tests since some streams are no longer…
brianjlai Dec 17, 2022
9dd5afe
auto-bump connector version
octavia-squidington-iii Dec 18, 2022
e699cd1
Merge branch 'master' into brian/tiktok_remove_hourly_audience_reports
girarda Dec 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 0 additions & 28 deletions airbyte-integrations/connectors/source-tiktok-marketing/integration_tests/streams.json
View file
Open in desktop
Expand Up @@ -108,13 +108,6 @@
"json_schema": {},
"supported_sync_modes": ["full_refresh"]
},
{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This streams.json I don't think is actually used in our acceptance test YAML but just keeping this up to date regardless

"name": "advertisers_audience_reports_hourly",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["stat_time_hour"]
},
{
"name": "advertisers_audience_reports_daily",
"json_schema": {},
Expand All @@ -127,41 +120,20 @@
"json_schema": {},
"supported_sync_modes": ["full_refresh"]
},
{
"name": "ads_audience_reports_hourly",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["stat_time_hour"]
},
{
"name": "ads_audience_reports_daily",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["stat_time_day"]
},
{
"name": "ad_group_audience_reports_hourly",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["stat_time_hour"]
},
{
"name": "ad_group_audience_reports_daily",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["stat_time_day"]
},
{
"name": "campaigns_audience_reports_by_country_hourly",
"json_schema": {},
"supported_sync_modes": ["full_refresh", "incremental"],
"source_defined_cursor": true,
"default_cursor_field": ["stat_time_hour"]
},
{
"name": "campaigns_audience_reports_by_country_daily",
"json_schema": {},
Expand Down
15 changes: 10 additions & 5 deletions airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/source.py
View file
Open in desktop
Expand Up @@ -158,18 +158,23 @@ def streams(self, config: Mapping[str, Any]) -> List[Stream]:

# 2. Basic report streams:
reports = [AdsReports, AdGroupsReports, CampaignsReports]
audience_reports = [AdsAudienceReports, AdGroupAudienceReports, CampaignsAudienceReportsByCountry]
if is_production:
# 2.1 streams work only in prod env
reports.extend([AdvertisersReports, AdvertisersAudienceReports])
reports.append(AdvertisersReports)
audience_reports.append(AdvertisersAudienceReports)

for Report in reports:
for Granularity in [Hourly, Daily, Lifetime]:
streams.append(get_report_stream(Report, Granularity)(**args))

# 3. Audience report streams:
# Audience report supports lifetime metrics only at the ADVERTISER level (see 2.1).
for Report in [AdsAudienceReports, AdGroupAudienceReports, CampaignsAudienceReportsByCountry]:
for Granularity in [Hourly, Daily]:
streams.append(get_report_stream(Report, Granularity)(**args))
for Report in audience_reports:
# As per TikTok's documentation, audience reports only support daily (not hourly) time dimension for metrics
streams.append(get_report_stream(Report, Daily)(**args))

# Audience report supports lifetime metrics only at the ADVERTISER level (see 2.1).
if Report == AdvertisersAudienceReports:
streams.append(get_report_stream(Report, Lifetime)(**args))

return streams