Source TikTok-Marketing: added more clear message for 504 error #23091

darynaishchenko · 2023-02-15T16:37:19Z

added more clear message for 504 error
on-call: https://github.com/airbytehq/oncall/issues/980

darynaishchenko · 2023-02-15T16:39:21Z

/test connector=connectors/source-tiktok-marketing

🕑 connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4186083592
❌ connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4186083592
🐛 https://gradle.com/s/nzkinaslfaxm6

Build Failed

Test summary info:

Could not find result summary

darynaishchenko · 2023-02-16T09:14:21Z

/test connector=connectors/source-tiktok-marketing

🕑 connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4192429990
❌ connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4192429990
🐛 https://gradle.com/s/jm3hjoxowkm6e

Build Failed

Test summary info:

Could not find result summary

…odeError

darynaishchenko · 2023-02-16T09:31:13Z

/test connector=connectors/source-tiktok-marketing

🕑 connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4192567903
❌ connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4192567903
🐛 https://gradle.com/s/hsnvtwd2fv47s

Build Failed

Test summary info:

Could not find result summary

darynaishchenko · 2023-02-17T11:39:15Z

/test connector=connectors/source-tiktok-marketing

🕑 connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4203384045
❌ connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4203384045
🐛 https://gradle.com/s/xiy3xn54lydho

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream ads_r...
FAILED test_core.py::TestBasicRead::test_read[inputs1] - Failed: Stream ads_r...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [6] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_incremental.py:212: Skipping new incremental test based on acceptance-test-config.yml
============= 2 failed, 80 passed, 8 skipped in 2177.37s (0:36:17) =============

darynaishchenko · 2023-02-20T18:15:34Z

/test connector=connectors/source-tiktok-marketing

🕑 connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4226105600
✅ connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4226105600
Python tests coverage:

Name                                  Stmts   Miss  Cover
---------------------------------------------------------
source_tiktok_marketing/__init__.py       2      0   100%
source_tiktok_marketing/source.py        66      7    89%
source_tiktok_marketing/streams.py      361     41    89%
---------------------------------------------------------
TOTAL                                   429     48    89%
	 Name                                                    Stmts   Miss  Cover   Missing
	 -------------------------------------------------------------------------------------
	 connector_acceptance_test/base.py                          12      4    67%   16-19
	 connector_acceptance_test/config.py                       142      5    96%   87, 93, 242, 246-247
	 connector_acceptance_test/conftest.py                     220    102    54%   37, 43-45, 50, 55, 60, 83, 89, 95-97, 116, 121-123, 129-131, 137-138, 143-144, 149, 160, 169-178, 184-189, 204, 228, 259, 265, 273-281, 289-302, 310-323, 328-334, 341-352, 359-375
	 connector_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 connector_acceptance_test/tests/test_core.py              476    117    75%   53, 58, 97-108, 113-120, 124-125, 129-130, 380, 400, 438, 476-493, 506-517, 521-526, 532, 565-570, 608-615, 658-660, 663, 728-736, 748-751, 756, 812-813, 819, 822, 858-868, 881-906
	 connector_acceptance_test/tests/test_incremental.py       162     14    91%   58-65, 70-83, 252
	 connector_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 connector_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 connector_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 connector_acceptance_test/utils/connector_runner.py       134     33    75%   30-33, 53-54, 57-61, 64-65, 80-82, 85-87, 90-92, 95-97, 100-102, 132-133, 167-169, 216
	 connector_acceptance_test/utils/json_schema_helper.py     114     13    89%   31-32, 39, 42, 66-69, 97, 121, 203-205
	 -------------------------------------------------------------------------------------
	 TOTAL                                                    1716    348    80%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [6] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_incremental.py:212: Skipping new incremental test based on acceptance-test-config.yml
================== 82 passed, 8 skipped in 2194.91s (0:36:34) ==================

…odeError

lazebnyi

Please add allowedHosts.

maxi297

The requested changes are trivial or more of an open discussion therefore I'll approve the PR

maxi297 · 2023-02-22T15:06:32Z

airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/streams.py

@@ -427,7 +431,7 @@ def stream_slices(self, **kwargs) -> Iterable[Optional[Mapping[str, Any]]]:
        ids = self.get_advertiser_ids()
        start, end, step = 0, len(ids), 100
        for i in range(start, end, step):
-            yield {"advertiser_ids": ids[i: min(end, i + step)]}
+            yield {"advertiser_ids": ids[i : min(end, i + step)]}


nit: this does not align with PEP 8: E203 whitespace before ':' even though our black formatter does not pick it up

maxi297 · 2023-02-22T15:14:33Z

airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/streams.py

        """
        try:
            data = response.json()
        except Exception:
+            if response.status_code == 504:


I have two questions:

Do we expect Tiktok to maybe someday have the response body as JSON as well? If so, this code will not work anymore

Why don't we add this as part of the HttpStream? I seems like whenever there will be a 504, this will be an error that we will want to have clear logging on. @brianjlai is there a reason why we would not put this in the CDK itself?

Answering for first question, docs, tik tok has declaretaled status codes in json response, there aren't any status for 504, which is html.

…odeError

darynaishchenko · 2023-02-27T17:30:45Z

/test connector=connectors/source-tiktok-marketing

🕑 connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4285138836
✅ connectors/source-tiktok-marketing https://github.com/airbytehq/airbyte/actions/runs/4285138836
Python tests coverage:

Name                                  Stmts   Miss  Cover
---------------------------------------------------------
source_tiktok_marketing/__init__.py       2      0   100%
source_tiktok_marketing/source.py        66      7    89%
source_tiktok_marketing/streams.py      361     41    89%
---------------------------------------------------------
TOTAL                                   429     48    89%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [6] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_incremental.py:212: Skipping new incremental test based on acceptance-test-config.yml
================== 82 passed, 8 skipped in 2197.32s (0:36:37) ==================

darynaishchenko · 2023-02-27T18:18:05Z

/publish connector=connectors/source-tiktok-marketing

🕑 Publishing the following connectors:
connectors/source-tiktok-marketing
https://github.com/airbytehq/airbyte/actions/runs/4285537376

Connector	Did it publish?	Were definitions generated?
connectors/source-tiktok-marketing	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…ytehq#23091) * added handling of 504 error * added changelog * fix log message * updated expected records * added allowedHosts and fix formatting * auto-bump connector version --------- Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

added handling of 504 error

b2659ba

darynaishchenko self-assigned this Feb 15, 2023

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/tiktok-marketing labels Feb 15, 2023

added changelog

c33c0cc

Merge branch 'master' into daryna/source-tiktok-marketing/fix-JSONDec…

1b95d0e

…odeError

fix log message

ccf83ff

updated expected records

0e8f7ba

darynaishchenko requested review from lazebnyi and bazarnov February 21, 2023 08:25

bazarnov approved these changes Feb 21, 2023

View reviewed changes

darynaishchenko requested a review from maxi297 February 21, 2023 12:58

Merge branch 'master' into daryna/source-tiktok-marketing/fix-JSONDec…

41086b9

…odeError

lazebnyi requested changes Feb 22, 2023

View reviewed changes

maxi297 approved these changes Feb 22, 2023

View reviewed changes

added allowedHosts and fix formatting

a41e117

darynaishchenko requested a review from lazebnyi February 22, 2023 16:37

Merge branch 'master' into daryna/source-tiktok-marketing/fix-JSONDec…

f36b6d0

…odeError

lazebnyi approved these changes Feb 22, 2023

View reviewed changes

Merge branch 'master' into daryna/source-tiktok-marketing/fix-JSONDec…

8a7dc65

…odeError

auto-bump connector version

5f6b6d8

darynaishchenko merged commit 806c4b1 into master Feb 27, 2023

darynaishchenko deleted the daryna/source-tiktok-marketing/fix-JSONDecodeError branch February 27, 2023 21:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source TikTok-Marketing: added more clear message for 504 error #23091

Source TikTok-Marketing: added more clear message for 504 error #23091

darynaishchenko commented Feb 15, 2023

darynaishchenko commented Feb 15, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 16, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 16, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 17, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 20, 2023 •

edited by github-actions bot

Loading

lazebnyi left a comment

maxi297 left a comment

maxi297 Feb 22, 2023

maxi297 Feb 22, 2023

darynaishchenko Feb 22, 2023

darynaishchenko commented Feb 27, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 27, 2023 •

edited by github-actions bot

Loading

Source TikTok-Marketing: added more clear message for 504 error #23091

Source TikTok-Marketing: added more clear message for 504 error #23091

Conversation

darynaishchenko commented Feb 15, 2023

darynaishchenko commented Feb 15, 2023 • edited by github-actions bot Loading

Build Failed

darynaishchenko commented Feb 16, 2023 • edited by github-actions bot Loading

Build Failed

darynaishchenko commented Feb 16, 2023 • edited by github-actions bot Loading

Build Failed

darynaishchenko commented Feb 17, 2023 • edited by github-actions bot Loading

Build Failed

darynaishchenko commented Feb 20, 2023 • edited by github-actions bot Loading

Build Passed

lazebnyi left a comment

Choose a reason for hiding this comment

maxi297 left a comment

Choose a reason for hiding this comment

maxi297 Feb 22, 2023

Choose a reason for hiding this comment

maxi297 Feb 22, 2023

Choose a reason for hiding this comment

darynaishchenko Feb 22, 2023

Choose a reason for hiding this comment

darynaishchenko commented Feb 27, 2023 • edited by github-actions bot Loading

Build Passed

darynaishchenko commented Feb 27, 2023 • edited by github-actions bot Loading

darynaishchenko commented Feb 15, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 16, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 16, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 17, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 20, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 27, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Feb 27, 2023 •

edited by github-actions bot

Loading