🎉 Source Zendesk-Chat: add oauth flow support #7313

bazarnov · 2021-10-24T15:04:45Z

What

#6287 - Source Zendesk Chat: support oauth
#8819 - Source Zendesk Chat: 'created_at' can be null for old items

How

added the spec.json annotation with oauth2.0
corrected authentication in source.py
added ZendeskChatOauthFlow.java
Fixed the 🐛 Source Zendesk chat: created_at field parsing error in Bans stream #8819 created_at for bans stream

OAuth Sreenshots of working flow:

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions
Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

antixar

Please check the old auth logic. I'm not sure to its current access_token is not compatible with oauths one.
And we should to test both config combinations. (old and new format)

airbyte-integrations/connectors/source-zendesk-chat/source_zendesk_chat/source.py

airbyte-integrations/connectors/source-zendesk-chat/source_zendesk_chat/spec.json

bazarnov · 2021-10-28T12:17:54Z

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1394541873
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1394541873
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                              Stmts   Miss  Cover
	 -----------------------------------------------------
	 main_dev.py                           6      6     0%
	 source_zendesk_chat/__init__.py       2      2     0%
	 source_zendesk_chat/api.py          124    124     0%
	 -----------------------------------------------------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zendesk-chat/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 source_zendesk_chat/source.py        23     23     0%
	 -----------------------------------------------------
	 TOTAL                               155    155     0%

bazarnov · 2021-12-22T13:28:59Z

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1611504703
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1611504703
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                              Stmts   Miss  Cover
	 -----------------------------------------------------
	 main_dev.py                           6      6     0%
	 -----------------------------------------------------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zendesk-chat/.venv/lib/python3.8/site-packages/coverage/control.py:768: CoverageWarning: No data was collected. (no-data-collected)
	 source_zendesk_chat/__init__.py       2      2     0%
	 source_zendesk_chat/source.py        26     26     0%
	 source_zendesk_chat/streams.py      146    146     0%
	 -----------------------------------------------------
	 TOTAL                               180    180     0%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      6    92%
	 source_acceptance_test/conftest.py                     109    109     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              242     96    60%
	 source_acceptance_test/tests/test_full_refresh.py       38      0   100%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  54     17    69%
	 source_acceptance_test/utils/compare.py                 62     23    63%
	 source_acceptance_test/utils/connector_runner.py       110     48    56%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  979    404    59%

Python short test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.7/site-packages/source_acceptance_test/tests/test_incremental.py:21: `future_state_path` not specified, skipping
======================== 20 passed, 1 skipped in 30.84s ========================

bazarnov · 2021-12-23T09:34:44Z

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1614987326
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1614987326
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      6    92%
	 source_acceptance_test/conftest.py                     109    109     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              242     96    60%
	 source_acceptance_test/tests/test_full_refresh.py       38      0   100%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  54     17    69%
	 source_acceptance_test/utils/compare.py                 62     23    63%
	 source_acceptance_test/utils/connector_runner.py       110     48    56%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  979    404    59%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                              Stmts   Miss  Cover
	 -----------------------------------------------------
	 main_dev.py                           6      6     0%
	 source_zendesk_chat/__init__.py       2      2     0%
	 source_zendesk_chat/source.py        26     26     0%
	 source_zendesk_chat/streams.py      146    146     0%
	 -----------------------------------------------------
	 TOTAL                               180    180     0%

Python short test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.7/site-packages/source_acceptance_test/tests/test_incremental.py:21: `future_state_path` not specified, skipping
======================== 24 passed, 1 skipped in 29.79s ========================

antixar

LGTM!
Old spec config are compatible with the new one. And there is a corresponding test.
@bazarnov , please, publish your changes

bazarnov · 2021-12-23T09:52:39Z

/publish connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1615045213
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1615045213

bazarnov added 8 commits October 20, 2021 17:20

added oauth support

0eb073f

Merge remote-tracking branch 'origin/master' into bazarnov/zd-chat-oauth

5d3077d

Merge remote-tracking branch 'origin/master' into bazarnov/zd-chat-oauth

2182be8

added java part of oauth

2e95066

changed spec.json to have credentials nested object

f292b68

corrected oauthFlow.java

dac667e

added title to start date in spec.json

a1118be

small corrections to OAuthFlow.java

3f2eeba

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Oct 24, 2021

added changelog

d8bb699

bazarnov linked an issue Oct 24, 2021 that may be closed by this pull request

Source Zendesk Chat: support oauth #6287

Closed

bazarnov self-assigned this Oct 24, 2021

bazarnov mentioned this pull request Oct 24, 2021

Source Zendesk Chat: implement oauth2 java-part #7314

Closed

alexandr-shegeda mentioned this pull request Oct 25, 2021

[EPIC] Implement OAuth Flow for Cloud connectors(duplicate) #7341

Closed

33 tasks

bazarnov requested review from midavadim and antixar October 25, 2021 17:22

antixar suggested changes Oct 27, 2021

View reviewed changes

airbyte-integrations/connectors/source-zendesk-chat/source_zendesk_chat/source.py Outdated Show resolved Hide resolved

airbyte-integrations/connectors/source-zendesk-chat/source_zendesk_chat/spec.json Show resolved Hide resolved

Merge remote-tracking branch 'origin/master' into bazarnov/zd-chat-oauth

2dd3c85

bazarnov temporarily deployed to more-secrets October 28, 2021 09:27 Inactive

updated after review

afd16be

bazarnov requested a review from antixar October 28, 2021 10:16

bazarnov temporarily deployed to more-secrets October 28, 2021 10:17 Inactive

add new secret to workflows

e6abf76

bazarnov temporarily deployed to more-secrets October 28, 2021 10:22 Inactive

formated

0ed2e1d

bazarnov temporarily deployed to more-secrets October 28, 2021 10:41 Inactive

jrhizor temporarily deployed to more-secrets October 28, 2021 11:31 Inactive

jrhizor temporarily deployed to more-secrets October 28, 2021 11:38 Inactive

bazarnov added 2 commits December 22, 2021 14:27

Merge remote-tracking branch 'origin/master' into bazarnov/zd-chat-oauth

a80bd88

updated

c4a6e62

bazarnov temporarily deployed to more-secrets December 22, 2021 12:54 Inactive

jrhizor temporarily deployed to more-secrets December 22, 2021 12:55 Inactive

jrhizor temporarily deployed to more-secrets December 22, 2021 13:13 Inactive

jrhizor temporarily deployed to more-secrets December 22, 2021 13:31 Inactive

merged with PR #9040

c24187c

bazarnov linked an issue Dec 22, 2021 that may be closed by this pull request

🐛 Source Zendesk chat: created_at field parsing error in Bans stream #8819

Closed

bazarnov temporarily deployed to more-secrets December 22, 2021 18:42 Inactive

grubberr mentioned this pull request Dec 22, 2021

🐛 Source Zendesk Chat: 'created_at' can be null for old items #9040

Closed

16 tasks

Merge remote-tracking branch 'origin/master' into bazarnov/zd-chat-oauth

a3d07e8

bazarnov temporarily deployed to more-secrets December 23, 2021 09:25 Inactive

added old config test in SAT

fcb19e0

bazarnov temporarily deployed to more-secrets December 23, 2021 09:34 Inactive

jrhizor temporarily deployed to more-secrets December 23, 2021 09:36 Inactive

antixar approved these changes Dec 23, 2021

View reviewed changes

bumped version

b8a5abd

bazarnov temporarily deployed to more-secrets December 23, 2021 09:53 Inactive

jrhizor temporarily deployed to more-secrets December 23, 2021 09:54 Inactive

new spec generated

8d4a37c

bazarnov temporarily deployed to more-secrets December 23, 2021 10:07 Inactive

bazarnov merged commit 5ed94a8 into master Dec 23, 2021

bazarnov deleted the bazarnov/zd-chat-oauth branch December 23, 2021 10:24

jrhizor mentioned this pull request Dec 27, 2021

Bump Airbyte version from 0.35.1-alpha to 0.35.2-alpha #9119

Merged

bazarnov mentioned this pull request Aug 17, 2022

Source Zendesk Chat: support oauth #15479

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Source Zendesk-Chat: add oauth flow support #7313

🎉 Source Zendesk-Chat: add oauth flow support #7313

bazarnov commented Oct 24, 2021 •

edited

Loading

antixar left a comment

bazarnov commented Oct 28, 2021 •

edited by github-actions bot

Loading

bazarnov commented Dec 22, 2021 •

edited by github-actions bot

Loading

bazarnov commented Dec 23, 2021 •

edited by github-actions bot

Loading

antixar left a comment

bazarnov commented Dec 23, 2021 •

edited by github-actions bot

Loading

🎉 Source Zendesk-Chat: add oauth flow support #7313

🎉 Source Zendesk-Chat: add oauth flow support #7313

Conversation

bazarnov commented Oct 24, 2021 • edited Loading

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

antixar left a comment

Choose a reason for hiding this comment

bazarnov commented Oct 28, 2021 • edited by github-actions bot Loading

bazarnov commented Dec 22, 2021 • edited by github-actions bot Loading

bazarnov commented Dec 23, 2021 • edited by github-actions bot Loading

antixar left a comment

Choose a reason for hiding this comment

bazarnov commented Dec 23, 2021 • edited by github-actions bot Loading

bazarnov commented Oct 24, 2021 •

edited

Loading

bazarnov commented Oct 28, 2021 •

edited by github-actions bot

Loading

bazarnov commented Dec 22, 2021 •

edited by github-actions bot

Loading

bazarnov commented Dec 23, 2021 •

edited by github-actions bot

Loading

bazarnov commented Dec 23, 2021 •

edited by github-actions bot

Loading