Source Confluence: certificate to Beta #23988

darynaishchenko · 2023-03-13T16:05:20Z

resolved Source Confluence - Add page body in confluence connector #22410
increased test coverage

darynaishchenko · 2023-03-13T16:05:47Z

/test connector=connectors/source-confluence

🕑 connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4407033594
✅ connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4407033594
Python tests coverage:

Name                            Stmts   Miss  Cover
---------------------------------------------------
source_confluence/__init__.py       2      0   100%
source_confluence/source.py        79     19    76%
---------------------------------------------------
TOTAL                              81     19    77%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
======================== 33 passed, 2 skipped in 37.83s ========================

darynaishchenko · 2023-03-14T10:22:01Z

/test connector=connectors/source-confluence

🕑 connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4414642493
✅ connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4414642493
Python tests coverage:

Name                            Stmts   Miss  Cover
---------------------------------------------------
source_confluence/__init__.py       2      0   100%
source_confluence/source.py        79      1    99%
---------------------------------------------------
TOTAL                              81      1    99%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
======================== 33 passed, 2 skipped in 36.92s ========================

darynaishchenko · 2023-03-21T17:00:26Z

/test connector=connectors/source-confluence

🕑 connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4481770562
✅ connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4481770562
Python tests coverage:

Name                            Stmts   Miss  Cover
---------------------------------------------------
source_confluence/__init__.py       2      0   100%
source_confluence/source.py        79      1    99%
---------------------------------------------------
TOTAL                              81      1    99%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
======================== 33 passed, 2 skipped in 40.27s ========================

…ssions

darynaishchenko · 2023-03-21T17:33:03Z

/test connector=connectors/source-confluence

🕑 connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4482088511
✅ connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4482088511
Python tests coverage:

Name                            Stmts   Miss  Cover
---------------------------------------------------
source_confluence/__init__.py       2      0   100%
source_confluence/source.py        88      4    95%
---------------------------------------------------
TOTAL                              90      4    96%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
======================== 33 passed, 2 skipped in 41.01s ========================

lazebnyi

LGTM! Suggested small changes to stream method.

lazebnyi · 2023-03-21T18:56:16Z

airbyte-integrations/connectors/source-confluence/source_confluence/source.py

+        try:
+            response = requests.get(url, headers=auth.get_auth_header())
+            response.raise_for_status()
+            return [Pages(config), BlogPosts(config), Space(config), Group(config), Audit(config)]
+        except requests.exceptions.RequestException as e:
+            logger.warning(f"An exception occurred while trying to access Audit stream: {str(e)}. Skipping this stream.")
+            return [Pages(config), BlogPosts(config), Space(config), Group(config)]


Suggested change

try:

response = requests.get(url, headers=auth.get_auth_header())

response.raise_for_status()

return [Pages(config), BlogPosts(config), Space(config), Group(config), Audit(config)]

except requests.exceptions.RequestException as e:

logger.warning(f"An exception occurred while trying to access Audit stream: {str(e)}. Skipping this stream.")

return [Pages(config), BlogPosts(config), Space(config), Group(config)]

streams = [Pages(config), BlogPosts(config), Space(config), Group(config), Audit(config)]

try:

response = requests.get(url, headers=auth.get_auth_header())

response.raise_for_status()

except requests.exceptions.RequestException as e:

logger.warning(f"An exception occurred while trying to access Audit stream: {str(e)}. Skipping this stream.")

streams.pop()

finally:

return streams

Above described more pythonic way to Audit validation. But better way is add validation method what return bool value and depend on that we will add Audit stream to our streams list.

P-Code:

def account_plan_validation(self, config, auth): url = f"https://{config['domain_name']}/wiki/rest/api/audit?limit=1" is_primium_or_standard_plan = False try: response = requests.get(url, headers=auth.get_auth_header()) response.raise_for_status() is_primium_or_standard_plan = True except requests.exceptions.RequestException as e: logger.warning(f"An exception occurred while trying to access Audit stream: {str(e)}. Skipping this stream.") finally: return is_primium_or_standard_plan def streams(self, config: Mapping[str, Any]) -> List[Stream]: ... streams = [Pages(config), BlogPosts(config), Space(config), Group(config)] if self.account_plan_validation(config, auth): streams.append(Audit(config)) return streams

darynaishchenko · 2023-03-22T09:26:57Z

/test connector=connectors/source-confluence

🕑 connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4488312702
✅ connectors/source-confluence https://github.com/airbytehq/airbyte/actions/runs/4488312702
Python tests coverage:

Name                            Stmts   Miss  Cover
---------------------------------------------------
source_confluence/__init__.py       2      0   100%
source_confluence/source.py        94      3    97%
---------------------------------------------------
TOTAL                              96      3    97%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
======================== 33 passed, 2 skipped in 38.47s ========================

darynaishchenko · 2023-03-22T09:56:26Z

/publish connector=connectors/source-confluence

🕑 Publishing the following connectors:
connectors/source-confluence
https://github.com/airbytehq/airbyte/actions/runs/4488547652

Connector	Did it publish?	Were definitions generated?
connectors/source-confluence	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

* added view and storage to pages, updated SAT * added unittests * added changelog, fixed type in docs * added checking for stream Audit to skip it if user doesn't have permissions * refactored plan validation * auto-bump connector version --------- Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

added view and storage to pages, updated SAT

5a9bd6d

darynaishchenko self-assigned this Mar 13, 2023

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/confluence labels Mar 13, 2023

added unittests

a2c5253

added changelog, fixed type in docs

e8800b9

darynaishchenko requested review from davydov-d and lazebnyi March 15, 2023 15:01

lazebnyi approved these changes Mar 15, 2023

View reviewed changes

Merge branch 'master' into daryna/source-confluence/update-to-beta

b56d236

added checking for stream Audit to skip it if user doesn't have permi…

e92bd86

…ssions

darynaishchenko requested a review from lazebnyi March 21, 2023 17:51

lazebnyi approved these changes Mar 21, 2023

View reviewed changes

refactored plan validation

519cc01

auto-bump connector version

8b961c0

darynaishchenko merged commit af369de into master Mar 22, 2023

darynaishchenko deleted the daryna/source-confluence/update-to-beta branch March 22, 2023 10:32

sajarin mentioned this pull request Apr 18, 2023

Source Confluence: Added body details to API call #22410 #22663

Closed

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Confluence: certificate to Beta #23988

Source Confluence: certificate to Beta #23988

darynaishchenko commented Mar 13, 2023 •

edited

Loading

darynaishchenko commented Mar 13, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 14, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 21, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 21, 2023 •

edited by github-actions bot

Loading

lazebnyi left a comment

lazebnyi Mar 21, 2023

lazebnyi Mar 21, 2023

darynaishchenko commented Mar 22, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 22, 2023 •

edited by github-actions bot

Loading

Source Confluence: certificate to Beta #23988

Source Confluence: certificate to Beta #23988

Conversation

darynaishchenko commented Mar 13, 2023 • edited Loading

darynaishchenko commented Mar 13, 2023 • edited by github-actions bot Loading

Build Passed

darynaishchenko commented Mar 14, 2023 • edited by github-actions bot Loading

Build Passed

darynaishchenko commented Mar 21, 2023 • edited by github-actions bot Loading

Build Passed

darynaishchenko commented Mar 21, 2023 • edited by github-actions bot Loading

Build Passed

lazebnyi left a comment

Choose a reason for hiding this comment

lazebnyi Mar 21, 2023

Choose a reason for hiding this comment

lazebnyi Mar 21, 2023

Choose a reason for hiding this comment

darynaishchenko commented Mar 22, 2023 • edited by github-actions bot Loading

Build Passed

darynaishchenko commented Mar 22, 2023 • edited by github-actions bot Loading

darynaishchenko commented Mar 13, 2023 •

edited

Loading

darynaishchenko commented Mar 13, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 14, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 21, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 21, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 22, 2023 •

edited by github-actions bot

Loading

darynaishchenko commented Mar 22, 2023 •

edited by github-actions bot

Loading