Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source hubspot: Add new stream DealsArchived #24980

Merged
merged 17 commits into from Apr 10, 2023

Conversation

artem1205
Copy link
Collaborator

@artem1205 artem1205 commented Apr 7, 2023

What

Resolve #19838

How

add new stream for archived deals.
ClientSideIncrementalStream used as a base to bypass Hubspot api limitation on CRM search endpoint: 400 error is raised if archivedAt field used as a filter.

Recommended reading order

  1. y.python

🚨 User Impact 🚨

no breaking changes

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)

  • Secrets in the connector's spec are annotated with airbyte_secret

  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.

  • Code reviews completed

  • Connector version has been incremented

  • Documentation updated

    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

# Conflicts:
#	airbyte-integrations/connectors/source-hubspot/integration_tests/expected_records.jsonl
# Conflicts:
#	airbyte-integrations/connectors/source-hubspot/Dockerfile
# Conflicts:
#	airbyte-integrations/connectors/source-hubspot/integration_tests/abnormal_state.json
#	airbyte-integrations/connectors/source-hubspot/source_hubspot/streams.py
…ubspot-19838

# Conflicts:
#	airbyte-integrations/connectors/source-hubspot/integration_tests/expected_records.jsonl
@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Apr 7, 2023
# Conflicts:
#	airbyte-integrations/connectors/source-hubspot/Dockerfile
#	airbyte-integrations/connectors/source-hubspot/integration_tests/expected_records.jsonl
#	docs/integrations/sources/hubspot.md
@artem1205
Copy link
Collaborator Author

artem1205 commented Apr 7, 2023

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4637918802
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4637918802
🐛 https://gradle.com/s/lgscz5ecfpivs

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_source.py::test_streams - assert 27 == 26
	 �[31m================== �[31m�[1m1 failed�[0m, �[32m94 passed�[0m, �[33m422 warnings�[0m�[31m in 5.37s�[0m�[31m ==================�[0m

@artem1205
Copy link
Collaborator Author

artem1205 commented Apr 7, 2023

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4638093104
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4638093104
🐛 https://gradle.com/s/rrvxqaqlibx62

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream deals...
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
FAILED test_incremental.py::TestIncremental::test_state_with_abnormally_large_values[inputs0]
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
============= 4 failed, 37 passed, 1 skipped in 214.65s (0:03:34) ==============

@artem1205
Copy link
Collaborator Author

artem1205 commented Apr 7, 2023

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4638717305
❌ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4638717305
🐛 https://gradle.com/s/k3ekwqcycuxzk

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - Failed: Stream deals...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
============= 1 failed, 40 passed, 1 skipped in 383.79s (0:06:23) ==============

@artem1205
Copy link
Collaborator Author

artem1205 commented Apr 7, 2023

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4639020488
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/4639020488
Python tests coverage:

Name                          Stmts   Miss  Cover
-------------------------------------------------
source_hubspot/errors.py          6      0   100%
source_hubspot/constants.py       2      0   100%
source_hubspot/__init__.py        2      0   100%
source_hubspot/helpers.py        70      3    96%
source_hubspot/streams.py       894     77    91%
source_hubspot/source.py         60     19    68%
-------------------------------------------------
TOTAL                          1034     99    90%
Name                          Stmts   Miss  Cover
-------------------------------------------------
source_hubspot/errors.py          6      0   100%
source_hubspot/constants.py       2      0   100%
source_hubspot/__init__.py        2      0   100%
source_hubspot/helpers.py        70      7    90%
source_hubspot/source.py         60     14    77%
source_hubspot/streams.py       894    215    76%
-------------------------------------------------
TOTAL                          1034    236    77%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================== 41 passed, 1 skipped in 398.18s (0:06:38) ===================

@artem1205
Copy link
Collaborator Author

artem1205 commented Apr 10, 2023

/publish connector=connectors/source-hubspot

🕑 Publishing the following connectors:
connectors/source-hubspot
https://github.com/airbytehq/airbyte/actions/runs/4656096892


Connector Did it publish? Were definitions generated?
connectors/source-hubspot

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@artem1205 artem1205 requested a review from a team April 10, 2023 09:34
@artem1205 artem1205 merged commit 52ca199 into master Apr 10, 2023
24 of 25 checks passed
@artem1205 artem1205 deleted the artem1205/source-hubspot-19838 branch April 10, 2023 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/hubspot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Source HubSpot: not pick up archived entities
4 participants