Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 Source HubSpot: engagements stream, remove auto-generated properties #9385

Merged
merged 13 commits into from
Jan 13, 2022
Merged

🎉 Source HubSpot: engagements stream, remove auto-generated properties #9385

merged 13 commits into from
Jan 13, 2022

Conversation

grubberr
Copy link
Contributor

@grubberr grubberr commented Jan 10, 2022 •
edited
Loading

Signed-off-by: Sergey Chvalyuk grubberr@gmail.com

What

In current implementation we auto-generate schema with additional sub-object engagements.properties
but because engagements stream implemented with OLD legacy V1 API we don't need to do it.
The properties sub-object has to be added only for streams generated using NEW V3 API.

How

Describe the solution

Recommended reading order

  1. x.java
  2. y.python

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here
  • After the new connector version is published, connector version bumped in the seed directory as described here
  • Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

@grubberr
add_properties added
86d846f 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@github-actions github-actions bot added the area/connectors Connector related issues label Jan 10, 2022
@grubberr grubberr self-assigned this Jan 10, 2022
@grubberr grubberr temporarily deployed to more-secrets January 10, 2022 15:42 Inactive
@grubberr grubberr changed the title add_properties added 🎉 Source HubSpot: engagements stream, remove auto-generated properties Jan 10, 2022
@grubberr
Copy link
Contributor Author

grubberr commented Jan 10, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1678193978
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1678193978
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      6    92%
	 source_acceptance_test/conftest.py                     109    109     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              242     96    60%
	 source_acceptance_test/tests/test_full_refresh.py       38      0   100%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  54     17    69%
	 source_acceptance_test/utils/compare.py                 62     23    63%
	 source_acceptance_test/utils/connector_runner.py       110     48    56%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  979    404    59%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                         Stmts   Miss  Cover
	 ------------------------------------------------
	 source_hubspot/__init__.py       2      0   100%
	 source_hubspot/api.py          409     64    84%
	 source_hubspot/client.py        42     14    67%
	 source_hubspot/errors.py         6      0   100%
	 source_hubspot/source.py         4      0   100%
	 ------------------------------------------------
	 TOTAL                          463     78    83%

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 10, 2022 15:47 Inactive
@grubberr
hubspot.md updated
315fa35 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr grubberr linked an issue Jan 10, 2022 that may be closed by this pull request
🎉 Source Hubspot: engagements stream, remove auto-generated properties #9383
Closed
@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Jan 10, 2022
@grubberr grubberr temporarily deployed to more-secrets January 10, 2022 16:06 Inactive
@grubberr
fix airbyte_stream.default_cursor_field
204f1e8 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr
Merge branch 'master' into grubberr/9383-source-hubspot
f1551c7 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr
fix table layout
cae85fc 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr grubberr temporarily deployed to more-secrets January 11, 2022 12:10 Inactive
@grubberr
Copy link
Contributor Author

grubberr commented Jan 11, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1683448249
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1683448249
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      6    92%
	 source_acceptance_test/conftest.py                     109    109     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              242     96    60%
	 source_acceptance_test/tests/test_full_refresh.py       38      0   100%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  54     17    69%
	 source_acceptance_test/utils/compare.py                 62     23    63%
	 source_acceptance_test/utils/connector_runner.py       110     48    56%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  979    404    59%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                         Stmts   Miss  Cover
	 ------------------------------------------------
	 source_hubspot/__init__.py       2      0   100%
	 source_hubspot/api.py          419     69    84%
	 source_hubspot/client.py        40     12    70%
	 source_hubspot/errors.py         6      0   100%
	 source_hubspot/source.py         4      0   100%
	 ------------------------------------------------
	 TOTAL                          471     81    83%

@grubberr grubberr temporarily deployed to more-secrets January 11, 2022 16:14 Inactive
@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 11, 2022 16:16 Inactive
@grubberr
extend schemas/engagements.json
edb361d 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr grubberr temporarily deployed to more-secrets January 12, 2022 17:09 Inactive
@grubberr
fix unix newlines
87af927 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr grubberr temporarily deployed to more-secrets January 12, 2022 18:49 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 12, 2022 19:13 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 12, 2022 20:16 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 12, 2022 21:12 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 12, 2022 22:26 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 12, 2022 23:02 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 13, 2022 08:19 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 13, 2022 09:13 Inactive
@grubberr
Copy link
Contributor Author

/test connector=connectors/source-hubspot

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 13, 2022 13:37 Inactive
@grubberr
Copy link
Contributor Author

grubberr commented Jan 13, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1693063135
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1693063135
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      6    92%
	 source_acceptance_test/conftest.py                     109    109     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              242     96    60%
	 source_acceptance_test/tests/test_full_refresh.py       38      0   100%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  54     17    69%
	 source_acceptance_test/utils/compare.py                 62     23    63%
	 source_acceptance_test/utils/connector_runner.py       110     48    56%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  979    404    59%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                         Stmts   Miss  Cover
	 ------------------------------------------------
	 source_hubspot/__init__.py       2      0   100%
	 source_hubspot/api.py          419     69    84%
	 source_hubspot/client.py        40     12    70%
	 source_hubspot/errors.py         6      0   100%
	 source_hubspot/source.py         4      0   100%
	 ------------------------------------------------
	 TOTAL                          471     81    83%

@grubberr grubberr temporarily deployed to more-secrets January 13, 2022 14:04 Inactive
@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 13, 2022 14:04 Inactive
@grubberr
Copy link
Contributor Author

grubberr commented Jan 13, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1693147944
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1693147944

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets January 13, 2022 14:23 Inactive
@grubberr
source_specs.yaml updated
febfa3a 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
@grubberr grubberr temporarily deployed to more-secrets January 13, 2022 14:48 Inactive
@grubberr grubberr merged commit 9909cd6 into master Jan 13, 2022
@grubberr grubberr deleted the grubberr/9383-source-hubspot branch January 13, 2022 14:49
This was referenced Jan 13, 2022
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@midavadim midavadim midavadim approved these changes

@annalvova05 annalvova05 annalvova05 approved these changes

Assignees

@grubberr grubberr

Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

🎉 Source Hubspot: engagements stream, remove auto-generated properties
4 participants
@grubberr @midavadim @annalvova05 @octavia-squidington-iii