Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Source HubSpot: Fix empty string inside number / float datatype #5334

Merged
merged 8 commits into from
Aug 12, 2021

Conversation

bazarnov
Copy link
Collaborator

@bazarnov bazarnov commented Aug 11, 2021

What

#5293 - Source Hubspot fails in normalization step

How

  • added additional field_value verification for an empty string to api.py, if so - set the field_value to be zero, instead of empty string.
  • edited test_field_type_converting.py unit-test to simulate the issue of empty string inside numeric (float) type

NOTE:
This fix potentially could resolve the issue with Bad Double Value when using this connector along with Destination BigQuery for data replication. As far as BigQuery does not accept simple null values, and could not accept empty strings as numeric values, it requires additional data preprocessing before inject the data into the table.

Pre-merge Checklist

Updating a connector

Community member or Airbyter

  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

@github-actions github-actions bot added the area/connectors Connector related issues label Aug 11, 2021
@bazarnov bazarnov changed the title 🐛 Source HubSpot: Fix for bad double values when sync with BigQuery destination 🐛 Source HubSpot: Fix empty string inside number / float datatype Aug 11, 2021
@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Aug 11, 2021
@bazarnov
Copy link
Collaborator Author

bazarnov commented Aug 11, 2021

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1119528170
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1119528170

@bazarnov bazarnov self-assigned this Aug 11, 2021
@bazarnov bazarnov linked an issue Aug 11, 2021 that may be closed by this pull request
@jrhizor jrhizor temporarily deployed to more-secrets August 11, 2021 09:34 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets August 12, 2021 14:15 Inactive
@bazarnov
Copy link
Collaborator Author

bazarnov commented Aug 12, 2021

/test connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124238148
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124238148

@jrhizor jrhizor temporarily deployed to more-secrets August 12, 2021 14:24 Inactive
@bazarnov
Copy link
Collaborator Author

bazarnov commented Aug 12, 2021

/publish connector=connectors/source-hubspot

🕑 connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124341394
✅ connectors/source-hubspot https://github.com/airbytehq/airbyte/actions/runs/1124341394

@jrhizor jrhizor temporarily deployed to more-secrets August 12, 2021 14:55 Inactive
@bazarnov bazarnov merged commit 6f56833 into master Aug 12, 2021
@bazarnov bazarnov deleted the bazarnov/5293-hubspot-bad-double-values branch August 12, 2021 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Source Hubspot fails in normalization step.
3 participants