Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 Source Quickbooks: use number data type for decimal fields #4986

Merged
merged 15 commits into from
Aug 10, 2021
Merged

🐛 Source Quickbooks: use number data type for decimal fields #4986

merged 15 commits into from
Aug 10, 2021

Conversation

vitaliizazmic
Copy link
Contributor

What

Using number data type for decimal fields instead string

How

Created fork of singer tap, changed schemes. Also migrated Source to Airbyte CDK.

Closes #4292

Pre-merge Checklist

Expand the checklist which is relevant for this PR.

Connector checklist

  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • Secrets are annotated with airbyte_secret in the connector's spec
  • Credentials added to Github CI if needed and not already present. instructions for injecting secrets into CI.
  • Unit & integration tests added as appropriate (and are passing)
    • Community members: please provide proof of this succeeding locally e.g: screenshot or copy-paste acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • /test connector=connectors/<name> command as documented here is passing.
    • Community members can skip this, Airbyters will run this for you.
  • Code reviews completed
  • Documentation updated
    • README.md
    • docs/SUMMARY.md if it's a new connector
    • Created or updated reference docs in docs/integrations/<source or destination>/<name>.
    • Changelog in the appropriate page in docs/integrations/.... See changelog example
    • docs/integrations/README.md contains a reference to the new connector
    • Build status added to build page
  • Build is successful
  • Connector version bumped like described here
  • New Connector version released on Dockerhub by running the /publish command described here
  • No major blockers
  • PR merged into master branch
  • Follow up tickets have been created
  • Associated tickets have been closed & stakeholders notified

@github-actions github-actions bot added the area/connectors Connector related issues label Jul 26, 2021
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 26, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1068135021
❌ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1068135021

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 26, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1068194280
❌ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1068194280

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 26, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1068360204
✅ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1068360204

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 27, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1071744619
❌ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1071744619

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 28, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1073836599
❌ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1073836599

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 28, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1073861861
❌ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1073861861

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 28, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1075109284
✅ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1075109284

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 28, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1075181997
✅ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1075181997

keu
keu reviewed Jul 29, 2021
View reviewed changes
airbyte-integrations/connectors/source-quickbooks-singer/build.gradle Outdated
dependencies {
implementation files(project(':airbyte-integrations:bases:base-standard-source-test-file').airbyteDocker.outputs)
implementation files(project(':airbyte-integrations:bases:source-acceptance-test').airbyteDocker.outputs)
implementation files(project(':airbyte-integrations:bases:base-singer').airbyteDocker.outputs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't need docker image for singer anymore, we are using cdk PYPI package, which already includes singer wrappers

Suggested change
implementation files(project(':airbyte-integrations:bases:base-singer').airbyteDocker.outputs)

keu
keu reviewed Jul 29, 2021
View reviewed changes
airbyte-integrations/connectors/source-quickbooks-singer/acceptance-test-config.yml Outdated
empty_streams: ["budgets", "classes", "departments", "payment_methods", "transfers", "vendor_credits"]
# incremental:
# This test is commented out, because singer tap expects LastUpdatedTime for each stream in state.
# But records contain field MetaData with nested LastUpdatedTime. It causes test error.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not true, SAT supports absolute paths

keu
keu reviewed Jul 29, 2021
View reviewed changes
airbyte-integrations/connectors/source-quickbooks-singer/acceptance-test-docker.sh
@@ -0,0 +1,7 @@
#!/usr/bin/env sh
Copy link
Contributor

@keu keu Jul 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you update it in accordance to latest file from the templates?

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 29, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1078742263
✅ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1078742263

@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Jul 30, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1081167846
✅ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1081167846

@vitaliizazmic vitaliizazmic requested a review from keu July 30, 2021 05:50
@Phlair Phlair removed their request for review August 5, 2021 13:13
@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Aug 10, 2021
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Aug 10, 2021 •
edited by github-actions bot
Loading

/test connector=source-quickbooks-singer

🕑 source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1116137971
✅ source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1116137971

@jrhizor jrhizor temporarily deployed to more-secrets August 10, 2021 09:52 Inactive
@vitaliizazmic
Copy link
Contributor Author

vitaliizazmic commented Aug 10, 2021 •
edited by github-actions bot
Loading

/publish connector=connectors/source-quickbooks-singer

🕑 connectors/source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1116163078
✅ connectors/source-quickbooks-singer https://github.com/airbytehq/airbyte/actions/runs/1116163078

@jrhizor jrhizor temporarily deployed to more-secrets August 10, 2021 10:03 Inactive
@vitaliizazmic vitaliizazmic merged commit 6bcf811 into master Aug 10, 2021
@vitaliizazmic vitaliizazmic deleted the vitalii/4292_quickbooks_number_data_types branch August 10, 2021 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gaart gaart gaart approved these changes

@sherifnada sherifnada sherifnada approved these changes

@vovavovavovavova vovavovavovavova vovavovavovavova approved these changes

@keu keu Awaiting requested review from keu

@davinchia davinchia Awaiting requested review from davinchia

@tuliren tuliren Awaiting requested review from tuliren

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Source Quickbooks: Number data types are being transformed to strings
6 participants
@vitaliizazmic @gaart @keu @sherifnada @vovavovavovavova @jrhizor