New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🚨🚨 Source Hubspot - fixed incremental for engagement stream #27161
🚨🚨 Source Hubspot - fixed incremental for engagement stream #27161
Conversation
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
/test connector=connectors/source-hubspot
Build FailedTest summary info:
|
/test connector=connectors/source-hubspot
Build FailedTest summary info:
|
/test connector=connectors/source-hubspot
Build PassedTest summary info:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fill PR description
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Small refactoring required.
since_date = self._start_date | ||
if stream_state: | ||
since_date_timestamp = stream_state.get(self.updated_at_field) | ||
if since_date_timestamp: | ||
since_date = pendulum.from_timestamp(int(since_date_timestamp) / 1000) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More pythonic way)
since_date_timestamp = stream_state.get(self.updated_at_field) if stream_state else stream_state
since_date = pendulum.from_timestamp(int(since_date_timestamp) / 1000) if since_date_timestamp else self._start_date
@midavadim This is looking good. I noticed we checked
Do we have a link to the breaking change plan? |
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check. | ✅ |
Connector version increment check. | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check. | ✅ |
Connector version increment check. | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
@midavadim This will be scheduled for 06/29/23 at 12:00PM EST. I will send outreach today. Please let me know if anything changes! |
# Conflicts: # airbyte-integrations/connectors/source-hubspot/Dockerfile # airbyte-integrations/connectors/source-hubspot/metadata.yaml # docs/integrations/sources/hubspot.md
…emental-sync' into midavadim/26365-hubspot-fix-incremental-sync
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ❌ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ❌ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ❌ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ❌ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ❌ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
Coverage report for source-postgres
|
source-hubspot test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/source-hubspot/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
Code format checks | ✅ |
Connector package install | ✅ |
Build source-hubspot docker image for platform linux/x86_64 | ✅ |
Unit tests | ✅ |
Integration tests | ✅ |
Acceptance tests | ✅ |
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-hubspot test
What
Fixed increment sync for engagements stream,
How
'Recent' API is used for recent syncs of last recent 30 days and less than 10k records, otherwise full sync if performed by 'All' API
Recommended reading order
x.java
y.python
🚨 User Impact 🚨
Fixed schema for email_events stream. The correct type of "browser.version" field is list of strings.
Docs example also indicates the same type:
https://legacydocs.hubspot.com/docs/methods/email/get_events
"browser": {
"family": "Google Image Cache",
"name": "Google Image Cache",
"producer": "",
"producerUrl": "",
"type": "Proxy",
"url": "",
"version": []
},
For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.
If there are breaking changes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
Updating a connector
Community member or Airbyter
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
Connector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changes