Source SalesForce: move to next releaseStage #28781

artem1205 · 2023-07-27T14:14:58Z

What

Resolve:

How

bump airbyte_cdk version
handle API errors
refactor BULK API request params to align with slicing
add new field to spec: Force to use BULK API

🚨 User Impact 🚨

no breaking changes

Pre-merge Actions

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.

…cation

github-actions · 2023-07-27T14:15:22Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

This reverts commit bfdc7b6.

…cation-2 # Conflicts: # docs/integrations/sources/salesforce.md

octavia-squidington-iii · 2023-07-27T20:30:09Z

source-salesforce test report (commit `66273acff7`) - ❌

⏲️ Total pipeline duration: 02mn05s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	❌
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml

octavia-squidington-iii · 2023-07-27T20:38:59Z

source-salesforce test report (commit `66273acff7`) - ❌

⏲️ Total pipeline duration: 02mn12s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	❌
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	❌

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

octavia-squidington-iii · 2023-07-27T22:48:34Z

source-salesforce test report (commit `a4d939c239`) - ✅

⏲️ Total pipeline duration: 126mn09s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	✅
Acceptance tests	✅

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

brianjlai

Looks good so far. added a few comments and suggestions on testing

airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml

airbyte-integrations/connectors/source-salesforce/source_salesforce/source.py

airbyte-integrations/connectors/source-salesforce/unit_tests/api_test.py

docs/integrations/sources/salesforce.md

airbyte-integrations/connectors/source-salesforce/unit_tests/api_test.py

docs/integrations/sources/salesforce.md

octavia-squidington-iii · 2023-08-10T20:47:52Z

source-salesforce test report (commit `a1c4d00941`) - ❌

⏲️ Total pipeline duration: 123mn50s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

…er-certification-2' into artem1205/source-salesforce-silver-certification-2

octavia-squidington-iii · 2023-08-10T22:51:15Z

source-salesforce test report (commit `89a96de2fd`) - ❌

⏲️ Total pipeline duration: 91mn09s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

octavia-squidington-iii · 2023-08-11T07:36:25Z

source-salesforce test report (commit `fa150b263d`) - ❌

⏲️ Total pipeline duration: 28mn33s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

octavia-squidington-iii · 2023-08-11T09:49:56Z

source-salesforce test report (commit `143cb67501`) - ❌

⏲️ Total pipeline duration: 04mn10s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

This reverts commit 039a57d.

…cation-2 # Conflicts: # airbyte-integrations/connectors/source-salesforce/setup.py

octavia-squidington-iii · 2023-08-11T18:45:27Z

source-salesforce test report (commit `e35b650b68`) - ❌

⏲️ Total pipeline duration: 13mn50s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

octavia-squidington-iii · 2023-08-13T16:10:22Z

source-salesforce test report (commit `4ee7bc0e34`) - ❌

⏲️ Total pipeline duration: 92mn32s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

octavia-squidington-iii · 2023-08-13T18:59:13Z

source-salesforce test report (commit `dc45bcba83`) - ❌

⏲️ Total pipeline duration: 91mn32s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	❌
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

octavia-squidington-iii · 2023-08-13T22:23:06Z

source-salesforce test report (commit `a813d176a9`) - ✅

⏲️ Total pipeline duration: 101mn49s

Step	Result
Validate airbyte-integrations/connectors/source-salesforce/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅
Code format checks	✅
Connector package install	✅
Build source-salesforce docker image for platform linux/x86_64	✅
Unit tests	✅
Integration tests	✅
Acceptance tests	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-salesforce test

* Source SalesForce: handle bulk API errors * Source SalesForce: update docs * Source SalesForce: bump CDK version * Source SalesForce: raise config error * Source SalesForce: add stream slice test * Source SalesForce: add bulk pagination tests + fixes * Update Dockerfile * Source SalesForce: fix start_date in BULK request params * Source SalesForce: add pattern descriptor * Source SalesForce: update description in spec * Source SalesForce: fix stream_state < stream_slice * Source SalesForce: update comments * Source Salesforce: update docs + spec description * Source Salesforce: add unittest * Source Salesforce: add typing * Source Salesforce: fix import * Source Salesforce: fix formatting * Source Salesforce: remove extra comment * Apply suggestions from code review * Source Salesforce: fix unit tests * Source Salesforce: refactor --------- Co-authored-by: Sherif A. Nada <snadalive@gmail.com> Co-authored-by: Artem Inzhyyants <artem.inzhyyants@globallogic.com>

artem1205 and others added 14 commits July 12, 2023 20:17

Source SalesForce: bump airbyte_cdk version

104cf4b

Source SalesForce: handle bulk API errors

a536748

Source SalesForce: update docs

096ed10

Source SalesForce: bump CDK version

8916568

Merge branch 'master' into artem1205/source-salesforce-silver-certifi…

776f0b0

…cation

Source SalesForce: raise config error

8ffcf41

Source SalesForce: add stream slice test

9d34456

Source SalesForce: add bulk pagination tests + fixes

4ddb464

Source SalesForce: add logging + raise error instead of logs

bfdc7b6

Update Dockerfile

5cc3b04

Source SalesForce: fix start_date in BULK request params

e703670

Source SalesForce: add pattern descriptor

64e3f5c

Source SalesForce: update description in spec

f2c51bc

Source SalesForce: update docs

954445f

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/salesforce labels Jul 27, 2023

artem1205 added 3 commits July 27, 2023 16:16

Revert "Source SalesForce: add logging + raise error instead of logs"

83f5806

This reverts commit bfdc7b6.

Merge branch 'master' into artem1205/source-salesforce-silver-certifi…

0c7cf87

…cation-2 # Conflicts: # docs/integrations/sources/salesforce.md

Source Salesforce: update docs

66273ac

artem1205 marked this pull request as ready for review July 27, 2023 20:15

sherifnada reviewed Jul 27, 2023

View reviewed changes

airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml Show resolved Hide resolved

artem1205 added 2 commits July 27, 2023 22:35

Source Salesforce: Code format

c4f3fe1

Source Salesforce: Add test deps

a4d939c

brianjlai reviewed Jul 28, 2023

View reviewed changes

artem1205 marked this pull request as draft July 31, 2023 19:42

sherifnada added 2 commits August 10, 2023 12:34

Update salesforce.md

b162418

Update salesforce.md

e76dfe0

sherifnada reviewed Aug 10, 2023

View reviewed changes

docs/integrations/sources/salesforce.md Outdated Show resolved Hide resolved

Update docs/integrations/sources/salesforce.md

b206ee7

artem1205 marked this pull request as draft August 10, 2023 21:13

artem1205 added 2 commits August 10, 2023 23:15

Source Salesforce: refactor + update unit test

0fa9a2e

Merge remote-tracking branch 'origin/artem1205/source-salesforce-silv…

89a96de

…er-certification-2' into artem1205/source-salesforce-silver-certification-2

artem1205 marked this pull request as ready for review August 10, 2023 21:18

sherifnada approved these changes Aug 10, 2023

View reviewed changes

retrigger ci

fa150b2

retrigger ci

143cb67

retrigger ci

039a57d

octavia-squidington-iii added the connectors/source/linkedin-ads label Aug 11, 2023

artem1205 added 2 commits August 11, 2023 20:26

Revert "retrigger ci"

c72a74c

This reverts commit 039a57d.

Merge branch 'master' into artem1205/source-salesforce-silver-certifi…

e35b650

…cation-2 # Conflicts: # airbyte-integrations/connectors/source-salesforce/setup.py

retrigger ci

4ee7bc0

Source Zendesk Support: Fix integration test

dc45bcb

Source SalesForce: fix integration tests

a813d17

artem1205 merged commit c1b0692 into master Aug 14, 2023
21 of 22 checks passed

artem1205 deleted the artem1205/source-salesforce-silver-certification-2 branch August 14, 2023 07:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source SalesForce: move to next releaseStage #28781

Source SalesForce: move to next releaseStage #28781

artem1205 commented Jul 27, 2023 •

edited

github-actions bot commented Jul 27, 2023 •

edited by artem1205

octavia-squidington-iii commented Jul 27, 2023

octavia-squidington-iii commented Jul 27, 2023

octavia-squidington-iii commented Jul 27, 2023

brianjlai left a comment

octavia-squidington-iii commented Aug 10, 2023

octavia-squidington-iii commented Aug 10, 2023

octavia-squidington-iii commented Aug 11, 2023

octavia-squidington-iii commented Aug 11, 2023

octavia-squidington-iii commented Aug 11, 2023

octavia-squidington-iii commented Aug 13, 2023

octavia-squidington-iii commented Aug 13, 2023

octavia-squidington-iii commented Aug 13, 2023

Source SalesForce: move to next releaseStage #28781

Source SalesForce: move to next releaseStage #28781

Conversation

artem1205 commented Jul 27, 2023 • edited

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Actions

Community member or Airbyter

Airbyter

github-actions bot commented Jul 27, 2023 • edited by artem1205

Before Merging a Connector Pull Request

octavia-squidington-iii commented Jul 27, 2023

source-salesforce test report (commit 66273acff7) - ❌

octavia-squidington-iii commented Jul 27, 2023

source-salesforce test report (commit 66273acff7) - ❌

octavia-squidington-iii commented Jul 27, 2023

source-salesforce test report (commit a4d939c239) - ✅

brianjlai left a comment

Choose a reason for hiding this comment

octavia-squidington-iii commented Aug 10, 2023

source-salesforce test report (commit a1c4d00941) - ❌

octavia-squidington-iii commented Aug 10, 2023

source-salesforce test report (commit 89a96de2fd) - ❌

octavia-squidington-iii commented Aug 11, 2023

source-salesforce test report (commit fa150b263d) - ❌

octavia-squidington-iii commented Aug 11, 2023

source-salesforce test report (commit 143cb67501) - ❌

octavia-squidington-iii commented Aug 11, 2023

source-salesforce test report (commit e35b650b68) - ❌

octavia-squidington-iii commented Aug 13, 2023

source-salesforce test report (commit 4ee7bc0e34) - ❌

octavia-squidington-iii commented Aug 13, 2023

source-salesforce test report (commit dc45bcba83) - ❌

octavia-squidington-iii commented Aug 13, 2023

source-salesforce test report (commit a813d176a9) - ✅

artem1205 commented Jul 27, 2023 •

edited

github-actions bot commented Jul 27, 2023 •

edited by artem1205

source-salesforce test report (commit `66273acff7`) - ❌

source-salesforce test report (commit `66273acff7`) - ❌

source-salesforce test report (commit `a4d939c239`) - ✅

source-salesforce test report (commit `a1c4d00941`) - ❌

source-salesforce test report (commit `89a96de2fd`) - ❌

source-salesforce test report (commit `fa150b263d`) - ❌

source-salesforce test report (commit `143cb67501`) - ❌

source-salesforce test report (commit `e35b650b68`) - ❌

source-salesforce test report (commit `4ee7bc0e34`) - ❌

source-salesforce test report (commit `dc45bcba83`) - ❌

source-salesforce test report (commit `a813d176a9`) - ✅