🐛 Source Salesforce: make paging work, if `cursor_field` is not changed inside one page by grubberr · Pull Request #17615 · airbytehq/airbyte

Serhii Chvaliuk (grubberr) · 2022-10-05T16:26:55Z

What

This PR also related to this oncall https://github.com/airbytehq/oncall/issues/735

How

Data:

cursor_field=2022-10-01, Pk=1
cursor_field=2022-10-01, Pk=3
cursor_field=2022-10-01, Pk=5
cursor_field=2022-10-02, Pk=2
cursor_field=2022-10-02, Pk=4
cursor_field=2022-10-02, Pk=6

New querying with page_size=2

SELECT * FROM table WHERE cursor_field >= 2022-10-01 ORDER BY cursor_field, pk LIMIT 2

-> cursor_field=2022-10-01, Pk=1
-> cursor_field=2022-10-01, Pk=3

SELECT * FROM table WHERE (cursor_field == 2022-10-01 AND pk > 3) OR (cursor_field > 2022-10-01) ORDER BY cursor_field, pk LIMIT 2

-> cursor_field=2022-10-01, Pk=5
-> cursor_field=2022-10-02, Pk=2

SELECT * FROM table WHERE cursor_field >= 2022-10-02 ORDER BY cursor_field, pk LIMIT 2

-> cursor_field=2022-10-02, Pk=2 (duplicate, it was before this PR too)
-> cursor_field=2022-10-02, Pk=4

SELECT * FROM table WHERE (cursor_field == 2022-10-02 AND pk > 4) OR (cursor_field > 2022-10-02) ORDER BY cursor_field, pk LIMIT 2

-> cursor_field=2022-10-02, Pk=6

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) · 2022-10-05T16:30:19Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3191255626
❌ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3191255626
🐛 https://gradle.com/s/nm3lxqybi4noq

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/api_test.py::test_rate_limit_bulk - KeyError: None
	 �[31m============ �[31m�[1m1 failed�[0m, �[32m69 passed�[0m, �[33m223 warnings�[0m�[31m in 92.00s (0:01:31)�[0m�[31m =============�[0m

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) · 2022-10-06T08:06:31Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3195547328
❌ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3195547328
🐛 https://gradle.com/s/2333co4ldyzle

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
================== 1 failed, 29 passed in 2028.95s (0:33:48) ===================

Serhii Chvaliuk (grubberr) · 2022-10-06T08:57:50Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3195862011
✅ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3195862011
Python tests coverage:

Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/__init__.py            2      0   100%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/api.py               151     19    87%
source_salesforce/streams.py           309     68    78%
source_salesforce/rate_limiting.py      22      6    73%
source_salesforce/source.py             95     40    58%
source_salesforce/utils.py               8      7    12%
--------------------------------------------------------
TOTAL                                  595    141    76%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/utils.py               8      0   100%
source_salesforce/__init__.py            2      0   100%
source_salesforce/api.py               151     14    91%
source_salesforce/source.py             95      9    91%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/rate_limiting.py      22      3    86%
source_salesforce/streams.py           309     47    85%
--------------------------------------------------------
TOTAL                                  595     74    88%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       152     26    83%   21-23, 29-31, 36-43, 48-61, 239, 250-258
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1358    466    66%

Build Passed

Test summary info:

All Passed

Ana Loureiro (anamargaridarl) · 2022-10-06T09:55:04Z

+            order_by_fields = [self.cursor_field]
+            if self.primary_key:
+                order_by_fields.append(self.primary_key)
+            query += f"ORDER BY {','.join(order_by_fields)} ASC LIMIT {self.page_size}"


Could there be a case where:
date=2 primarykey=2
date=3 primarykey=1
If so, wouldn't we miss the second entry?

yes it seems you are right

it seems we need only add primarykey to filtering if we inside that same date batch

OMG things become even more complex

Yes, because the order by date is mandatory to make the incremental process work. (since we need the most recent date to be saved in the state)

I'm under the impression this discussion is resolved with the upstream logic. Isn't it?

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) · 2022-10-06T10:47:02Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3196541838
✅ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3196541838
Python tests coverage:

Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/__init__.py            2      0   100%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/api.py               151     19    87%
source_salesforce/streams.py           310     68    78%
source_salesforce/rate_limiting.py      22      6    73%
source_salesforce/source.py             95     40    58%
source_salesforce/utils.py               8      7    12%
--------------------------------------------------------
TOTAL                                  596    141    76%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/utils.py               8      0   100%
source_salesforce/__init__.py            2      0   100%
source_salesforce/api.py               151     14    91%
source_salesforce/source.py             95      9    91%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/rate_limiting.py      22      3    86%
source_salesforce/streams.py           310     47    85%
--------------------------------------------------------
TOTAL                                  596     74    88%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       152     26    83%   21-23, 29-31, 36-43, 48-61, 239, 250-258
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1358    466    66%

Build Passed

Test summary info:

All Passed

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) · 2022-10-06T11:49:06Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3196916062
✅ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3196916062
Python tests coverage:

Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/__init__.py            2      0   100%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/api.py               151     19    87%
source_salesforce/streams.py           309     66    79%
source_salesforce/rate_limiting.py      22      6    73%
source_salesforce/source.py             95     40    58%
source_salesforce/utils.py               8      7    12%
--------------------------------------------------------
TOTAL                                  595    139    77%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/utils.py               8      0   100%
source_salesforce/__init__.py            2      0   100%
source_salesforce/api.py               151     14    91%
source_salesforce/source.py             95      9    91%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/rate_limiting.py      22      3    86%
source_salesforce/streams.py           309     45    85%
--------------------------------------------------------
TOTAL                                  595     72    88%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       152     26    83%   21-23, 29-31, 36-43, 48-61, 239, 250-258
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1358    466    66%

Build Passed

Test summary info:

All Passed

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) · 2022-10-06T17:41:02Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3199265541
❌ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3199265541
🐛 https://gradle.com/s/orlt6b2l5lsv4

Build Failed

Test summary info:

Could not find result summary

Serhii Chvaliuk (grubberr) · 2022-10-06T18:41:51Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3199637366
❌ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3199637366
🐛 https://gradle.com/s/6r6m3n3276ei2

Build Failed

Test summary info:

Could not find result summary

Brian Lai (brianjlai) · 2022-10-07T06:16:42Z

        query = f"SELECT {','.join(selected_properties.keys())} FROM {self.name} "
        if start_date:
-            query += f"WHERE {self.cursor_field} >= {start_date} "
+            if primary_key and self.name not in UNSUPPORTED_FILTERING_STREAMS:


I understand why we need to check primary_key here, but what is the purpose of self.name not in UNSUPPORTED_FILTERING_STREAMS as part of the condition? Is this intended to dictate whether or not the stream supports using a WHERE clause in query?

This block feels very confusing to read with a lot of repeated conditions which makes it a little hard to reason about and I'm trying to think of ways we can streamline this query builder a bit

Yes, good point. I agree it's not very obvious.

In really I just try to be consistent with this part of code, look at master branch:

https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-salesforce/source_salesforce/streams.py#L489

Because my "improved" query strictly relaies on ORDER BY
if ORDER BY is not used I also skip my "improved" query

Brian Lai (@brianjlai) Searching the documentation, some of those UNSUPPORTED_FILTERING_STREAMS fields support only the "where" filter which is consistent with the master logic. Though, I am not sure about it (since for those fields without the "order" the next date can be random ?).

I tried to find any refactors to the code to maintain that consistency but failed to do so.

I tried to find any refactors to the code to maintain that consistency but failed to do so.

We could add a supports_filtering attribute to BulkIncrementalSalesforceStream and set this attribute on instantiation after checking if name in UNSUPPORTED_FILTERING_STREAMS

Augustin (alafanechere) · 2022-10-07T12:19:16Z

Serhii Chvaliuk (@grubberr) feel free to ask for a review from my side once you have implemented Brian Lai (@brianjlai) and Ana Loureiro (@anamargaridarl) suggestions.

Ana Loureiro (anamargaridarl) · 2022-10-10T08:32:40Z

Serhii Chvaliuk (@grubberr) The PR can be reviewed by Augustin (@alafanechere) from my side

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Augustin (alafanechere)

Nice job, thank you! Minor suggestions

Augustin (alafanechere) · 2022-10-12T09:39:23Z

+            order_by_fields = [self.cursor_field]
+            if self.primary_key:
+                order_by_fields.append(self.primary_key)
+            query += f"ORDER BY {','.join(order_by_fields)} ASC LIMIT {self.page_size}"


I'm under the impression this discussion is resolved with the upstream logic. Isn't it?

Augustin (alafanechere) · 2022-10-12T09:44:04Z

+            {"Field1": "test", "Id": 3, "LastModifiedDate": LastModifiedDate1},
+            {"Field1": "test", "Id": 5, "LastModifiedDate": LastModifiedDate1},
+            {"Field1": "test", "Id": 2, "LastModifiedDate": LastModifiedDate2},
+            {"Field1": "test", "Id": 2, "LastModifiedDate": LastModifiedDate2},  # duplicate record


Why did you add a duplicate to the test?

I just want to show here that duplicates are possible
it's not related to my changes

All right, is it something you've already observed with the API?

it's possible because SF can store multiple records for the same cursor field.
Because we query using >= on subsequent queries we can grab duplicates

I mean 2-nd query can produce some result which was on 1-st query

Understood, can we deduplicate downstream in the connector or is it too complex? Do you mind opening an issue if you think it's something we can address later?

yes I think it's possible for incremental streams which have primary_key

we can play with ORDER BY cursor_field, primary_key
or maybe store in memory all primary_key for the latest cursor

Anyway, I think it is posible but I would prefer to do it in another issue
because it can take some time

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

req -> get_query Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) · 2022-10-12T11:50:41Z

/test connector=connectors/source-salesforce

🕑 connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3234427684
✅ connectors/source-salesforce https://github.com/airbytehq/airbyte/actions/runs/3234427684
Python tests coverage:

Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/__init__.py            2      0   100%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/api.py               151     19    87%
source_salesforce/streams.py           307     66    79%
source_salesforce/rate_limiting.py      22      6    73%
source_salesforce/source.py             95     40    58%
source_salesforce/utils.py               8      7    12%
--------------------------------------------------------
TOTAL                                  593    139    77%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_salesforce/utils.py               8      0   100%
source_salesforce/__init__.py            2      0   100%
source_salesforce/api.py               151     14    91%
source_salesforce/source.py             95      9    91%
source_salesforce/exceptions.py          8      1    88%
source_salesforce/rate_limiting.py      22      3    86%
source_salesforce/streams.py           307     42    86%
--------------------------------------------------------
TOTAL                                  593     69    88%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       152     26    83%   21-23, 29-31, 36-43, 48-61, 239, 250-258
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1358    466    66%

Build Passed

Test summary info:

All Passed

Serhii Chvaliuk (grubberr) · 2022-10-12T19:36:05Z

Augustin (@alafanechere) what you do think can I publish and merge this PR ?

Serhii Chvaliuk (grubberr) · 2022-10-13T05:29:05Z

/publish connector=connectors/source-salesforce

🕑 Publishing the following connectors:
connectors/source-salesforce
https://github.com/airbytehq/airbyte/actions/runs/3239888199

Connector	Did it publish?	Were definitions generated?
connectors/source-salesforce	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…ed inside one page (airbytehq#17615) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) added 7 commits October 4, 2022 15:42

bump 0.3.4

0e52368

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

github.md updated

6e6a014

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/oncall-729-source_github

8c51631

github.md updated

7ea2bb3

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

bump 1.0.21

f7a031b

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/17148-source-salesforce

91f8790

salesforce.md updated

8f4d648

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) self-assigned this Oct 5, 2022

Serhii Chvaliuk (grubberr) linked an issue Oct 5, 2022 that may be closed by this pull request

Salesforce source: Stuck in Jobs InProgress #17148

Closed

github-actions Bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Oct 5, 2022

Octavia Squidington IV (octavia-squidington-iv) added the connectors/source/salesforce label Oct 5, 2022

sunny (sh4sh) mentioned this pull request Oct 5, 2022

Salesforce source: Stuck in Jobs InProgress #17148

Closed

Serhii Chvaliuk (grubberr) added 2 commits October 6, 2022 09:23

Merge branch 'master' into grubberr/17148-source-salesforce

ca1277d

streams.py updated

a62e5cb

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Ana Loureiro (anamargaridarl) reviewed Oct 6, 2022

View reviewed changes

Serhii Chvaliuk (grubberr) changed the title ~~🐛 Source Salesforce: improve BULK paging~~ 🐛 Source Salesforce: make paging works, if cursor_field not changed in one page Oct 6, 2022

Serhii Chvaliuk (grubberr) changed the title ~~🐛 Source Salesforce: make paging works, if cursor_field not changed in one page~~ 🐛 Source Salesforce: make paging work, if cursor_field is not changed inside one page Oct 6, 2022

self.prev_start_date added

f0e34be

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) added 5 commits October 6, 2022 13:48

salesforce.md updated

6da5b01

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

add comments

2fba31e

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

fix comment

8d6264e

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

simplify

97a6525

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

continue to improve query

f65ab74

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Serhii Chvaliuk (grubberr) requested a review from Ana Loureiro (anamargaridarl) October 6, 2022 11:49

Serhii Chvaliuk (grubberr) added 3 commits October 6, 2022 20:27

test_bulk_stream_paging added

307ed35

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/17148-source-salesforce

8deba56

test_bulk_stream_paging updated

e8e7ee4

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Brian Lai (brianjlai) reviewed Oct 7, 2022

View reviewed changes

Merge branch 'master' into grubberr/17148-source-salesforce

66a88fe

Serhii Chvaliuk (grubberr) added 3 commits October 12, 2022 11:24

Merge branch 'master' into grubberr/17148-source-salesforce

ed4e304

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/17148-source-salesforce

1da774a

pylint

e2117c7

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Augustin (alafanechere) approved these changes Oct 12, 2022

View reviewed changes

Serhii Chvaliuk (grubberr) added 4 commits October 12, 2022 13:46

Merge branch 'master' into grubberr/17148-source-salesforce

cf9b166

LastModifiedDate1 -> last_modified_date1

e2d069b

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

m -> mocked_requests

d58e3f1

req -> get_query Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

pylint

9b89fed

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Merge branch 'master' into grubberr/17148-source-salesforce

e0e6fe8

auto-bump connector version [ci skip]

4cdade6

Serhii Chvaliuk (grubberr) merged commit 981190a into master Oct 13, 2022

Serhii Chvaliuk (grubberr) deleted the grubberr/17148-source-salesforce branch October 13, 2022 06:05

This was referenced Oct 13, 2022

Bump Airbyte version from 0.40.14 to 0.40.15 #17961

Closed

Bump Airbyte version from 0.40.14 to 0.40.15 #17970

Merged

Johan Hammarstedt (jhammarstedt) pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022

🐛 Source Salesforce: make paging work, if cursor_field is not chang…

a270198

…ed inside one page (airbytehq#17615) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Conversation

Serhii Chvaliuk (grubberr) commented Oct 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Data:

New querying with page_size=2

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 5, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ana Loureiro (anamargaridarl) Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Augustin (alafanechere) commented Oct 7, 2022

Uh oh!

Ana Loureiro (anamargaridarl) commented Oct 10, 2022

Uh oh!

Augustin (alafanechere) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Serhii Chvaliuk (grubberr) commented Oct 5, 2022 •

edited

Loading

Serhii Chvaliuk (grubberr) commented Oct 5, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 •

edited by github-actions Bot

Loading

Ana Loureiro (anamargaridarl) Oct 6, 2022 •

edited

Loading

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 6, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 12, 2022 •

edited by github-actions Bot

Loading

Serhii Chvaliuk (grubberr) commented Oct 13, 2022 •

edited by github-actions Bot

Loading