Repository navigation
source-salesforce: performance tuning - #67509
Maxime Carbonneau-Leclerc (maxi297) merged 8 commits into
Conversation
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Helpful Resources
PR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
|
|
/bump-version changelog="Minor tuning to increase sync speed."
|
|
Maxime Carbonneau-Leclerc (maxi297)
left a comment
There was a problem hiding this comment.
Have we run regression test on this? Do we want to pin some versions on connections we know might have a speed boost from this?
|
Deploy preview for airbyte-docs ready! ✅ Preview Built with commit d70c8f8. |
|
It seems like regression testing is hitting the same problem Brian Lai (@brianjlai) had. Can we run the regression testing with specific connections? I'll have a look into this issue tomorrow |
|
See https://airbytehq-team.slack.com/archives/C02UJ11D3S6/p1759936631631699 for more details about the regression tests failure. In the meanwhile, can we re-run regression tests with a couple of specific connections? |
|
new approach. I plan to publish a pre-release (dev) version and pin some connections. Pre-release publish workflow: https://github.com/airbytehq/airbyte/actions/runs/18381278400 |
Maxime Carbonneau-Leclerc (maxi297)
left a comment
There was a problem hiding this comment.
Waiting for a confirmation from David Gold (@dbgold17) but I would start the rollout next Monday
|
🚢 it |
|
Regression tests: https://github.com/airbytehq/airbyte/actions/runs/18820177051 EDIT: Issue with byos. Gonna use https://github.com/airbytehq/airbyte/actions/runs/18820761165 |
|
https://github.com/airbytehq/airbyte-platform-internal/pull/18170 has been merged. Retriggering regression tests on https://github.com/airbytehq/airbyte/actions/runs/18849208256 |
|
Regression tests did not run because it can't find connections. In the meanwhile, I've tested pinning versions for a couple of users and everything seemed fine. I'll move forward with the progressive rollout and investigate the regression test issue later today |
1e9b6b5
into
master
Co-authored-by: maxime.c <maxime@airbyte.io>
What
Per a discussion with Maxime Carbonneau-Leclerc (@maxi297), we believe increasing the maximum number of workers and maximum number of concurrent async jobs will speed up connector performance, especially for larger syncs.
An added benefit of making this change is it should allow us to test out the ability of the new connector performance dashboard to measure the impact.
EDIT (max): I've also added the normalization performance improvements as part of this PR. This should address https://github.com/airbytehq/oncall/issues/9850
Can this PR be safely reverted and rolled back?