Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Set up leader selection for multiple main instances #7527

Merged
merged 14 commits into from
Oct 30, 2023

Conversation

ivov
Copy link
Contributor

@ivov ivov commented Oct 26, 2023

https://linear.app/n8n/issue/PAY-933/set-up-leader-selection-for-multiple-main-instances

  • Set up new envs
  • Add config and license checks
  • Implement MultiMainInstancePublisher
  • Expand RedisServicePubSubPublisher to support MultiMainInstancePublisher
  • Init MultiMainInstancePublisher on startup and destroy on shutdown
  • Add to sandbox plans
  • Test manually

Note: This is only for setup - coordinating in reaction to leadership changes will come in later PRs.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 26, 2023

Great PR! Please pay attention to the following items before merging:

Files matching packages/**:

  • If fixing bug, added test to cover scenario.
  • If addressing forum or Github issue, added link to description.

Files matching packages/**/*.ts:

  • Added unit tests to cover new or updated functionality.

Make sure to check off this list before asking for review.

@codecov
Copy link

codecov bot commented Oct 26, 2023

Codecov Report

Attention: 86 lines in your changes are missing coverage. Please review.

Comparison is base (ea5cd31) 33.60% compared to head (ca7b40a) 33.59%.
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7527      +/-   ##
==========================================
- Coverage   33.60%   33.59%   -0.01%     
==========================================
  Files        3403     3404       +1     
  Lines      207709   207789      +80     
  Branches    22467    22487      +20     
==========================================
+ Hits        69795    69802       +7     
- Misses     136767   136840      +73     
  Partials     1147     1147              
Files Coverage Δ
...i/src/ExternalSecrets/ExternalSecretsManager.ee.ts 71.35% <100.00%> (ø)
packages/cli/src/config/schema.ts 83.33% <ø> (ø)
packages/cli/src/constants.ts 100.00% <ø> (ø)
packages/cli/src/controllers/e2e.controller.ts 0.00% <ø> (ø)
...li/src/eventbus/MessageEventBus/MessageEventBus.ts 45.45% <100.00%> (ø)
...ges/cli/src/services/orchestration.base.service.ts 73.07% <100.00%> (+1.07%) ⬆️
...ces/orchestration/main/handleCommandMessageMain.ts 70.27% <100.00%> (ø)
...es/cli/src/controllers/orchestration.controller.ts 0.00% <0.00%> (ø)
packages/cli/src/License.ts 42.85% <20.00%> (-0.85%) ⬇️
...orchestration/main/SingleMainInstance.publisher.ts 57.14% <57.14%> (ø)
... and 3 more

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Oct 26, 2023
@ivov ivov marked this pull request as ready for review October 27, 2023 10:19
Copy link
Contributor

@flipswitchingmonkey flipswitchingmonkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cypress
Copy link

cypress bot commented Oct 30, 2023

2 flaky tests on run #2626 ↗︎

0 262 0 0 Flakiness 2

Details:

🌳 🖥️ browsers:node18.12.0-chrome107 🤖 ivov 🗃️ e2e/*
Project: n8n Commit: ca7b40aab4
Status: Passed Duration: 00:42 💡
Started: Oct 30, 2023 12:32 PM Ended: Oct 30, 2023 1:32 PM
Flakiness  29-templates.cy.ts • 1 flaky test

View Output Video

Test Artifacts
Templates > can import template Output Screenshots Video
Flakiness  28-debug.cy.ts • 1 flaky test

View Output Video

Test Artifacts
Debug > should be able to debug executions Output Screenshots Video

Review all test suite changes for PR #7527 ↗︎

@github-actions
Copy link
Contributor

✅ All Cypress E2E specs passed

@ivov
Copy link
Contributor Author

ivov commented Oct 30, 2023

@flipswitchingmonkey Again please?

@github-actions
Copy link
Contributor

⚠️ Some Cypress E2E specs are failing, please fix them before merging

@github-actions
Copy link
Contributor

✅ All Cypress E2E specs passed

@ivov ivov merged commit 442c73e into master Oct 30, 2023
56 of 57 checks passed
@ivov ivov deleted the pay-933-set-up-leader-key-checks-for-leader-election branch October 30, 2023 15:22
@github-actions github-actions bot mentioned this pull request Nov 2, 2023
krynble added a commit that referenced this pull request Nov 2, 2023
#
[1.15.0](https://github.com/n8n-io/n8n/compare/n8n@1.14.0...n8n@1.15.0)
(2023-11-02)


### Bug Fixes

* **core:** Ensure execution deletion in worker lifecycle hook
([#7481](#7481))
([742c8a8](742c8a8))
* **core:** Fix data encryption on credentials import
([#7560](#7560))
([b350568](b350568))
* **core:** Fix issue that prevents owner logging in when using ldap
([#7408](#7408))
([479f902](479f902))
* **core:** Handle missing resultData in runData
([#7523](#7523))
([1055bd3](1055bd3))
* **core:** Permission check for subworkflow properly checking for
workflow settings ([#7576](#7576))
([437c95e](437c95e))
* **core:** Prevent executions from becoming forever running
([#7569](#7569))
([9bdb85c](9bdb85c))
* **core:** Upgrade crypto-js to address CVE-2023-46233
([#7519](#7519))
([65e5593](65e5593))
* **editor:** Do not truncate form inputs
([#7528](#7528))
([ae616f1](ae616f1))
* **editor:** Fix NDV close after using input select
([#7544](#7544))
([3b5e181](3b5e181))
* **editor:** Fix NDV unexpected re-render
([#7532](#7532))
([2853fcf](2853fcf))
* **editor:** Fix route component caching, incorrect use of array reduce
method and enable WF history feature
([#7434](#7434))
([12a89e6](12a89e6))
* **editor:** Fixes the issue that Switch Node can not be created
([#7516](#7516))
([df89685](df89685))
* **editor:** Handle `localStorage` being blocked/unavailable
([#7348](#7348))
([c05bc67](c05bc67))
* **Jira Software Node:** Handle missing issue types in issue types
loader ([#7534](#7534))
([9762705](9762705))
* **Switch Node:** Allow sortable Switch rules
([#7555](#7555))
([7a56e58](7a56e58))


### Features

* **core:** Add optional Error-Output
([#7460](#7460))
([655efea](655efea))
* **core:** Make queue mode settings configurable
([#7526](#7526))
([3d95b24](3d95b24))
* **core:** Set up leader selection for multiple main instances
([#7527](#7527))
([442c73e](442c73e))
* **editor:** Implement the `UserStack` design system component
([#7559](#7559))
([ce14f62](ce14f62))
* **HTTP Request Node:** Add pagination support
([#5993](#5993))
([cc2bd2e](cc2bd2e))
* **HTTP Request Node:** Update icon and default color
([#7572](#7572))
([ff279ab](ff279ab))
* **n8n Form Trigger Node:** Add text area and password input types
([#7474](#7474))
([b72040a](b72040a))
* **editor:** Dark mode is here! You can change it under personal
settings.([#6980](#6980))
([0746783](0746783))

---------

Co-authored-by: krynble <krynble@users.noreply.github.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
@github-actions github-actions bot mentioned this pull request Nov 2, 2023
krynble added a commit that referenced this pull request Nov 2, 2023
##
[1.15.1](https://github.com/n8n-io/n8n/compare/n8n@1.14.0...n8n@1.15.1)
(2023-11-02)


### Bug Fixes

* **core:** Ensure execution deletion in worker lifecycle hook
([#7481](#7481))
([742c8a8](742c8a8))
* **core:** Fix data encryption on credentials import
([#7560](#7560))
([b350568](b350568))
* **core:** Fix issue that prevents owner logging in when using ldap
([#7408](#7408))
([479f902](479f902))
* **core:** Handle missing resultData in runData
([#7523](#7523))
([1055bd3](1055bd3))
* **core:** Permission check for subworkflow properly checking for
workflow settings ([#7576](#7576))
([437c95e](437c95e))
* **core:** Prevent executions from becoming forever running
([#7569](#7569))
([9bdb85c](9bdb85c))
* **core:** Upgrade crypto-js to address CVE-2023-46233
([#7519](#7519))
([65e5593](65e5593))
* **editor:** Do not truncate form inputs
([#7528](#7528))
([ae616f1](ae616f1))
* **editor:** Fix NDV close after using input select
([#7544](#7544))
([3b5e181](3b5e181))
* **editor:** Fix NDV unexpected re-render
([#7532](#7532))
([2853fcf](2853fcf))
* **editor:** Fix route component caching, incorrect use of array reduce
method and enable WF history feature
([#7434](#7434))
([12a89e6](12a89e6))
* **editor:** Fixes the issue that Switch Node can not be created
([#7516](#7516))
([df89685](df89685))
* **editor:** Handle `localStorage` being blocked/unavailable
([#7348](#7348))
([c05bc67](c05bc67))
* Fix dark mode small issues
([#7573](#7573))
([1d81afc](1d81afc))
* **Jira Software Node:** Handle missing issue types in issue types
loader ([#7534](#7534))
([9762705](9762705))
* **Switch Node:** Allow sortable Switch rules
([#7555](#7555))
([7a56e58](7a56e58))


### Features

* **core:** Add optional Error-Output
([#7460](#7460))
([655efea](655efea))
* **core:** Make queue mode settings configurable
([#7526](#7526))
([3d95b24](3d95b24))
* **core:** Set up leader selection for multiple main instances
([#7527](#7527))
([442c73e](442c73e))
* **editor:** Implement the `UserStack` design system component
([#7559](#7559))
([ce14f62](ce14f62))
* **HTTP Request Node:** Add pagination support
([#5993](#5993))
([cc2bd2e](cc2bd2e))
* **HTTP Request Node:** Update icon and default color
([#7572](#7572))
([ff279ab](ff279ab))
* **n8n Form Trigger Node:** Add text area and password input types
([#7474](#7474))
([b72040a](b72040a))
* **editor:** Dark mode is here! You can change it under personal
settings.([#6980](#6980))
([0746783](0746783))

---------

Co-authored-by: krynble <krynble@users.noreply.github.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
@janober
Copy link
Member

janober commented Nov 2, 2023

Got released with n8n@1.15.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants