Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): Ensure external hooks post workflow execute run in queue mode #7947

Conversation

krynble
Copy link
Contributor

@krynble krynble commented Dec 7, 2023

Summary

Since 1.8.x a refactor removed the call to workflow.postExecute's External hook from the execution path. This PR adds it back to the correct place, where workers are supposed to call this, allowing us to avoid having to re-read the execution data in the caller just for the hooks.

It is important to have the hooks running in the worker whenever possible to prevent having to read the full execution data in the caller.

How to test the change:

  1. Use the attached hooks file external-hooks.txt setting it up via environment variable using export EXTERNAL_HOOK_FILES=/path/to/hooks/external-hooks.js
  2. Set up queue mode loading this file in both main and workers
  3. See that the message logs will not be displayed without this fix

Issues fixed

Include links to Github issue or Community forum post or Linear ticket:

Review / Merge checklist

  • PR title and summary are descriptive. Remember, the title automatically goes into the changelog. Use (no-changelog) otherwise. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.

    A bug is not considered fixed, unless a test is added to prevent it from happening again. A feature is not complete without tests.

    (internal) You can use Slack commands to trigger e2e tests or deploy test instance or deploy early access version on Cloud.

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Dec 7, 2023
Copy link
Contributor

@ivov ivov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed working, thanks for the fix!

Copy link

cypress bot commented Dec 7, 2023

1 flaky test on run #3227 ↗︎

0 297 5 0 Flakiness 1

Details:

🌳 🖥️ browsers:node18.12.0-chrome107 🤖 krynble 🗃️ e2e/*
Project: n8n Commit: bfdc68c3c0
Status: Passed Duration: 06:35 💡
Started: Dec 7, 2023 9:46 AM Ended: Dec 7, 2023 9:53 AM
Flakiness  cypress/e2e/27-two-factor-authentication.cy.ts • 1 flaky test

View Output Video

Test Artifacts
Two-factor authentication > Should be able to login with MFA token Screenshots Video

Review all test suite changes for PR #7947 ↗︎

Copy link
Contributor

github-actions bot commented Dec 7, 2023

✅ All Cypress E2E specs passed

@krynble krynble merged commit 3ba7deb into master Dec 7, 2023
21 checks passed
@krynble krynble deleted the pay-1102-bug-in-queue-mode-the-workflowpostexecute-hook-is-not branch December 7, 2023 10:00
This was referenced Dec 13, 2023
@netroy netroy changed the title fix: Ensure external hooks post workflow execute run in queue mode fix(core): Ensure external hooks post workflow execute run in queue mode Dec 13, 2023
ivov added a commit that referenced this pull request Dec 13, 2023
#
[1.21.0](https://github.com/n8n-io/n8n/compare/n8n@1.20.0...n8n@1.21.0)
(2023-12-13)


### Bug Fixes

* **core:** Ensure inviter and invitee are set correctly in invite link
([#7943](#7943))
([386bd61](386bd61))
* **core:** Fix user comparison in same-user subworkflow caller policy
([#7913](#7913))
([92bab72](92bab72))
* **core:** Perform multi-main leader check against key ID
([#7964](#7964))
([1a87f70](1a87f70))
* **core:** Ensure external hooks post workflow execute run in queue
mode ([#7947](#7947))
([3ba7deb](3ba7deb))
* **core:** Fix issue preventing secrets from loading if the path
contains - or / ([#7988](#7988))
([0ac9594](0ac9594))
* **core:** Restrict updating/deleting of shared but not owned
credentials ([#7950](#7950))
([42e828d](42e828d))
* **core:** Prevent workflow history saving error from happening
([#7812](#7812))
([e5581ce](e5581ce))
* **editor:** Add missing string for worker in log streaming
([#7971](#7971))
([148bc1d](148bc1d))
* **editor:** Allow SSH protocol in git repository URL for environments
([#7944](#7944))
([bc1c72f](bc1c72f))
* **editor:** Fix bug with node names with certain characters
([#8013](#8013))
([26f0d57](26f0d57))
* **editor:** Fix Webhook URL expansion icon
([#8011](#8011))
([b00b905](b00b905))
* **editor:** Prevent opening NDV search if `/` is typed in a
contenteditable element
([#7968](#7968))
([e8a493f](e8a493f))
* **editor:** Return early in ws message handler if no 'command' keyword
is found ([#7946](#7946))
([5b2defc](5b2defc))
* **FileMaker Node:** Prevent erroring on zero fields loaded
([#7955](#7955))
([10ad386](10ad386))
* **Google Sheets Node:** Prevent erroring on zero sheet search results
([#7957](#7957))
([9b877a9](9b877a9))
* **Google Sheets Node:** Prevent erroring when fetching mapping columns
([#7972](#7972))
([29a1066](29a1066))
* **Postgres Node:** Do not include id column in upsert fields selection
if it's not unique ([#7975](#7975))
([435392c](435392c))
* **Postgres Trigger Node:** Increase manual trigger timeout from 30 to
60 seconds ([#8015](#8015))
([09a5729](09a5729))
* **Webhook Node:** Binary data handling
([#7804](#7804))
([565b409](565b409))
* **Webhook Node:** Do not create binary data when there is no data in
the request ([#8000](#8000))
([70f0755](70f0755))


### Features

* **core:** Add config option for external secret update interval
([#7995](#7995))
([b6c1c04](b6c1c04))
* AI nodes usability fixes + Summarization Chain V2
([#7949](#7949))
([dcf1286](dcf1286))
* **editor:** Data transformation nodes and actions in Nodes Panel
([#7760](#7760))
([675ec21](675ec21))
* **editor:** Add AppCues tracking for onboarding event
([#7945](#7945))
([04cabaf](04cabaf))
* **editor:** Add option to disable NDV in workflow previews
([#7990](#7990))
([393afef](393afef))
* **editor:** Filter component + implement in If node
([#7490](#7490))
([8a53434](8a53434))
* **editor:** Show template credential setup based on feature flag
([#7989](#7989))
([08ee307](08ee307))
* **editor:** Introduce advanced permissions
([#7844](#7844))
([dbd62a4](dbd62a4))
* **Google Ads Node:** Update to support v15
([#7962](#7962))
([7f01269](7f01269))
* **Local File Trigger Node:** Add polling option typically good to
watch network files/folders
([#7942](#7942))
([2fbdfec](2fbdfec))
* **n8n Form Trigger Node:** Improvements
([#7571](#7571))
([953a58f](953a58f))

Co-authored-by: ivov <ivov@users.noreply.github.com>
@janober
Copy link
Member

janober commented Dec 13, 2023

Got released with n8n@1.21.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants