Skip to content

[fix][fn] Reorder Function Worker shutdown to stop scheduler before runtime manager#26136

Merged
lhotari merged 1 commit into
apache:masterfrom
Dream95:fix_fn-shutdown
Jul 2, 2026
Merged

[fix][fn] Reorder Function Worker shutdown to stop scheduler before runtime manager#26136
lhotari merged 1 commit into
apache:masterfrom
Dream95:fix_fn-shutdown

Conversation

@Dream95

@Dream95 Dream95 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #24358

Motivation

According to the available logs, it looks like a deadlock occurs when the scheduler thread holds a lock in FunctionRuntimeManager while trying to start a function. This blocks PulsarWorkerService.stop() during its close() call, so it never reaches the logic to stop the scheduler or coordinator. That's why the system keeps hanging during test teardowns.

Modifications

Reordered PulsarWorkerService.stop() to stop scheduling activity before stopping the runtime manager.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is already covered by existing tests.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

…untime manager

Signed-off-by: Dream95 <zhou_8621@163.com>
@lhotari lhotari merged commit 7a40ada into apache:master Jul 2, 2026
44 checks passed
lhotari pushed a commit that referenced this pull request Jul 2, 2026
…untime manager (#26136)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 7a40ada)
lhotari pushed a commit that referenced this pull request Jul 3, 2026
…untime manager (#26136)

Signed-off-by: Dream95 <zhou_8621@163.com>
(cherry picked from commit 7a40ada)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Deadlock in Function Worker shutdown

2 participants