Former master reports "failed to reassign persistent tasks" repeatedly #58531
Assignees
Labels
>bug
:Distributed/Task Management
Issues for anything around the Tasks API - both persistent and node level.
Team:Distributed
Meta label for distributed team
Comments
DaveCTurner
added
>bug
:Distributed/Task Management
|
Pinging @elastic/es-distributed (:Distributed/Task Management) |
|
This is a silly bug. If a persistent task cannot be immediately reassigned and has caused periodic rechecks to be scheduled (on the master node at that time) then those rechecks continue to be rescheduled regardless of whether the node is still the master. I will open a PR to fix it. |
droberts195
added a commit
to droberts195/elasticsearch
that referenced
this issue
Jun 25, 2020
If a persistent task cannot be assigned on the first attempt then the master node will schedule periodic rechecks to see if the assignment requirements have been met. These periodic rechecks should be cancelled if the node ceases to be master. Previously they weren't, leading to exceptions being logged repeatedly. This PR cancels the rechecks on learning that the node is no longer the master. Fixes elastic#58531
droberts195
added a commit
that referenced
this issue
Jun 25, 2020
If a persistent task cannot be assigned on the first attempt then the master node will schedule periodic rechecks to see if the assignment requirements have been met. These periodic rechecks should be cancelled if the node ceases to be master. Previously they weren't, leading to exceptions being logged repeatedly. This PR cancels the rechecks on learning that the node is no longer the master. Fixes #58531
droberts195
added a commit
that referenced
this issue
Jun 25, 2020
If a persistent task cannot be assigned on the first attempt then the master node will schedule periodic rechecks to see if the assignment requirements have been met. These periodic rechecks should be cancelled if the node ceases to be master. Previously they weren't, leading to exceptions being logged repeatedly. This PR cancels the rechecks on learning that the node is no longer the master. Fixes #58531
droberts195
added a commit
that referenced
this issue
Jun 25, 2020
If a persistent task cannot be assigned on the first attempt then the master node will schedule periodic rechecks to see if the assignment requirements have been met. These periodic rechecks should be cancelled if the node ceases to be master. Previously they weren't, leading to exceptions being logged repeatedly. This PR cancels the rechecks on learning that the node is no longer the master. Fixes #58531
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Following a slightly messy master election I saw a 7.6.1 cluster with the two unelected master nodes both reporting this message every 30 seconds apparently indefinitely:
Possibly there's a race in the
PersistentTasksClusterServicearound election time? Not sure, I am not very familiar with this area.The text was updated successfully, but these errors were encountered: