Skip to content

sort task history updates by ExtendedTaskState enum ordinal, not timestamp#1188

Merged
ssalinas merged 3 commits into
masterfrom
out_of_order_updates
Aug 22, 2016
Merged

sort task history updates by ExtendedTaskState enum ordinal, not timestamp#1188
ssalinas merged 3 commits into
masterfrom
out_of_order_updates

Conversation

@ssalinas

@ssalinas ssalinas commented Aug 2, 2016

Copy link
Copy Markdown
Contributor

I found a few cases where it could happen that what we would consider the 'current state' of the task, is not the update that has the latest timestamp:

  • Rapid updates in succession, causing the history updates to have identical or out-of-order timestamps
  • cleaning triggered early in task lifecycle, followed quickly by a starting/running update will indefinitely leave the task in that state (not allowing it to clean/shut down properly)

This PR updates the sort called when fetching task histories to sort by the ExtendedTaskSTate ordinal instead of relying only on timestamp. This should give us the correct 'current' state in all cases since the enum is defined in the order we expect statuses to progress during the task lifecycle.

/cc @tpetr @stevenschlansker @subratbasnet #1187

}
}
Collections.sort(updates, comparator);
SingularityTaskHistoryUpdate lastUpdate = Iterables.getLast(updates);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could consider using Collections.max here for a slight efficiency gain, also avoids mutating an argument (which is usually dicey code style IMO)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 on Collections.max, updated

@tpetr

tpetr commented Aug 2, 2016

Copy link
Copy Markdown
Contributor

(FYI @wsorenson)

@ssalinas

ssalinas commented Aug 2, 2016

Copy link
Copy Markdown
Contributor Author

Another fyi, the original change was introduced by me back in #932 when trying to improve task search speed. The list of updates being returned by a new way of fetching them was an unmodifiable list, so I had resorted to checking timestamps. Should have been checking ordinals instead there

@grepsr

grepsr commented Aug 4, 2016

Copy link
Copy Markdown

I've tested this extensively yesterday with over a 100 tasks. The earlier problem did not recur. So looks like this working!

Thanks!

@ssalinas ssalinas modified the milestone: 0.10.0 Aug 4, 2016
@ssalinas ssalinas modified the milestones: 0.10.0, 0.11.0, 0.10.1 Aug 19, 2016
@ssalinas

Copy link
Copy Markdown
Contributor Author

Merging this for 0.10.1 bug fix release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants