Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

client: don't emit task shutdown delay event if not waiting #16281

Merged
merged 3 commits into from Mar 3, 2023
Merged

client: don't emit task shutdown delay event if not waiting #16281

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f8536a2
client: don't emit task shutdown delay event if not waiting
lgfa29 Mar 1, 2023
ad1884d
test: update `TestTaskRunner_NoShutdownDelay` to use `must`
lgfa29 Mar 3, 2023
4de2c6e
changelog: add entry for #16281
lgfa29 Mar 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 10 additions & 4 deletions client/allocrunner/taskrunner/task_runner.go
View file
Open in desktop
Expand Up @@ -985,10 +985,16 @@ func (tr *TaskRunner) handleKill(resultCh <-chan *drivers.ExitResult) *drivers.E
// This allows for things like service de-registration to run
// before waiting to kill task
if delay := tr.Task().ShutdownDelay; delay != 0 {
tr.logger.Debug("waiting before killing task", "shutdown_delay", delay)

ev := structs.NewTaskEvent(structs.TaskWaitingShuttingDownDelay).
SetDisplayMessage(fmt.Sprintf("Waiting for shutdown_delay of %s before killing the task.", delay))
var ev *structs.TaskEvent
if tr.alloc.DesiredTransition.ShouldIgnoreShutdownDelay() {
tr.logger.Debug("skipping shutdown_delay", "shutdown_delay", delay)
ev = structs.NewTaskEvent(structs.TaskSkippingShutdownDelay).
SetDisplayMessage(fmt.Sprintf("Skipping shutdown_delay of %s before killing the task.", delay))
} else {
tr.logger.Debug("waiting before killing task", "shutdown_delay", delay)
ev = structs.NewTaskEvent(structs.TaskWaitingShuttingDownDelay).
SetDisplayMessage(fmt.Sprintf("Waiting for shutdown_delay of %s before killing the task.", delay))
}
tr.UpdateState(structs.TaskStatePending, ev)

select {
Expand Down
10 changes: 10 additions & 0 deletions client/allocrunner/taskrunner/task_runner_test.go
View file
Open in desktop
Expand Up @@ -1108,6 +1108,16 @@ func TestTaskRunner_NoShutdownDelay(t *testing.T) {

err := <-killed
require.NoError(t, err, "killing task returned unexpected error")

// Check that we only emit the expected events.
hasEvent := false
for _, ev := range tr.state.Events {
require.NotEqual(t, structs.TaskWaitingShuttingDownDelay, ev.Type)
if ev.Type == structs.TaskSkippingShutdownDelay {
hasEvent = true
}
}
require.True(t, hasEvent)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker, but I think this kind of loop is what must.Contains was designed for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't quite make that work. The best I was able to do was something like this:

want := &structs.TaskEvent{Type: structs.TaskSkippingShutdownDelay}
must.SliceContainsFunc(t, tr.state.Events, want, func(a, b *structs.TaskEvent) bool {
	return a.Type == b.Type
})

But SliceNotContainsFunc, which I would have to use to check for the absence of TaskWaitingShuttingDownDelay doesn't exist yet, so I left the initial implementation but replacing require with must.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SliceNotContainsFunc

probably should exist, created shoenig/test#110

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks!

I was thinking about the ergonomics of this use case, and I think it could be helpful to have something like these:

s := []string{"hit", "miss"}

must.SliceAtLeastOne(s, func(i string) bool {
  return i == "hit"
}) // success

must.SliceAll(s, func(i string) bool {
  return i == "hit"
}) // fail

must.SliceNone(s, func(i string) bool {
  return i == "none"
}) // success

}

// TestTaskRunner_Dispatch_Payload asserts that a dispatch job runs and the
Expand Down
4 changes: 4 additions & 0 deletions nomad/structs/structs.go
View file
Open in desktop
Expand Up @@ -8348,6 +8348,10 @@ const (
// TaskWaitingShuttingDownDelay indicates that the task is waiting for
// shutdown delay before being TaskKilled
TaskWaitingShuttingDownDelay = "Waiting for shutdown delay"

// TaskSkippingShutdownDelay indicates that the task operation was
// configured to ignore the shutdown delay value set for the tas.
TaskSkippingShutdownDelay = "Skipping shutdown delay"
)

// TaskEvent is an event that effects the state of a task and contains meta-data
Expand Down