Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tr/service_hook: prevent Update from running before Poststart finish #7600

Merged
merged 3 commits into from Apr 6, 2020

Conversation

nickethier
Copy link
Member

The service hook requires fields initialized during Poststart when executing other lifecycle functions. In particular Update can be called at any time including before Poststart. This manifests as an error because the driver network information has not been set yet.

fixes #5767

@nickethier
Copy link
Member Author

Shout out to @42wim for doing the leg work to track down this bug 🎉

Copy link
Contributor

@notnoop notnoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one question about managing hook fields.

client/allocrunner/taskrunner/service_hook.go Outdated Show resolved Hide resolved
client/allocrunner/taskrunner/service_hook.go Outdated Show resolved Hide resolved
)

// Statically assert the stats hook implements the expected interfaces
var _ interfaces.TaskPoststartHook = (*serviceHook)(nil)
var _ interfaces.TaskExitedHook = (*serviceHook)(nil)
var _ interfaces.TaskPreKillHook = (*serviceHook)(nil)
var _ interfaces.TaskUpdateHook = (*serviceHook)(nil)

func TestUpdate_beforePoststart(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good simple test!

Copy link
Contributor

@notnoop notnoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - minor suggestions but merge away! Thanks!

@@ -164,6 +173,7 @@ func (h *serviceHook) Exited(context.Context, *interfaces.TaskExitedRequest, *in
h.mu.Lock()
defer h.mu.Unlock()
h.deregister()
h.initialRegistration = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, in Stop() or move to deregister?

require.NoError(t, hook.Poststart(context.Background(), &interfaces.TaskPoststartRequest{}, &interfaces.TaskPoststartResponse{}))
require.Len(t, c.GetOps(), 5)
require.NoError(t, hook.Update(context.Background(), &interfaces.TaskUpdateRequest{Alloc: alloc}, &interfaces.TaskUpdateResponse{}))
require.Len(t, c.GetOps(), 6)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also test PreKilled then update after the last update?

@nickethier nickethier merged commit 9df9e5e into master Apr 6, 2020
@nickethier nickethier deleted the b-5767 branch April 6, 2020 20:52
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Service hook fails sometimes
2 participants