Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: shutdown deadlock #2944

Merged
merged 9 commits into from
May 16, 2022
Merged

fix: shutdown deadlock #2944

merged 9 commits into from
May 16, 2022

Conversation

notanatol
Copy link
Contributor

@notanatol notanatol commented May 10, 2022

Checklist

Description

Closes #2931
Closes #2926


This change is Reviewable

Copy link
Contributor

@mrekucci mrekucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 10 of 10 files at r1, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @notanatol)


pkg/goroutine/goroutine.go line 1 at r1 (raw file):

package goroutine

Missing header.

select {
case <-timer.C:
case <-time.After(1 * time.Second):
k.logger.Debugf("kademlia timer not drained after 1 second, goroutine dump...\n%s\n*** end\n", goroutine.Dump())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use pprof.Lookup("goroutine").WriteTo(os.Stdout, 1)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed it to use pprof

_ = pprof.Lookup("goroutine").WriteTo(out, 2)
}

func Stack() string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not used anywhere. do we need the new package to wrap a one-liner?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed, just syntactically looked nice.

removed

pkg/node/node.go Show resolved Hide resolved
pkg/node/node.go Outdated
@@ -969,6 +980,9 @@ func (b *Bee) Shutdown(ctx context.Context) error {

tryClose(b.apiCloser, "api")

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was 15s and now is 5s. it needs to be evaluated whether this is the right value, and what are the side effects of a non-clean api shutdown (since this propagates to other components that are later shut-down.

Copy link
Contributor Author

@notanatol notanatol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 11 files reviewed, 4 unresolved discussions (waiting on @acud)


pkg/node/node.go line 673 at r3 (raw file):

Previously, acud (acud) wrote…

this is actually an abnormal state but the select would just cause the software to continue booting, and only later shutting down. i think that actually returning an error here and causing the defer statement to initiate the shutdown would be more correct, no?

yes, returning an error now


pkg/node/node.go line 983 at r3 (raw file):

Previously, acud (acud) wrote…

this was 15s and now is 5s. it needs to be evaluated whether this is the right value, and what are the side effects of a non-clean api shutdown (since this propagates to other components that are later shut-down.

reverted

@notanatol notanatol added the ready for review The PR is ready to be reviewed label May 12, 2022
Copy link
Member

@acud acud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 10 files at r1, 2 of 4 files at r3, 2 of 2 files at r4, 1 of 1 files at r5, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @notanatol)

@acud
Copy link
Member

acud commented May 13, 2022

@notanatol can you push another commit so that the static analyzer can run again? I adjusted the rules. Should pass now

Copy link
Contributor Author

@notanatol notanatol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Reviewable status: 10 of 11 files reviewed, all discussions resolved (waiting on @acud and @mrekucci)

Copy link
Contributor Author

@notanatol notanatol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 10 files at r1, 1 of 4 files at r3, 1 of 2 files at r4, 1 of 1 files at r5, 1 of 1 files at r6, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @mrekucci)

Copy link
Member

@acud acud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, thanks

@notanatol notanatol merged commit c8316d4 into master May 16, 2022
@notanatol notanatol deleted the fix-shutdown-2 branch May 16, 2022 13:32
@istae istae added this to the 1.6.1 milestone Jun 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pull-request ready for review The PR is ready to be reviewed
Projects
None yet
5 participants