Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add core state lock deadlock detection config option #17020

Closed
wants to merge 6 commits into from

Conversation

troyready
Copy link
Contributor

Previously, a build flag could be specified to enable the go-deadlock library, but this change makes it a regular service config option.

It's possible that this is still too a bit too conservative of an approach: because go-deadlock has a disable option that is about as efficient as possible we could just use that instead of this conditional assignment.

@troyready
Copy link
Contributor Author

Testing here might also be too clever by half: I didn't want to punt on adding a test but also wanted to avoid mocking away too much from go-deadlock's logging/exit. There's perhaps a better approach in using go-deadlock's config options to not exit instead.

vault/expiration.go Outdated Show resolved Hide resolved
vault/expiration.go Outdated Show resolved Hide resolved
vault/expiration.go Outdated Show resolved Hide resolved
This was previously available via a build flag but will now be available
as a configuration option.
Previous implementation was unnecessary
Can use deadlock global options to avoid testing shenanigans
@troyready
Copy link
Contributor Author

Ah, sorry for the obtuseness @ncabatoff -- I had drawn some incorrect conclusions due to errant test results. I've simplified the implementation as discussed.

@troyready
Copy link
Contributor Author

I'm pretty sure the failing race condition test is expected due to the deadlock induced in the new test. I'm torn about best way to fix it:

  • Could disable the test execution during CircleCI's race-condition checking (e.g. a new environment variable set during those tests that causes the specific unit test to short-circuit)
  • Alternatively, could scale the new test back and not actually induce the deadlock. Maybe checking the config option is good enough?

@@ -3497,3 +3498,51 @@ func TestExpiration_listIrrevocableLeases_includeAll(t *testing.T) {
t.Errorf("bad lease count. expected %d, got %d", expectedNumLeases, numLeases)
}
}

func InduceDeadlock(t *testing.T, vaultcore *Core, expected uint32) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little weird to have this in expiration_test.go, and have it depend on vaultcore.expiration, since there's nothing specifically about expiration in the test itself.

@jasonodonnell
Copy link
Contributor

Closing in favor of #18604 .

@elliesterner elliesterner deleted the detect_deadlocks branch April 23, 2024 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants