Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-16053: Fix leaks of KDC server in tests #15079

Merged
merged 1 commit into from Dec 28, 2023

Conversation

divijvaidya
Copy link
Contributor

@divijvaidya divijvaidya commented Dec 27, 2023

Problem

We are facing OOM while running test suite for Apache Kafka as discussed in https://lists.apache.org/thread/d5js0xpsrsvhgjb10mbzo9cwsy8087x4

Changes

This JIRA fixes ~123MB of heap leak caused by DefaultDirectoryService objects in the heap memory. The DefaultDirectoryService is used by MiniKdc.scala. This change fixes a couple of leaks where the kdc service is not closed at the end of test run.

Screenshot 2023-12-27 at 13 18 33

@divijvaidya divijvaidya added tests Test fixes (including flaky tests) backport-candidate This pull request is a candidate to be backported to previous versions labels Dec 27, 2023
@divijvaidya
Copy link
Contributor Author

divijvaidya commented Dec 27, 2023

@dajac @jolshan please review. This is not the root cause of our recent OOM errors but is a contributor to it.

(cc: @stanislavkozlovski as a potential backport to 3.7)

Copy link
Contributor

@jolshan jolshan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of existing failures on this run -- but I didn't see any for the tests modified here.

@divijvaidya divijvaidya merged commit a56b63e into apache:trunk Dec 28, 2023
1 check failed
divijvaidya added a commit that referenced this pull request Dec 28, 2023
This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>
divijvaidya added a commit that referenced this pull request Dec 28, 2023
This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>
@showuon
Copy link
Contributor

showuon commented Dec 28, 2023

Nice catch!

@divijvaidya
Copy link
Contributor Author

backported to 3.6 and 3.7 branch

gaurav-narula pushed a commit to gaurav-narula/kafka that referenced this pull request Jan 24, 2024
This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>
anurag-harness pushed a commit to anurag-harness/kafka that referenced this pull request Feb 9, 2024
This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>
anurag-harness added a commit to anurag-harness/kafka that referenced this pull request Feb 9, 2024
… (#497)

This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>

Co-authored-by: Divij Vaidya <diviv@amazon.com>
yyu1993 pushed a commit to yyu1993/kafka that referenced this pull request Feb 15, 2024
This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>
clolov pushed a commit to clolov/kafka that referenced this pull request Apr 5, 2024
This commit closes the KDC server properly in `CustomQuotaCallbackTest` and `AclAuthorizerWithZkSaslTest`.

Reviewers: Justine Olshan <jolshan@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-candidate This pull request is a candidate to be backported to previous versions tests Test fixes (including flaky tests)
Projects
None yet
3 participants