-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Improve][Zeta] Add classloader cache mode to fix metaspace leak #6355
Conversation
public void testFakeSourceToInMemorySink() throws IOException, InterruptedException { | ||
LOG.info("test classloader with cache mode: {}", cacheMode()); | ||
for (int i = 0; i < 10; i++) { | ||
// load in memory sink which already leak thread with classloader | ||
Container.ExecResult execResult = executeJob(server, CONF_FILE); | ||
Assertions.assertEquals(0, execResult.getExitCode()); | ||
Assertions.assertTrue(containsDaemonThread()); | ||
if (cacheMode()) { | ||
Assertions.assertEquals(3, getClassLoaderCount()); | ||
} else { | ||
Assertions.assertEquals(2 + i, getClassLoaderCount()); | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The difference of cache mode enable or disable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great! LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good feature!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
waiting #6356 |
# Conflicts: # seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/master/JobMaster.java
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Purpose of this pull request
This PR try to do two thing.
classloader-cache-mode
inseatunnel.yaml
. This config is mainly to solve the leakage problem of classloader due to the failure of static thread to be destroyed normally. Through this configuration, all different jobs but using the same jar path will reuse the same classloader to reduce the creation and leakage of static threads.Does this PR introduce any user-facing change?
no
How was this patch tested?
add new test.
Check list
New License Guide
release-note
.