-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Improvement] Package sun.security.krb5 is not visible in Java 11 and 17 #625
Comments
Yes. But I don't have much time in this month, perhaps next month. |
No worries. It's not urgent. |
https://github.com/kaijchen/incubator-uniffle/actions/runs/4228912219/jobs/7344901292#step:5:2884
|
@kaijchen Can you assign this to me? |
Assigned, thanks for taking this issue. |
@kaijchen Thank you very much! |
I read this part of the code carefully, and I summarized the relevant information as follows: We have 2 ways to solve the issue.
The code is based on the hadoop trunk branch. The code is different from hadoop-2.8.5, but this does not affect the conclusion.
We use UserGroupInformation#loginUserFromKeytabAndReturnUGI
UserGroupInformation#doSubjectLogin
HadoopConfiguration#new HadoopLoginContext()
HadoopLoginContext#constructor
LoginContext#constructor(Configuration config is HadoopConfiguration)
LoginContext#constructor#init#getAppConfigurationEntry
after
|
cc @zuston |
Wow, really thanks for your great work. @slfan1989
+1.
This part of code is introduced in #184 . If I remember correctly, this is just for unit test to simulate setting the profile w/o, because all tests run in one JVM. If test cases could pass, I prefer removing this. |
Thanks @slfan1989 for the investigation, +1 for this proposal. I'll change the title of this issue. |
…11 and 17. (#726) ### What changes were proposed in this pull request? Try remove `sun.security.krb5.Config.refresh();` ### Why are the changes needed? `sun.security.krb5` is not visible in Java 11 and 17. We need to compile on JDK11 and JDK17 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test verification. Co-authored-by: slfan1989 <louj1988@@>
… Java 11 and 17. (apache#726) ### What changes were proposed in this pull request? Try remove `sun.security.krb5.Config.refresh();` ### Why are the changes needed? `sun.security.krb5` is not visible in Java 11 and 17. We need to compile on JDK11 and JDK17 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test verification. Co-authored-by: slfan1989 <louj1988@@>
… Java 11 and 17. (apache#726) ### What changes were proposed in this pull request? Try remove `sun.security.krb5.Config.refresh();` ### Why are the changes needed? `sun.security.krb5` is not visible in Java 11 and 17. We need to compile on JDK11 and JDK17 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test verification. Co-authored-by: slfan1989 <louj1988@@>
…refresh. (#932) ### What changes were proposed in this pull request? sun.security.krb5.Config should be refresh when krb5 conf file is setup. ### Why are the changes needed? We found error when use hadoop3.2 profile. By my debug, I found the static variable 'singleton' in sun.security.krb5.Config is not reload. We should refresh when krb5 config is setup. As described in #625, I refresh only in java8. Build error logs: ``` ------------------------------------------------------------------------------- Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.034 s <<< FAILURE! - in org.apache.uniffle.common.security.SecurityContextFactoryTest org.apache.uniffle.common.security.SecurityContextFactoryTest Time elapsed: 1.034 s <<< ERROR! java.lang.Exception: java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.uniffle.common.KerberizedHadoop.setup(KerberizedHadoop.java:111) at org.apache.uniffle.common.KerberizedHadoopBase.init(KerberizedHadoopBase.java:41) at org.apache.uniffle.common.security.SecurityContextFactoryTest.beforeAll(SecurityContextFactoryTest.java:38) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:126) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptBeforeAllMethod(TimeoutExtension.java:68) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllMethods$11(ClassBasedTestDescriptor.java:397) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllMethods(ClassBasedTestDescriptor.java:395) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:209) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:80) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:148) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.util.ArrayList.forEach(ArrayList.java:1259) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86) at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86) at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:71) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:319) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:365) at org.apache.uniffle.common.KerberizedHadoop.startKerberizedDFS(KerberizedHadoop.java:258) at org.apache.uniffle.common.KerberizedHadoop.setup(KerberizedHadoop.java:107) ... 61 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:110) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:69) ... 65 more Caused by: KrbException: Cannot locate default realm at sun.security.krb5.Config.getDefaultRealm(Config.java:1137) ... 71 more ``` ### How was this patch tested? CI test
Code of Conduct
Search before asking
What would you like to be improved?
Sun Kerberos binding, i.e. package
sun.security.krb5
is blocking us to build Uniffle in Java 11 and Java 17.How should we improve?
We can replace
sun.security.krb5
by Apache Kerby.Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: