[FLINK-29761][runtime][security] Simplify HadoopModule #21160
Conversation
| && !StringUtils.isBlank(securityConfig.getPrincipal())) { | ||
| String keytabPath = (new File(securityConfig.getKeytab())).getAbsolutePath(); | ||
|
|
||
| UserGroupInformation.loginUserFromKeytab(securityConfig.getPrincipal(), keytabPath); |
There was a problem hiding this comment.
@steveloughran since you have quite some XP w/ UGI a quick question:
AFAIK UserGroupInformation.loginUserFromKeytab does the same just like
ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI
UserGroupInformation.setLoginUser(ugi);
right?
There was a problem hiding this comment.
yes, plus sets up a renewer thread to do relogin on a regular basis.
| Method loginUserFromSubjectMethod = | ||
| UserGroupInformation.class.getMethod( | ||
| "loginUserFromSubject", Subject.class); | ||
| loginUserFromSubjectMethod.invoke(null, (Subject) null); |
There was a problem hiding this comment.
@steveloughran here is another similar question:
AFAIK UserGroupInformation.loginUserFromSubject does the same just like
ugi = UserGroupInformation.getUGIFromTicketCache
UserGroupInformation.setLoginUser(ugi);
right?
There was a problem hiding this comment.
ooh, its more complex than that. look at the source.
|
@steveloughran So if I understand correctly it would be better to use the original UGI calls instead of the other APIs because:
Correct me if I'm wrong. |
gaborgsomogyi
force-pushed
the
FLINK-29761
branch
from
6d3989d
to
0b3f379
Compare
|
Based on the fact that the UGI APIs are not doing the same things I've refactored the original PR and using the exact same API calls. Re-tested it manually and works fine. |
gaborgsomogyi
force-pushed
the
FLINK-29761
branch
from
0b3f379
to
4b49fea
Compare
|
@flinkbot run azure |
gaborgsomogyi
force-pushed
the
FLINK-29761
branch
from
4b49fea
to
1cfa567
Compare
|
@flinkbot run azure |
gaborgsomogyi
force-pushed
the
FLINK-29761
branch
from
1cfa567
to
f2b1644
Compare
flink-runtime/src/test/java/org/apache/flink/runtime/hadoop/HadoopUserUtilsITCasae.java
Outdated
Show resolved
Hide resolved
gaborgsomogyi
force-pushed
the
FLINK-29761
branch
2 times, most recently
from
9429c05
to
2b4cae1
Compare
gaborgsomogyi
force-pushed
the
FLINK-29761
branch
from
2b4cae1
to
83dd347
Compare
|
@flinkbot run azure |
| // UserGroupInformation.loginUserFromSubject(null); | ||
| Method loginUserFromSubjectMethod = | ||
| UserGroupInformation.class.getMethod( | ||
| "loginUserFromSubject", Subject.class); |
There was a problem hiding this comment.
Just to help reviewers this API has been added in Hadoop v2.3.0. Please see this.
There was a problem hiding this comment.
Thanks, @gaborgsomogyi. Looks good, added a minor javadoc comment which I will squash into your commit on merge.
Merging later today EU time unless anyone has objections.
What is the purpose of the change
HadoopModuleis quite complex and contains reflection which can be simplified. In this PR I've made this simplification keeping the original functionality.The other important intention is that
flink-runtimemodule has used several things fromflink-hadoop-fsmodule which is just bad from architectural perspective. In this PR this I've eliminated this.Brief change log
KerberosLoginProviderinHadoopModuleVerifying this change
Does this pull request potentially affect one of the following parts:
@Public(Evolving): noDocumentation