Skip to content

[Bug] [Datasource-Hive] ConcurrentModificationException in UserGroupInformationFactory #18330

Description

@eye-gu

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

private static final Map<String, UserGroupInformation> userGroupInformationMap = new HashMap<>();
private static final ScheduledExecutorService kerberosRenewalService =
ThreadUtils.newSingleDaemonScheduledExecutorService("Hive-Kerberos-Renewal-Thread-");
static {
kerberosRenewalService.scheduleWithFixedDelay(() -> {
if (userGroupInformationMap.isEmpty()) {
return;
}
userGroupInformationMap.forEach((key, ugi) -> {
try {
if (ugi.isFromKeytab()) {
ugi.checkTGTAndReloginFromKeytab();
}
log.info("Relogin from keytab success, user: {}", key);
} catch (Exception e) {
log.error("Relogin from keytab failed, user: {}", key, e);
}
});
}, 0, 5, TimeUnit.MINUTES);
}
public synchronized static UserGroupInformation login(String userName) {
UserGroupInformation userGroupInformation = userGroupInformationMap.get(userName);
if (userGroupInformation == null) {
if (!openKerberos()) {
userGroupInformation = createRemoteUser(userName);
} else {
userGroupInformation = createKerberosUser();
}
userGroupInformationMap.put(userName, userGroupInformation);
}
currentLoginTimesMap.compute(userName, (k, v) -> v == null ? 1 : v + 1);
return userGroupInformation;
}
public synchronized static void logout(String userName) {
Integer currentLoginTimes = currentLoginTimesMap.get(userName);
if (currentLoginTimes == null) {
return;
}
if (currentLoginTimes <= 1) {
currentLoginTimesMap.remove(userName);
userGroupInformationMap.remove(userName);
} else {
currentLoginTimesMap.put(userName, currentLoginTimes - 1);
}
}

In UserGroupInformationFactory, login() and logout() are synchronized, but the static initializer schedules a background Kerberos renewal task that reads the same HashMap without synchronization. This will cause ConcurrentModificationException

What you expected to happen

Kerberos renewal thread should safely iterate the map

How to reproduce

Occasional

Anything else

No response

Version

dev

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Labels

backendbugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions