Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resolve anonymous roles and deduplicate roles during authentication #53453

Merged
merged 25 commits into from
Apr 30, 2020
Merged

Resolve anonymous roles and deduplicate roles during authentication #53453

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8dac757
Move anonymous role merging from authz to authc
ywangd Mar 20, 2020
09df4fb
Update rest tests
ywangd Mar 20, 2020
e24072a
Add unit tests for authenticationService
ywangd Mar 20, 2020
2cf8516
Remove unnecessary authorizationService test
ywangd Mar 20, 2020
8c5531b
Fix test
ywangd Mar 20, 2020
903b805
style
ywangd Mar 20, 2020
c171757
Merge remote-tracking branch 'origin/master' into es-47195-anonymous-…
ywangd Mar 24, 2020
c43f8ba
Address feedback
ywangd Mar 24, 2020
857c9e4
Checkstyle
ywangd Mar 24, 2020
1deaa58
Remove extra empty line
ywangd Mar 24, 2020
fe6eae8
Address feedback
ywangd Mar 24, 2020
2fcbfad
Address feedback
ywangd Mar 24, 2020
72d2667
Fix tests
ywangd Mar 24, 2020
65faa2c
Fix typo
ywangd Mar 24, 2020
c686985
Refactor for clarity
ywangd Mar 24, 2020
faa0471
Refactor for more predictable behaviour
ywangd Mar 24, 2020
384aa17
Address feedback
ywangd Mar 26, 2020
6fa37e4
Merge remote-tracking branch 'origin/master' into es-47195-anonymous-…
ywangd Mar 26, 2020
6fd8249
Merge remote-tracking branch 'origin/master' into es-47195-anonymous-…
ywangd Mar 26, 2020
cc7f259
Merge remote-tracking branch 'origin/master' into es-47195-anonymous-…
ywangd Apr 8, 2020
9a8bd63
Deduplicate roles for file and native realms earlier in the chain
ywangd Apr 8, 2020
6af416c
Remove unnecessary tests
ywangd Apr 8, 2020
2892218
Update tests
ywangd Apr 8, 2020
c6a8aa1
Merge remote-tracking branch 'origin/master' into es-47195-anonymous-…
ywangd Apr 22, 2020
c63c49f
Merge remote-tracking branch 'origin/master' into es-47195-anonymous-…
ywangd Apr 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...src/main/java/org/elasticsearch/xpack/core/security/action/user/AuthenticateResponse.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ public void writeTo(StreamOutput out) throws IOException {
authentication.writeTo(out);
}

}
}
2 changes: 1 addition & 1 deletion ...curity/src/test/java/org/elasticsearch/xpack/security/EnableSecurityOnBasicLicenseIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ private void checkAuthentication() throws IOException {
final Map<String, Object> auth = getAsMap("/_security/_authenticate");
// From file realm, configured in build.gradle
assertThat(ObjectPath.evaluate(auth, "username"), equalTo("security_test_user"));
assertThat(ObjectPath.evaluate(auth, "roles"), contains("security_test_role"));
assertThat(ObjectPath.evaluate(auth, "roles"), contains("security_test_role", "anonymous"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we guarantee order ? if not maybe containsInAnyOrder() is better here ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. The order is preserved. But I don't think we need guarantee it.

}

private void checkAllowedWrite(String indexName) throws IOException {
Expand Down
45 changes: 44 additions & 1 deletion .../security/src/main/java/org/elasticsearch/xpack/security/authc/AuthenticationService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,11 @@
import org.elasticsearch.xpack.core.security.authz.AuthorizationEngine.EmptyAuthorizationInfo;
import org.elasticsearch.xpack.core.security.support.Exceptions;
import org.elasticsearch.xpack.core.security.user.AnonymousUser;
import org.elasticsearch.xpack.core.security.user.AsyncSearchUser;
import org.elasticsearch.xpack.core.security.user.SystemUser;
import org.elasticsearch.xpack.core.security.user.User;
import org.elasticsearch.xpack.core.security.user.XPackSecurityUser;
import org.elasticsearch.xpack.core.security.user.XPackUser;
import org.elasticsearch.xpack.security.audit.AuditTrail;
import org.elasticsearch.xpack.security.audit.AuditTrailService;
import org.elasticsearch.xpack.security.audit.AuditUtil;
Expand Down Expand Up @@ -657,7 +660,8 @@ void finishAuthentication(User finalUser) {
logger.debug("user [{}] is disabled. failing authentication", finalUser);
listener.onFailure(request.authenticationFailed(authenticationToken));
} else {
final Authentication finalAuth = new Authentication(finalUser, authenticatedBy, lookedupBy);
final Authentication finalAuth = new Authentication(
maybeMergeAnonymousRolesForUser(finalUser), authenticatedBy, lookedupBy);
writeAuthToContext(finalAuth);
}
}
Expand Down Expand Up @@ -690,6 +694,45 @@ void writeAuthToContext(Authentication authentication) {
private void authenticateToken(AuthenticationToken token) {
this.consumeToken(token);
}

private User maybeMergeAnonymousRolesForUser(User user) {
if (SystemUser.is(user) || XPackUser.is(user) || XPackSecurityUser.is(user) || AsyncSearchUser.is(user)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also a bit worried about duplicating the check here. Should we make

elasticsearch/x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authz/AuthorizationService.java

Line 418 in 9467dbf

private boolean isInternalUser(User user) {
public, update it to take AsyncSearchUSer into consideration ( which was missed when we introduced AsyncSearchUser - point in case ) and use that as a single source of truth for the check here ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes makes sense. Long logical expression is never good as an if condition and should be wrapped inside a method.

But maybe we could find a better home for this method. How about User#isInternal(User user)?

The AsyncSearchUser is also missed in InternalUserSerializationHelper. The new Use#isInternal method can be leveraged in here as well. It won't be able to access AuthorizationService#isInternalUser due to circular dependency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did the refactoring. But decided to leave InternalUserSerializationHelper out for now. Changing it breaks backwards compatibility, which I think deseves its own issue.

return user;
} else if (isAnonymousUserEnabled && anonymousUser.equals(user) == false) {
if (anonymousUser.roles().length == 0) {
throw new IllegalStateException("anonymous is only enabled when the anonymous user has roles");
}
User userWithMergedRoles = new User(
user.principal(),
mergeAnonymousRoles(user.roles()),
user.fullName(),
user.email(),
user.metadata(),
user.enabled()
);
if (user.isRunAs()) {
final User authenticatedUserWithMergedRoles = new User(
user.authenticatedUser().principal(),
mergeAnonymousRoles(user.authenticatedUser().roles()),
user.authenticatedUser().fullName(),
user.authenticatedUser().email(),
user.authenticatedUser().metadata(),
user.authenticatedUser().enabled()
);
userWithMergedRoles = new User(userWithMergedRoles, authenticatedUserWithMergedRoles);
}
return userWithMergedRoles;
} else {
return user;
}
}

private String[] mergeAnonymousRoles(String[] existingRoles) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this implies that we have some special handling when merging the anonymous roles, while we just merge the two String arrays. I get that this is cleaner than doing it twice above but maybe a more generic mergeRoles where you pass both existingRoles and anonymoysUser.roles? Just a suggestion though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good suggestion. It is cleaner that way. I updated.

String[] mergedRoles = new String[existingRoles.length + anonymousUser.roles().length];
System.arraycopy(existingRoles, 0, mergedRoles, 0, existingRoles.length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we create a Set here and Collections.addAll so that we don't end up with duplicate roles in case the anonymous roles contains a role the user already has ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intentially didn't do deduplication here. Deduplication, filtering out un-resolvable roles, preempting the superuser role are all done in CompositeRolesStore#getRoles, which comes after authentication.

Even without the anonymous roles, it is still possible to create an user with duplicated roles, e.g.
{"roles":["x","x","x"],"password":...} and these roles will all show up in the authentication response. So I decided to retain the existing behaviour.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should entertain this existing behavior for no reason, and since we go in the trouble of merging the roles we should do de-duplication here . We could tackle this in a follow up where we for instance should not allow a user to be created with "roles" :["x","x","x"] in the first place as this leniency makes no sense and I would argue that if someone created a user with "roles" :["x","x"] , they probably meant "roles" :["x","y"] and mistyped , so it's better to tell them up front/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging only happens when anonymous access is enabled. If we de-duplicate here, it must also be applied when anonymous access is not enabled. So I re-arranged the code.

System.arraycopy(anonymousUser.roles(), 0, mergedRoles, existingRoles.length, anonymousUser.roles().length);
return mergedRoles;
}
}

abstract static class AuditableRequest {
Expand Down
12 changes: 0 additions & 12 deletions ...urity/src/main/java/org/elasticsearch/xpack/security/authz/store/CompositeRolesStore.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@
import org.elasticsearch.xpack.core.security.authz.store.RoleRetrievalResult;
import org.elasticsearch.xpack.core.security.support.CacheIteratorHelper;
import org.elasticsearch.xpack.core.security.support.MetadataUtils;
import org.elasticsearch.xpack.core.security.user.AnonymousUser;
import org.elasticsearch.xpack.core.security.user.AsyncSearchUser;
import org.elasticsearch.xpack.core.security.user.SystemUser;
import org.elasticsearch.xpack.core.security.user.User;
Expand Down Expand Up @@ -100,9 +99,7 @@ public class CompositeRolesStore {
private final DocumentSubsetBitsetCache dlsBitsetCache;
private final ThreadContext threadContext;
private final AtomicLong numInvalidation = new AtomicLong();
private final AnonymousUser anonymousUser;
private final ApiKeyService apiKeyService;
private final boolean isAnonymousEnabled;
private final List<BiConsumer<Set<String>, ActionListener<RoleRetrievalResult>>> builtInRoleProviders;
private final List<BiConsumer<Set<String>, ActionListener<RoleRetrievalResult>>> allRoleProviders;

Expand Down Expand Up @@ -145,8 +142,6 @@ public CompositeRolesStore(Settings settings, FileRolesStore fileRolesStore, Nat
allList.addAll(rolesProviders);
this.allRoleProviders = Collections.unmodifiableList(allList);
}
this.anonymousUser = new AnonymousUser(settings);
this.isAnonymousEnabled = AnonymousUser.isAnonymousEnabled(settings);
}

public void roles(Set<String> roleNames, ActionListener<Role> roleActionListener) {
Expand Down Expand Up @@ -236,13 +231,6 @@ public void getRoles(User user, Authentication authentication, ActionListener<Ro
}, roleActionListener::onFailure));
} else {
Set<String> roleNames = new HashSet<>(Arrays.asList(user.roles()));
if (isAnonymousEnabled && anonymousUser.equals(user) == false) {
if (anonymousUser.roles().length == 0) {
throw new IllegalStateException("anonymous is only enabled when the anonymous user has roles");
}
Collections.addAll(roleNames, anonymousUser.roles());
}

if (roleNames.isEmpty()) {
roleActionListener.onResponse(Role.EMPTY);
} else if (roleNames.contains(ReservedRolesStore.SUPERUSER_ROLE_DESCRIPTOR.getName())) {
Expand Down
45 changes: 45 additions & 0 deletions ...rity/src/test/java/org/elasticsearch/xpack/security/authc/AuthenticationServiceTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,11 @@
import org.elasticsearch.xpack.core.security.authz.AuthorizationEngine.EmptyAuthorizationInfo;
import org.elasticsearch.xpack.core.security.index.RestrictedIndicesNames;
import org.elasticsearch.xpack.core.security.user.AnonymousUser;
import org.elasticsearch.xpack.core.security.user.AsyncSearchUser;
import org.elasticsearch.xpack.core.security.user.SystemUser;
import org.elasticsearch.xpack.core.security.user.User;
import org.elasticsearch.xpack.core.security.user.XPackSecurityUser;
import org.elasticsearch.xpack.core.security.user.XPackUser;
import org.elasticsearch.xpack.security.audit.AuditTrail;
import org.elasticsearch.xpack.security.audit.AuditTrailService;
import org.elasticsearch.xpack.security.audit.AuditUtil;
Expand Down Expand Up @@ -901,6 +904,48 @@ public void testAnonymousUserTransportWithDefaultUser() throws Exception {
assertThreadContextContainsAuthentication(result);
}

public void testInheritAnonymousUserRoles() {
Settings settings = Settings.builder()
.putList(AnonymousUser.ROLES_SETTING.getKey(), "r3", "r4", "r5")
.build();
final AnonymousUser anonymousUser = new AnonymousUser(settings);
service = new AuthenticationService(settings, realms, auditTrailService,
new DefaultAuthenticationFailureHandler(Collections.emptyMap()),
threadPool, anonymousUser, tokenService, apiKeyService);
User user1 = new User("username", "r1", "r2");
when(firstRealm.token(threadContext)).thenReturn(token);
when(firstRealm.supports(token)).thenReturn(true);
mockAuthenticate(firstRealm, token, user1);
// this call does not actually go async
final AtomicBoolean completed = new AtomicBoolean(false);
service.authenticate(restRequest, true, ActionListener.wrap(authentication -> {
assertThat(authentication.getUser().roles(), equalTo(new String[]{"r1", "r2", "r3", "r4", "r5"}));
setCompletedToTrue(completed);
}, this::logAndFail));
assertTrue(completed.get());
}

public void testSystemUsersDoNotInheritAnonymousRoles() {
Settings settings = Settings.builder()
.putList(AnonymousUser.ROLES_SETTING.getKey(), "r3", "r4", "r5")
.build();
final AnonymousUser anonymousUser = new AnonymousUser(settings);
service = new AuthenticationService(settings, realms, auditTrailService,
new DefaultAuthenticationFailureHandler(Collections.emptyMap()),
threadPool, anonymousUser, tokenService, apiKeyService);
when(firstRealm.token(threadContext)).thenReturn(token);
when(firstRealm.supports(token)).thenReturn(true);
final User sysUser = randomFrom(SystemUser.INSTANCE, XPackUser.INSTANCE, XPackSecurityUser.INSTANCE, AsyncSearchUser.INSTANCE);
mockAuthenticate(firstRealm, token, sysUser);
// this call does not actually go async
final AtomicBoolean completed = new AtomicBoolean(false);
service.authenticate(restRequest, true, ActionListener.wrap(authentication -> {
assertThat(authentication.getUser().roles(), equalTo(sysUser.roles()));
setCompletedToTrue(completed);
}, this::logAndFail));
assertTrue(completed.get());
}

public void testRealmTokenThrowingException() throws Exception {
final String reqId = AuditUtil.getOrGenerateRequestId(threadContext);
when(firstRealm.token(threadContext)).thenThrow(authenticationError("realm doesn't like tokens"));
Expand Down
39 changes: 0 additions & 39 deletions .../src/test/java/org/elasticsearch/xpack/security/authz/store/CompositeRolesStoreTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,6 @@
import org.elasticsearch.xpack.core.security.authz.store.RoleRetrievalResult;
import org.elasticsearch.xpack.core.security.index.RestrictedIndicesNames;
import org.elasticsearch.xpack.core.security.support.MetadataUtils;
import org.elasticsearch.xpack.core.security.user.AnonymousUser;
import org.elasticsearch.xpack.core.security.user.AsyncSearchUser;
import org.elasticsearch.xpack.core.security.user.SystemUser;
import org.elasticsearch.xpack.core.security.user.User;
Expand Down Expand Up @@ -90,7 +89,6 @@
import static org.hamcrest.Matchers.anyOf;
import static org.hamcrest.Matchers.containsInAnyOrder;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasItem;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.nullValue;
Expand Down Expand Up @@ -901,43 +899,6 @@ nativeRolesStore, reservedRolesStore, mock(NativePrivilegeStore.class), null, mo
assertEquals(Role.EMPTY, roles);
}

public void testAnonymousUserEnabledRoleAdded() {
Settings settings = Settings.builder()
.put(SECURITY_ENABLED_SETTINGS)
.put(AnonymousUser.ROLES_SETTING.getKey(), "anonymous_user_role")
.build();
final FileRolesStore fileRolesStore = mock(FileRolesStore.class);
doCallRealMethod().when(fileRolesStore).accept(any(Set.class), any(ActionListener.class));
final NativeRolesStore nativeRolesStore = mock(NativeRolesStore.class);
doCallRealMethod().when(nativeRolesStore).accept(any(Set.class), any(ActionListener.class));
doAnswer(invocationOnMock -> {
Set<String> names = (Set<String>) invocationOnMock.getArguments()[0];
if (names.size() == 1 && names.contains("anonymous_user_role")) {
RoleDescriptor rd = new RoleDescriptor("anonymous_user_role", null, null, null);
return Collections.singleton(rd);
}
return Collections.emptySet();
}).
when(fileRolesStore).roleDescriptors(anySetOf(String.class));
doAnswer((invocationOnMock) -> {
ActionListener<RoleRetrievalResult> callback = (ActionListener<RoleRetrievalResult>) invocationOnMock.getArguments()[1];
callback.onResponse(RoleRetrievalResult.failure(new RuntimeException("intentionally failed!")));
return null;
}).when(nativeRolesStore).getRoleDescriptors(isA(Set.class), any(ActionListener.class));
final ReservedRolesStore reservedRolesStore = spy(new ReservedRolesStore());

final CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(settings, fileRolesStore, nativeRolesStore,
reservedRolesStore, mock(NativePrivilegeStore.class), null, mock(ApiKeyService.class), null, null);
verify(fileRolesStore).addListener(any(Consumer.class)); // adds a listener in ctor

PlainActionFuture<Role> rolesFuture = new PlainActionFuture<>();
final User user = new User("no role user");
Authentication auth = new Authentication(user, new RealmRef("name", "type", "node"), null);
compositeRolesStore.getRoles(user, auth, rolesFuture);
final Role roles = rolesFuture.actionGet();
assertThat(Arrays.asList(roles.names()), hasItem("anonymous_user_role"));
}

public void testDoesNotUseRolesStoreForXPacAndAsyncSearchUser() {
final FileRolesStore fileRolesStore = mock(FileRolesStore.class);
doCallRealMethod().when(fileRolesStore).accept(any(Set.class), any(ActionListener.class));
Expand Down
10 changes: 8 additions & 2 deletions ...c/test/java/org/elasticsearch/xpack/security/rest/action/RestAuthenticateActionTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,14 @@ public void testAuthenticateApi() throws Exception {
assertThat(objectPath.evaluate("lookup_realm.name").toString(), equalTo("file"));
assertThat(objectPath.evaluate("lookup_realm.type").toString(), equalTo("file"));
List<String> roles = objectPath.evaluate("roles");
assertThat(roles.size(), is(1));
assertThat(roles, contains(SecuritySettingsSource.TEST_ROLE));

if (anonymousEnabled) {
assertThat(roles.size(), is(3));
assertThat(roles, contains(SecuritySettingsSource.TEST_ROLE, SecuritySettingsSource.TEST_ROLE, "foo"));
} else {
assertThat(roles.size(), is(1));
assertThat(roles, contains(SecuritySettingsSource.TEST_ROLE));
}
}

public void testAuthenticateApiWithoutAuthentication() throws Exception {
Expand Down