Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2869] feat(core): Add the core logic of authorization #3050

Open
wants to merge 79 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
85c56aa
Supports multiple securable objects
May 21, 2024
2dab53a
Remove limit
May 22, 2024
f97d872
Remove blank
May 22, 2024
4ebe2ce
Update SQL script
May 22, 2024
f8d0649
Modify document and add upgrade script
May 22, 2024
65108ca
Add new uts
May 23, 2024
d71132f
fix style
May 23, 2024
68a2cd4
Change type
May 24, 2024
1487d89
Add fields
May 24, 2024
927ec29
Use another table to store securable objects
May 24, 2024
80a3335
Merge remote-tracking branch 'upstream/main' into ISSUE-3343
May 24, 2024
dd237ba
Commit information
May 24, 2024
89c3e42
Commit information
May 24, 2024
7a55a0e
Change to MetadataObject.Type
May 24, 2024
f079305
fix style
May 24, 2024
1891cec
Change SecurableObject type
May 24, 2024
a2f8d5f
Update SQL
May 24, 2024
8a95066
Revert "Add fields"
May 24, 2024
0e3fb3d
format
May 24, 2024
e0a33b1
format
May 24, 2024
75d92c3
format
May 24, 2024
d351ae7
Data migiration
May 27, 2024
6ff647e
Add auto increment
May 27, 2024
1e0da55
Address comments
May 27, 2024
e145558
Use entity id
May 28, 2024
c290088
Add mapper
May 29, 2024
8484d04
Use entityId
May 30, 2024
d73248f
Fix tests
May 31, 2024
aa432bd
fix comment
May 31, 2024
435c334
rename
May 31, 2024
5463b4a
Remove dirty logic from POConverts
May 31, 2024
eb94073
Add more uts
Jun 3, 2024
2dba190
Merge remote-tracking branch 'upstream/main' into ISSUE-3343
Jun 3, 2024
32c5eca
fix compile
Jun 3, 2024
0be53e7
fix compile
Jun 3, 2024
4f84ac2
Address comments
Jun 4, 2024
46d620e
fix tests
Jun 4, 2024
9a3ed93
[#2869] feat(server): Add the authorization for admins
Apr 21, 2024
12c59ef
Remove supportsSecurableObjectType
May 23, 2024
2d85afc
Remove more filters
May 23, 2024
736e5bc
Remove unused methods
May 23, 2024
eb627e2
Left imports
May 24, 2024
51d7efd
address comments
Jun 6, 2024
e9317aa
Address comments
Jun 6, 2024
c191142
Add comment
Jun 6, 2024
a021351
Add comment
Jun 6, 2024
edc15ef
address comments
Jun 6, 2024
48a1ac8
Add ut
May 27, 2024
8510093
Merge remote-tracking branch 'upstream/main' into ISSUE-2869-4
Jun 6, 2024
d1fc027
add ut
Jun 7, 2024
140ad19
Address comments
Jun 7, 2024
168919d
Address comments
Jun 7, 2024
ec41730
fix style
Jun 7, 2024
033ec85
fix ut
Jun 7, 2024
3719280
remove extra blank
Jun 7, 2024
85fbb8a
Merge branch 'ISSUE-3343' into ISSUE-2869-4
Jun 7, 2024
44abe05
ut
Jun 7, 2024
0ead8f9
Add ut
jerqi Jun 7, 2024
13a5f57
Merge remote-tracking branch 'upstream/main' into ISSUE-2869-4
jerqi Jun 7, 2024
c6fd30a
fix ut
jerqi Jun 11, 2024
4396e35
fix ut
jerqi Jun 11, 2024
0bd0a6c
fix ut
jerqi Jun 12, 2024
1c4c896
fix ut
jerqi Jun 12, 2024
3426521
Address comments
jerqi Jun 19, 2024
a522cde
Merge remote-tracking branch 'upstream/main' into ISSUE-2869-4
jerqi Jun 19, 2024
280d716
fix style
jerqi Jun 19, 2024
bbfbb11
fix ut
jerqi Jun 19, 2024
6463b3f
address comments
jerqi Jun 19, 2024
bd240ce
Merge remote-tracking branch 'upstream/main' into ISSUE-2869-4
jerqi Jun 21, 2024
a3ad730
fix conflicts
jerqi Jun 21, 2024
319a91f
address comments
jerqi Jun 26, 2024
f39f262
Add ut
jerqi Jun 26, 2024
c23924a
Remove todo
jerqi Jun 26, 2024
3dc29ed
fix style
jerqi Jun 26, 2024
940ea14
fix ut
jerqi Jun 26, 2024
31a3a2b
address comments
jerqi Jun 27, 2024
a714cdd
refactor another class
jerqi Jun 27, 2024
b7c30fa
fix ut
jerqi Jun 27, 2024
bd54c41
Remove admin filter
jerqi Jun 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
@Evolving
public interface Role extends Auditable {

/** The reserved role names' prefix */
String SYSTEM_RESERVED_ROLE_NAME_PREFIX = "system_role";
/**
* The name of the role.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,8 @@ public static SecurableObject ofFileset(
/**
* All metalakes is a special securable object .You can give the securable object the privileges
* `CREATE METALAKE`, etc. It means that you can create any which doesn't exist. This securable
* object is only used for metalake admin. You can't grant any privilege to this securable object.
* You can't bind this securable object to any role, too.
* object is reserved for system. You can't grant any privilege to this securable object. You
* can't bind this securable object to any role, too.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There you mentioned that "You can give the securable object the privileges CREATE METALAKE", but here you mentioned that "You can't grant any privilege to this securable object". I think these two sentences are self-contradictory, you should have a more accurate description.

Besides, why you can't bind this securable object to a role?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, for users, users can't grant any privilege to allMetalakes. But system pre-created a role binds allMetalakes. AllMetalakes isn't used for the normal user.

We shouldn't allow user to use allMetalakes. It can inflluence the privileges of all metalakes.It's too dangerous.

*
* @param privileges The privileges of the all metalakes
* @return The created {@link SecurableObject}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright 2024 Datastrato Pvt Ltd.
* This software is licensed under the Apache License version 2.
*/
package com.datastrato.gravitino.exceptions;

import com.google.errorprone.annotations.FormatMethod;
import com.google.errorprone.annotations.FormatString;

/** Exception thrown when a user doesn't have the privilege to perform an action. */
public class ForbiddenException extends GravitinoRuntimeException {

/**
* Constructs a new exception with the specified detail message.
*
* @param message the detail message.
* @param args the arguments to the message.
*/
@FormatMethod
public ForbiddenException(@FormatString String message, Object... args) {
super(message, args);
}

/**
* Constructs a new exception with the specified detail message and cause.
*
* @param cause the cause.
* @param message the detail message.
* @param args the arguments to the message.
*/
@FormatMethod
public ForbiddenException(Throwable cause, @FormatString String message, Object... args) {
super(cause, message, args);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright 2024 Datastrato Pvt Ltd.
* This software is licensed under the Apache License version 2.
*/
package com.datastrato.gravitino.exceptions;

import com.google.errorprone.annotations.FormatMethod;
import com.google.errorprone.annotations.FormatString;

/** An exception thrown when privileges already exist. */
public class PrivilegesAlreadyGrantedException extends AlreadyExistsException {
/**
* Constructs a new exception with the specified detail message.
*
* @param message the detail message.
* @param args the arguments to the message.
*/
@FormatMethod
public PrivilegesAlreadyGrantedException(@FormatString String message, Object... args) {
super(message, args);
}

/**
* Constructs a new exception with the specified detail message and cause.
*
* @param cause the cause.
* @param message the detail message.
* @param args the arguments to the message.
*/
@FormatMethod
public PrivilegesAlreadyGrantedException(
Throwable cause, @FormatString String message, Object... args) {
super(cause, message, args);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import com.datastrato.gravitino.exceptions.NonEmptySchemaException;
import com.datastrato.gravitino.exceptions.NotFoundException;
import com.datastrato.gravitino.exceptions.PartitionAlreadyExistsException;
import com.datastrato.gravitino.exceptions.PrivilegesAlreadyGrantedException;
import com.datastrato.gravitino.exceptions.RESTException;
import com.datastrato.gravitino.exceptions.RoleAlreadyExistsException;
import com.datastrato.gravitino.exceptions.SchemaAlreadyExistsException;
Expand Down Expand Up @@ -143,6 +144,15 @@ public static Consumer<ErrorResponse> groupErrorHandler() {
return GroupErrorHandler.INSTANCE;
}

/**
* Creates an error handler specific to MetalakeAdmin operations.
*
* @return A Consumer representing the MetalakeAdmin error handler.
*/
public static Consumer<ErrorResponse> metalakeAdminErrorHandler() {
return MetalakeAdminErrorHandler.INSTANCE;
}

/**
* Creates an error handler specific to Role operations.
*
Expand Down Expand Up @@ -567,6 +577,41 @@ public void accept(ErrorResponse errorResponse) {
}
}

/** Error handler specific to MetalakeAdmin operations. */
@SuppressWarnings("FormatStringAnnotation")
private static class MetalakeAdminErrorHandler extends RestErrorHandler {

private static final MetalakeAdminErrorHandler INSTANCE = new MetalakeAdminErrorHandler();

@Override
public void accept(ErrorResponse errorResponse) {
String errorMessage = formatErrorMessage(errorResponse);

switch (errorResponse.getCode()) {
case ErrorConstants.ILLEGAL_ARGUMENTS_CODE:
throw new IllegalArgumentException(errorMessage);

case ErrorConstants.NOT_FOUND_CODE:
if (errorResponse.getType().equals(NoSuchMetalakeException.class.getSimpleName())) {
throw new NoSuchMetalakeException(errorMessage);
} else if (errorResponse.getType().equals(NoSuchUserException.class.getSimpleName())) {
throw new NoSuchUserException(errorMessage);
} else {
throw new NotFoundException(errorMessage);
}

case ErrorConstants.ALREADY_EXISTS_CODE:
throw new PrivilegesAlreadyGrantedException(errorMessage);

case ErrorConstants.INTERNAL_ERROR_CODE:
throw new RuntimeException(errorMessage);

default:
super.accept(errorResponse);
}
}
}

/** Error handler specific to Role operations. */
@SuppressWarnings("FormatStringAnnotation")
private static class RoleErrorHandler extends RestErrorHandler {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import com.datastrato.gravitino.exceptions.NoSuchMetalakeException;
import com.datastrato.gravitino.exceptions.NoSuchRoleException;
import com.datastrato.gravitino.exceptions.NoSuchUserException;
import com.datastrato.gravitino.exceptions.PrivilegesAlreadyGrantedException;
import com.datastrato.gravitino.exceptions.RoleAlreadyExistsException;
import com.datastrato.gravitino.exceptions.UserAlreadyExistsException;
import com.google.common.base.Preconditions;
Expand Down Expand Up @@ -325,10 +326,11 @@ public Group getGroup(String metalake, String group)
*
* @param user The name of the User.
* @return The added User instance.
* @throws UserAlreadyExistsException If a metalake admin with the same name already exists.
* @throws PrivilegesAlreadyGrantedException If the metalake admin with the same name already
* added.
* @throws RuntimeException If adding the User encounters storage issues.
*/
public User addMetalakeAdmin(String user) throws UserAlreadyExistsException {
public User addMetalakeAdmin(String user) throws PrivilegesAlreadyGrantedException {
UserAddRequest req = new UserAddRequest(user);
req.validate();

Expand All @@ -338,7 +340,7 @@ public User addMetalakeAdmin(String user) throws UserAlreadyExistsException {
req,
UserResponse.class,
Collections.emptyMap(),
ErrorHandlers.userErrorHandler());
ErrorHandlers.metalakeAdminErrorHandler());
resp.validate();

return resp.getUser();
Expand All @@ -358,7 +360,7 @@ public boolean removeMetalakeAdmin(String user) {
String.format(API_ADMIN_PATH, user),
RemoveResponse.class,
Collections.emptyMap(),
ErrorHandlers.userErrorHandler());
ErrorHandlers.metalakeAdminErrorHandler());
resp.validate();

return resp.removed();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import com.datastrato.gravitino.dto.responses.ErrorResponse;
import com.datastrato.gravitino.dto.responses.RemoveResponse;
import com.datastrato.gravitino.dto.responses.UserResponse;
import com.datastrato.gravitino.exceptions.UserAlreadyExistsException;
import com.datastrato.gravitino.exceptions.PrivilegesAlreadyGrantedException;
import java.time.Instant;
import org.apache.hc.core5.http.Method;
import org.junit.jupiter.api.Assertions;
Expand Down Expand Up @@ -45,14 +45,14 @@ public void testAddMetalakeAdmin() throws Exception {
Assertions.assertNotNull(addedUser);
assertUser(addedUser, mockUser);

// test UserAlreadyExistsException
// test PrivilegeGrantAlreadyExistsException
ErrorResponse errResp1 =
ErrorResponse.alreadyExists(
UserAlreadyExistsException.class.getSimpleName(), "user already exists");
PrivilegesAlreadyGrantedException.class.getSimpleName(), "user already exists");
buildMockResource(Method.POST, userPath, request, errResp1, SC_CONFLICT);
Exception ex =
Assertions.assertThrows(
UserAlreadyExistsException.class, () -> client.addMetalakeAdmin(username));
PrivilegesAlreadyGrantedException.class, () -> client.addMetalakeAdmin(username));
Assertions.assertEquals("user already exists", ex.getMessage());

// test RuntimeException
Expand Down
8 changes: 1 addition & 7 deletions core/src/main/java/com/datastrato/gravitino/Entity.java
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ public interface Entity extends Serializable {
/** The system reserved catalog name. */
String SYSTEM_CATALOG_RESERVED_NAME = "system";

/** The authorization catalog name in the system metalake. */
String AUTHORIZATION_CATALOG_NAME = "authorization";

/** The user schema name in the system catalog. */
String USER_SCHEMA_NAME = "user";

Expand All @@ -35,15 +32,12 @@ public interface Entity extends Serializable {
/** The role schema name in the system catalog. */
String ROLE_SCHEMA_NAME = "role";

/** The admin schema name in the authorization catalog of the system metalake. */
String ADMIN_SCHEMA_NAME = "admin";

/** The tag schema name in the system catalog. */
String TAG_SCHEMA_NAME = "tag";

/**
* All metalakes are a virtual entity. It represents all the metalakes. We don't store it. We use
* a specific type to represent its entity type.
* a specific type to represent its entity id.
*/
String ALL_METALAKES_ENTITY_TYPE = "ROOT";

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,16 @@
import com.datastrato.gravitino.exceptions.NoSuchMetalakeException;
import com.datastrato.gravitino.exceptions.NoSuchRoleException;
import com.datastrato.gravitino.exceptions.NoSuchUserException;
import com.datastrato.gravitino.exceptions.PrivilegesAlreadyGrantedException;
import com.datastrato.gravitino.exceptions.RoleAlreadyExistsException;
import com.datastrato.gravitino.exceptions.UserAlreadyExistsException;
import com.datastrato.gravitino.meta.RoleEntity;
import com.datastrato.gravitino.storage.IdGenerator;
import com.datastrato.gravitino.utils.Executable;
import com.google.common.annotations.VisibleForTesting;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

/**
* AccessControlManager is used for manage users, roles, admin, grant information, this class is an
Expand All @@ -37,8 +40,8 @@ public class AccessControlManager {
private final Object nonAdminOperationLock = new Object();

public AccessControlManager(EntityStore store, IdGenerator idGenerator, Config config) {
this.adminManager = new AdminManager(store, idGenerator, config);
this.roleManager = new RoleManager(store, idGenerator, config);
this.adminManager = new AdminManager(store, idGenerator, config);
this.userGroupManager = new UserGroupManager(store, idGenerator, roleManager);
this.permissionManager = new PermissionManager(store, roleManager);
}
Expand Down Expand Up @@ -204,15 +207,16 @@ public User revokeRolesFromUser(String metalake, List<String> roles, String user
*
* @param user The name of the User.
* @return The added User instance.
* @throws UserAlreadyExistsException If a metalake admin with the same name already exists.
* @throws PrivilegesAlreadyGrantedException If the metalake admin with the same name already
* added.
* @throws RuntimeException If adding the User encounters storage issues.
*/
public User addMetalakeAdmin(String user) throws UserAlreadyExistsException {
public User addMetalakeAdmin(String user) throws PrivilegesAlreadyGrantedException {
return doWithAdminLock(() -> adminManager.addMetalakeAdmin(user));
}

/**
* Removes a metalake admin.
* Removes a metalake admin. Only service admins can manage metalake admins.
*
* @param user The name of the User.
* @return True if the User was successfully removed, false only when there's no such metalake
Expand All @@ -233,16 +237,6 @@ public boolean isServiceAdmin(String user) {
return adminManager.isServiceAdmin(user);
}

/**
* Judges whether the user is the metalake admin.
*
* @param user the name of the user
* @return True if the user is metalake admin, otherwise false.
*/
public boolean isMetalakeAdmin(String user) {
return doWithAdminLock(() -> adminManager.isMetalakeAdmin(user));
}

/**
* Creates a new Role.
*
Expand Down Expand Up @@ -294,6 +288,17 @@ public boolean deleteRole(String metalake, String role) throws NoSuchMetalakeExc
return doWithNonAdminLock(() -> roleManager.deleteRole(metalake, role));
}

public List<RoleEntity> listRolesByUser(String metalake, String currentUser) {
return doWithNonAdminLock(
() -> {
User user = getUser(metalake, currentUser);
// TODO: get roles from the group
return user.roles().stream()
.map(role -> roleManager.getRole(metalake, role))
.collect(Collectors.toList());
});
}

@VisibleForTesting
RoleManager getRoleManager() {
return roleManager;
Expand Down
Loading
Loading