Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

9293 - Apply filter-based auth for all API endpoints (2/2) #9360

Merged
merged 33 commits into from
Feb 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
acabbb4
Refactor: using new filter-based auth on access endpoints
GPortas Jan 31, 2023
d98e8fc
Refactor: using new filter-based auth on admin endpoints
GPortas Jan 31, 2023
08cefe3
Refactor: using new filter-based auth on missing dataset endpoint
GPortas Jan 31, 2023
306116a
Refactor: using new filter-based auth on dataverses endpoints
GPortas Jan 31, 2023
b3921a4
Refactor: using new filter-based auth on files endpoints
GPortas Jan 31, 2023
e31b894
Fixed: access endpoint auth arguments
GPortas Jan 31, 2023
1e934b4
Refactor: using new filter-based auth on harvesting clients endpoints
GPortas Jan 31, 2023
92086b3
Refactor: using new filter-based auth on notifications endpoints
GPortas Jan 31, 2023
ffab810
Refactor: using new filter-based auth on pids endpoints
GPortas Jan 31, 2023
3eebaf1
Refactor: using new filter-based auth on prov endpoints
GPortas Jan 31, 2023
d15ae3b
Refactor: using new filter-based auth on roles endpoints
GPortas Jan 31, 2023
5bf1a27
Refactor: using new filter-based auth on users endpoints
GPortas Jan 31, 2023
7ccb78b
Refactor: using new filter-based auth on dataverses and admin endpoints
GPortas Jan 31, 2023
e9b04b3
Refactor: using new filter-based auth on endpoints and unused depreca…
GPortas Jan 31, 2023
36dbd9b
Refactor: changed to filter-based auth from direct findAuthenticatedU…
GPortas Jan 31, 2023
b13e054
Refactor: removed findAuthenticatedUserOrDie() from AbstractApiBean a…
GPortas Feb 1, 2023
e8163bd
Refactor: Access API endpoint class to avoid any endpoint or related …
GPortas Feb 2, 2023
1304ed6
Refactor: replaced rest of findUserOrDie calls and removed the method…
GPortas Feb 2, 2023
6a64c0a
Refactor: unused method, imports and declared EJBs removed
GPortas Feb 2, 2023
5909950
Fixed: DataversesTest
GPortas Feb 2, 2023
45817f3
Merge branch '9293-filter-api-auth' into 9293-filter-api-auth-final-r…
GPortas Feb 2, 2023
94082eb
Added: minor refactoring and IT test fixes
GPortas Feb 3, 2023
56fb798
Fixed: SearchIT.testSubtreePermissions wrong response code when no AP…
GPortas Feb 3, 2023
5d7b0ec
Fixed: DeactivateUsersIT.testDeactivateUser
GPortas Feb 3, 2023
b4459fd
Fixed: BuiltinUsersIT.testFindByToken for bad API key
GPortas Feb 3, 2023
2f64428
Fixed: DatasetsIT.testPrivateUrl wrong status code when bad API token…
GPortas Feb 3, 2023
b428cb6
Fixed: AdminIT tests anon calls status codes
GPortas Feb 3, 2023
735712a
Fixed: AdminIT tests
GPortas Feb 3, 2023
38ba4b0
Fixed: reverted wrong IT test changes
GPortas Feb 3, 2023
9ce4795
Changed: DataRetrieverAPI.retrieveMyDataAsJsonString now uses auth-fi…
GPortas Feb 9, 2023
83dddce
Merge branch '9293-filter-api-auth' into 9293-filter-api-auth-final-r…
GPortas Feb 15, 2023
50934ce
Merge branch 'develop' into 9293-filter-api-auth-final-refactor
GPortas Feb 22, 2023
5f52321
Added: SignedUrlAuthMechanism IT
GPortas Feb 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
174 changes: 3 additions & 171 deletions src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@
import edu.harvard.iq.dataverse.authorization.RoleAssignee;
import edu.harvard.iq.dataverse.authorization.groups.GroupServiceBean;
import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
import edu.harvard.iq.dataverse.authorization.users.GuestUser;
import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser;
import edu.harvard.iq.dataverse.authorization.users.User;
import edu.harvard.iq.dataverse.confirmemail.ConfirmEmailServiceBean;
import edu.harvard.iq.dataverse.datacapturemodule.DataCaptureModuleServiceBean;
Expand All @@ -43,22 +41,18 @@
import edu.harvard.iq.dataverse.externaltools.ExternalToolServiceBean;
import edu.harvard.iq.dataverse.license.LicenseServiceBean;
import edu.harvard.iq.dataverse.metrics.MetricsServiceBean;
import edu.harvard.iq.dataverse.privateurl.PrivateUrlServiceBean;
import edu.harvard.iq.dataverse.locality.StorageSiteServiceBean;
import edu.harvard.iq.dataverse.search.savedsearch.SavedSearchServiceBean;
import edu.harvard.iq.dataverse.settings.JvmSettings;
import edu.harvard.iq.dataverse.settings.SettingsServiceBean;
import edu.harvard.iq.dataverse.util.BundleUtil;
import edu.harvard.iq.dataverse.util.SystemConfig;
import edu.harvard.iq.dataverse.util.UrlSignerUtil;
import edu.harvard.iq.dataverse.util.json.JsonParser;
import edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder;
import edu.harvard.iq.dataverse.validation.PasswordValidatorServiceBean;
import java.io.StringReader;
import java.net.URI;
import java.util.Arrays;
import java.util.Collections;
import java.util.Enumeration;
import java.util.UUID;
import java.util.concurrent.Callable;
import java.util.logging.Level;
Expand All @@ -77,7 +71,6 @@
import javax.persistence.NoResultException;
import javax.persistence.PersistenceContext;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.core.*;
import javax.ws.rs.core.Response.ResponseBuilder;
Expand All @@ -97,6 +90,7 @@ public abstract class AbstractApiBean {
private static final String ALIAS_KEY=":alias";
public static final String STATUS_WF_IN_PROGRESS = "WORKFLOW_IN_PROGRESS";
public static final String DATAVERSE_WORKFLOW_INVOCATION_HEADER_NAME = "X-Dataverse-invocationID";
public static final String RESPONSE_MESSAGE_AUTHENTICATED_USER_REQUIRED = "Only authenticated users can perform the requested operation";

/**
* Utility class to convey a proper error response using Java's exceptions.
Expand Down Expand Up @@ -209,9 +203,6 @@ String getWrappedMessageWhenJson() {
@EJB
protected SavedSearchServiceBean savedSearchSvc;

@EJB
protected PrivateUrlServiceBean privateUrlSvc;

@EJB
protected ConfirmEmailServiceBean confirmEmailSvc;

Expand Down Expand Up @@ -278,7 +269,7 @@ public JsonParser call() throws Exception {
/**
* Functional interface for handling HTTP requests in the APIs.
*
* @see #response(edu.harvard.iq.dataverse.api.AbstractApiBean.DataverseRequestHandler)
* @see #response(edu.harvard.iq.dataverse.api.AbstractApiBean.DataverseRequestHandler, edu.harvard.iq.dataverse.authorization.users.User)
*/
protected static interface DataverseRequestHandler {
Response handle( DataverseRequest u ) throws WrappedResponse;
Expand Down Expand Up @@ -320,13 +311,6 @@ protected String getRequestApiKey() {

return headerParamApiKey!=null ? headerParamApiKey : queryParamApiKey;
}

protected String getRequestWorkflowInvocationID() {
String headerParamWFKey = httpRequest.getHeader(DATAVERSE_WORKFLOW_INVOCATION_HEADER_NAME);
String queryParamWFKey = httpRequest.getParameter("invocationID");

return headerParamWFKey!=null ? headerParamWFKey : queryParamWFKey;
}

protected User getRequestUser(ContainerRequestContext crc) {
return (User) crc.getProperty(ApiConstants.CONTAINER_REQUEST_CONTEXT_USER);
Expand Down Expand Up @@ -371,125 +355,13 @@ protected RoleAssignee findAssignee(String identifier) {
}

/**
*
* @param apiKey the key to find the user with
* @return the user, or null
* @see #findUserOrDie(java.lang.String)
*/
protected AuthenticatedUser findUserByApiToken( String apiKey ) {
return authSvc.lookupUser(apiKey);
}

/**
* Returns the user of pointed by the API key, or the guest user
* @return a user, may be a guest user.
* @throws edu.harvard.iq.dataverse.api.AbstractApiBean.WrappedResponse iff there is an api key present, but it is invalid.
*
* @deprecated Do not use this method.
* This method is expected to be removed once all API endpoints use the filter-based authentication.
* @see <a href="https://github.com/IQSS/dataverse/issues/9293">#9293</a>
* In case you are implementing a new endpoint that requires user authentication, here is an example of how to apply the new filter-based authentication:
* {@link edu.harvard.iq.dataverse.api.Datasets#getDataset(ContainerRequestContext, String, UriInfo, HttpHeaders, HttpServletResponse)}
*/
@Deprecated
protected User findUserOrDie() throws WrappedResponse {
final String requestApiKey = getRequestApiKey();
final String requestWFKey = getRequestWorkflowInvocationID();
if (requestApiKey == null && requestWFKey == null && getRequestParameter(UrlSignerUtil.SIGNED_URL_TOKEN)==null) {
return GuestUser.get();
}
PrivateUrlUser privateUrlUser = privateUrlSvc.getPrivateUrlUserFromToken(requestApiKey);
// For privateUrlUsers restricted to anonymized access, all api calls are off-limits except for those used in the UI
// to download the file or image thumbs
if (privateUrlUser != null) {
if (privateUrlUser.hasAnonymizedAccess()) {
String pathInfo = httpRequest.getPathInfo();
String prefix= "/access/datafile/";
if (!(pathInfo.startsWith(prefix) && !pathInfo.substring(prefix.length()).contains("/"))) {
logger.info("Anonymized access request for " + pathInfo);
throw new WrappedResponse(error(Status.UNAUTHORIZED, "API Access not allowed with this Key"));
}
}
return privateUrlUser;
}
return findAuthenticatedUserOrDie(requestApiKey, requestWFKey);
}

/**
* Finds the authenticated user, based on (in order):
* <ol>
* <li>The key in the HTTP header {@link #DATAVERSE_KEY_HEADER_NAME}</li>
* <li>The key in the query parameter {@code key}
* </ol>
*
* If no user is found, throws a wrapped bad api key (HTTP UNAUTHORIZED) response.
*
* @return The authenticated user which owns the passed api key.
* @throws edu.harvard.iq.dataverse.api.AbstractApiBean.WrappedResponse in case said user is not found.
*
* @deprecated Do not use this method.
* This method is expected to be removed once all API endpoints use the filter-based authentication.
* @see <a href="https://github.com/IQSS/dataverse/issues/9293">#9293</a>
* Replaced by:
* {@link #getRequestAuthenticatedUserOrDie(ContainerRequestContext)}
*/
@Deprecated
protected AuthenticatedUser findAuthenticatedUserOrDie() throws WrappedResponse {
return findAuthenticatedUserOrDie(getRequestApiKey(), getRequestWorkflowInvocationID());
}


private AuthenticatedUser findAuthenticatedUserOrDie( String key, String wfid ) throws WrappedResponse {
if (key != null) {
// No check for deactivated user because it's done in authSvc.lookupUser.
AuthenticatedUser authUser = authSvc.lookupUser(key);

if (authUser != null) {
authUser = userSvc.updateLastApiUseTime(authUser);

return authUser;
}
else {
throw new WrappedResponse(badApiKey(key));
}
} else if (wfid != null) {
AuthenticatedUser authUser = authSvc.lookupUserForWorkflowInvocationID(wfid);
if (authUser != null) {
return authUser;
} else {
throw new WrappedResponse(badWFKey(wfid));
}
} else if (getRequestParameter(UrlSignerUtil.SIGNED_URL_TOKEN) != null) {
AuthenticatedUser authUser = getAuthenticatedUserFromSignedUrl();
if (authUser != null) {
return authUser;
}
}
//Just send info about the apiKey - workflow users will learn about invocationId elsewhere
throw new WrappedResponse(badApiKey(null));
}

private AuthenticatedUser getAuthenticatedUserFromSignedUrl() {
AuthenticatedUser authUser = null;
// The signedUrl contains a param telling which user this is supposed to be for.
// We don't trust this. So we lookup that user, and get their API key, and use
// that as a secret in validating the signedURL. If the signature can't be
// validated with their key, the user (or their API key) has been changed and
// we reject the request.
// ToDo - add null checks/ verify that calling methods catch things.
String user = httpRequest.getParameter("user");
AuthenticatedUser targetUser = authSvc.getAuthenticatedUser(user);
String key = JvmSettings.API_SIGNING_SECRET.lookupOptional().orElse("")
+ authSvc.findApiTokenByUser(targetUser).getTokenString();
String signedUrl = httpRequest.getRequestURL().toString() + "?" + httpRequest.getQueryString();
String method = httpRequest.getMethod();
boolean validated = UrlSignerUtil.isValidUrl(signedUrl, user, method, key);
if (validated) {
authUser = targetUser;
}
return authUser;
}

protected Dataverse findDataverseOrDie( String dvIdtf ) throws WrappedResponse {
Dataverse dv = findDataverse(dvIdtf);
if ( dv == null ) {
Expand Down Expand Up @@ -745,36 +617,6 @@ protected Response response( Callable<Response> hdl ) {
}
}

/**
* The preferred way of handling a request that requires a user. The system
* looks for the user and, if found, handles it to the handler for doing the
* actual work.
*
* This is a relatively secure way to handle things, since if the user is not
* found, the response is about the bad API key, rather than something else
* (say, 404 NOT FOUND which leaks information about the existence of the
* sought object).
*
* @param hdl handling code block.
* @return HTTP Response appropriate for the way {@code hdl} executed.
*
* @deprecated Do not use this method.
* This method is expected to be removed once all API endpoints use the filter-based authentication.
* @see <a href="https://github.com/IQSS/dataverse/issues/9293">#9293</a>
* Replaced by:
* {@link #response(DataverseRequestHandler, User)}
*/
@Deprecated
protected Response response( DataverseRequestHandler hdl ) {
try {
return hdl.handle(createDataverseRequest(findUserOrDie()));
} catch ( WrappedResponse rr ) {
return rr.getResponse();
} catch ( Exception ex ) {
return handleDataverseRequestHandlerException(ex);
}
}

/***
* The preferred way of handling a request that requires a user. The method
* receives a user and handles it to the handler for doing the actual work.
Expand Down Expand Up @@ -916,19 +758,9 @@ protected Response forbidden( String msg ) {
protected Response conflict( String msg ) {
return error( Status.CONFLICT, msg );
}

protected Response badApiKey( String apiKey ) {
return error(Status.UNAUTHORIZED, (apiKey != null ) ? "Bad api key " : "Please provide a key query parameter (?key=XXX) or via the HTTP header " + DATAVERSE_KEY_HEADER_NAME);
}

protected Response badWFKey( String wfId ) {
String message = (wfId != null ) ? "Bad workflow invocationId " : "Please provide an invocationId query parameter (?invocationId=XXX) or via the HTTP header " + DATAVERSE_WORKFLOW_INVOCATION_HEADER_NAME;
return error(Status.UNAUTHORIZED, message );
}

protected Response authenticatedUserRequired() {
String message = "Only authenticated users can perform the requested operation";
return error(Status.UNAUTHORIZED, message );
return error(Status.UNAUTHORIZED, RESPONSE_MESSAGE_AUTHENTICATED_USER_REQUIRED);
}

protected Response permissionError( PermissionException pe ) {
Expand Down