Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renaming the new APIs for auxiliary files. (#7850) #7866

Merged
merged 3 commits into from
May 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/api/dataaccess.rst
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ especially with data files with large numbers of variables. See
Preprocessed Data
-----------------

``/api/access/datafile/$id/metadata/preprocessed``
``/api/access/datafile/$id?format=prep``

This method provides the "preprocessed data" - a summary record that describes the values of the data vectors in the tabular file, in JSON. These metadata values are used by TwoRavens, an external tool that integrates with a Dataverse installation. Please note that this format might change in the future.

Expand Down
4 changes: 2 additions & 2 deletions doc/sphinx-guides/source/developers/aux-file-support.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ To add an auxiliary file, specify the primary key of the datafile (FILE_ID), and
export TYPE='DP'
export SERVER_URL=https://demo.dataverse.org

curl -H X-Dataverse-key:$API_TOKEN -X POST -F "file=@$FILENAME" -F 'origin=myApp' -F 'isPublic=true' -F "type=$TYPE" "$SERVER_URL/api/access/datafile/$FILE_ID/metadata/$FORMAT_TAG/$FORMAT_VERSION"
curl -H X-Dataverse-key:$API_TOKEN -X POST -F "file=@$FILENAME" -F 'origin=myApp' -F 'isPublic=true' -F "type=$TYPE" "$SERVER_URL/api/access/datafile/$FILE_ID/auxiliary/$FORMAT_TAG/$FORMAT_VERSION"

You should expect a 200 ("OK") response and JSON with information about your newly uploaded auxiliary file.

Expand All @@ -34,4 +34,4 @@ formatTag and formatVersion (if applicable) associated with the auxiliary file:
export FORMAT_TAG='dpJson'
export FORMAT_VERSION='v1'

curl "$SERVER_URL/api/access/datafile/$FILE_ID/metadata/$FORMAT_TAG/$FORMAT_VERSION"
curl "$SERVER_URL/api/access/datafile/$FILE_ID/auxiliary/$FORMAT_TAG/$FORMAT_VERSION"
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ private void redirectToBatchDownloadAPI(String multiFileString, Boolean download
}

public void redirectToAuxFileDownloadAPI(Long fileId, String formatTag, String formatVersion) {
String fileDownloadUrl = "/api/access/datafile/" + fileId + "/metadata/" + formatTag + "/" + formatVersion;
String fileDownloadUrl = "/api/access/datafile/" + fileId + "/auxiliary/" + formatTag + "/" + formatVersion;
try {
FacesContext.getCurrentInstance().getExternalContext().redirect(fileDownloadUrl);
} catch (IOException ex) {
Expand Down
45 changes: 27 additions & 18 deletions src/main/java/edu/harvard/iq/dataverse/api/Access.java
Original file line number Diff line number Diff line change
Expand Up @@ -542,12 +542,11 @@ public String dataVariableMetadataDDI(@PathParam("varId") Long varId, @QueryPara
* GET method for retrieving various auxiliary files associated with
* a tabular datafile.
*
* TODO: Consider removing "metadata" from the path.
*/

@Path("datafile/{fileId}/metadata/{formatTag}/{formatVersion}")
@Path("datafile/{fileId}/auxiliary/{formatTag}/{formatVersion}")
@GET
public DownloadInstance tabularDatafileMetadataAux(@PathParam("fileId") String fileId,
public DownloadInstance downloadAuxiliaryFile(@PathParam("fileId") String fileId,
@PathParam("formatTag") String formatTag,
@PathParam("formatVersion") String formatVersion,
@QueryParam("key") String apiToken,
Expand All @@ -571,6 +570,17 @@ public DownloadInstance tabularDatafileMetadataAux(@PathParam("fileId") String f
DownloadInstance downloadInstance;
AuxiliaryFile auxFile = null;

/*
The special case for "preprocessed" metadata should not be here at all.
Access to the format should be handled by the /api/access/datafile/{id}?format=prep
form exclusively (this is what Data Explorer and Tworavens have been
using all along). We may have advertised /api/access/datafile/{id}/metadata/preprocessed
in the past - but it has been broken since 5.3 anyway, since the /{formatVersion}
element was added to the @Path.
Now that the api method has been renamed /api/access/datafile/{id}/auxiliary/...,
nobody should be using it to access the "preprocessed" format.
Leaving the special case below commented-out, for now. - L.A.

// formatTag=preprocessed is handled as a special case.
// This is (as of now) the only aux. tabular metadata format that Dataverse
// can generate (and cache) itself. (All the other formats served have
Expand All @@ -582,21 +592,21 @@ public DownloadInstance tabularDatafileMetadataAux(@PathParam("fileId") String f
if (downloadInstance.checkIfServiceSupportedAndSetConverter("format", "prep")) {
logger.fine("Preprocessed data for tabular file "+fileId);
}
} else {
// All other (deposited) formats:
auxFile = auxiliaryFileService.lookupAuxiliaryFile(df, formatTag, formatVersion);

if (auxFile == null) {
throw new NotFoundException("Auxiliary metadata format "+formatTag+" is not available for datafile "+fileId);
}
} else { */
// All other (deposited) formats:
auxFile = auxiliaryFileService.lookupAuxiliaryFile(df, formatTag, formatVersion);

// Don't consider aux file public unless data file is published.
if (auxFile.getIsPublic() && df.getPublicationDate() != null) {
publiclyAvailable = true;
}
downloadInstance = new DownloadInstance(dInfo);
downloadInstance.setAuxiliaryFile(auxFile);
if (auxFile == null) {
throw new NotFoundException("Auxiliary metadata format " + formatTag + " is not available for datafile " + fileId);
}

// Don't consider aux file public unless data file is published.
if (auxFile.getIsPublic() && df.getPublicationDate() != null) {
publiclyAvailable = true;
}
downloadInstance = new DownloadInstance(dInfo);
downloadInstance.setAuxiliaryFile(auxFile);
/*}*/

// Unless this format is explicitly authorized to be publicly available,
// the following will check access authorization (based on the access rules
Expand Down Expand Up @@ -1206,9 +1216,8 @@ private String getWebappImageResource(String imageName) {
* @param formDataBodyPart
* @return
*
* TODO: Consider removing "metadata" from the path.
*/
@Path("datafile/{fileId}/metadata/{formatTag}/{formatVersion}")
@Path("datafile/{fileId}/auxiliary/{formatTag}/{formatVersion}")
@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)

Expand Down
4 changes: 2 additions & 2 deletions src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java
Original file line number Diff line number Diff line change
Expand Up @@ -651,13 +651,13 @@ static Response uploadAuxFile(Long fileId, String pathToFile, String formatTag,
if (type != null) {
requestSpecification.multiPart("type", type);
}
return requestSpecification.post("/api/access/datafile/" + fileId + "/metadata/" + formatTag + "/" + formatVersion);
return requestSpecification.post("/api/access/datafile/" + fileId + "/auxiliary/" + formatTag + "/" + formatVersion);
}

static Response downloadAuxFile(Long fileId, String formatTag, String formatVersion, String apiToken) {
Response response = given()
.header(API_TOKEN_HTTP_HEADER, apiToken)
.get("/api/access/datafile/" + fileId + "/metadata/" + formatTag + "/" + formatVersion);
.get("/api/access/datafile/" + fileId + "/auxiliary/" + formatTag + "/" + formatVersion);
return response;
}

Expand Down