Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datasets API extension for the SPA #9592

Merged
merged 19 commits into from Jun 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
2ef9297
Added: API endpoint for getting dataset summary field names (pending IT)
GPortas May 12, 2023
4a622a0
Added: getDatasetSummaryFieldNames IT
GPortas May 15, 2023
15a47b1
Added: missing LogoutIT to integration-tests.txt
GPortas May 15, 2023
cedba0e
Merge branch 'develop' into 9588-datasets-api-extension
GPortas May 15, 2023
60fde0c
Stash: getAnonymizedDraftVersion datasets endpoint WIP
GPortas May 19, 2023
d89f301
Added: getAnonymizedDraftVersion datasets endpoint
GPortas May 19, 2023
16ab0e8
Fixed: getAnonymizedDraftVersion endpoint to be getPrivateUrlDatasetV…
GPortas May 22, 2023
6e1ab82
Added: handling when private url user not found in getPrivateUrlDatas…
GPortas May 22, 2023
9d9d74e
Added: license icon URI to dataset version payload when it exists
GPortas May 26, 2023
c198730
Added: getDatasetVersionCitation endpoint
GPortas May 26, 2023
47fab3c
Removed: custom anonymized field value from getPrivateUrlDatasetVersi…
GPortas May 27, 2023
53edae5
Added: getPrivateUrlDatasetVersionCitation endpoint
GPortas May 27, 2023
6784770
Removed: anonymizedAccess query param from getDatasetVersionCitation …
GPortas May 27, 2023
f561e39
Fixed: getDatasetVersionCitation endpoint
GPortas May 28, 2023
514dac6
Merge branch 'develop' into 9588-datasets-api-extension #9588
pdurbin Jun 12, 2023
f5653e3
clean up tests #9588
pdurbin Jun 12, 2023
c3a1cad
add docs and release note #9588
pdurbin Jun 15, 2023
3ff1f19
Merge branch 'develop' into 9588-datasets-api-extension
pdurbin Jun 15, 2023
e59043f
add release note #9588
pdurbin Jun 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions doc/release-notes/9588-datasets-api-extension.md
@@ -0,0 +1,6 @@
The following APIs have been added:

- /api/datasets/summaryFieldNames
- /api/datasets/privateUrlDatasetVersion/{privateUrlToken}
- /api/datasets/privateUrlDatasetVersion/{privateUrlToken}/citation
- /api/datasets/{datasetId}/versions/{version}/citation
44 changes: 44 additions & 0 deletions doc/sphinx-guides/source/api/native-api.rst
Expand Up @@ -2173,6 +2173,50 @@ Signposting is not supported for draft dataset versions.

curl -H "Accept:application/json" "$SERVER_URL/api/datasets/:persistentId/versions/$VERSION/linkset?persistentId=$PERSISTENT_IDENTIFIER"

Get Dataset By Private URL Token
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

export SERVER_URL=https://demo.dataverse.org
export PRIVATE_URL_TOKEN=a56444bc-7697-4711-8964-e0577f055fd2

curl "$SERVER_URL/api/datasets/privateUrlDatasetVersion/$PRIVATE_URL_TOKEN"

Get Citation
~~~~~~~~~~~~

.. code-block:: bash

export SERVER_URL=https://demo.dataverse.org
export PERSISTENT_IDENTIFIER=doi:10.5072/FK2/YD5QDG
export VERSION=1.0

curl -H "Accept:application/json" "$SERVER_URL/api/datasets/:persistentId/versions/$VERSION/{version}/citation?persistentId=$PERSISTENT_IDENTIFIER"

Get Citation by Private URL Token
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

export SERVER_URL=https://demo.dataverse.org
export PRIVATE_URL_TOKEN=a56444bc-7697-4711-8964-e0577f055fd2

curl "$SERVER_URL/api/datasets/privateUrlDatasetVersion/$PRIVATE_URL_TOKEN/citation"

.. _get-dataset-summary-field-names:

Get Summary Field Names
~~~~~~~~~~~~~~~~~~~~~~~

See :ref:`:CustomDatasetSummaryFields` in the Installation Guide for how the list of dataset fields that summarize a dataset can be customized. Here's how to list them:

.. code-block:: bash

export SERVER_URL=https://demo.dataverse.org

curl "$SERVER_URL/api/datasets/summaryFieldNames"

Files
-----

Expand Down
6 changes: 6 additions & 0 deletions doc/sphinx-guides/source/installation/config.rst
Expand Up @@ -3406,6 +3406,8 @@ Limit on how many guestbook entries to display on the guestbook-responses page.

``curl -X PUT -d 10000 http://localhost:8080/api/admin/settings/:GuestbookResponsesPageDisplayLimit``

.. _:CustomDatasetSummaryFields:

:CustomDatasetSummaryFields
+++++++++++++++++++++++++++

Expand All @@ -3415,6 +3417,10 @@ You can replace the default dataset metadata fields that are displayed above fil

You have to put the datasetFieldType name attribute in the :CustomDatasetSummaryFields setting for this to work.

The default fields are ``dsDescription,subject,keyword,publication,notesText``.

This setting can be retrieved via API. See :ref:`get-dataset-summary-field-names` in the API Guide.

:AllowApiTokenLookupViaApi
++++++++++++++++++++++++++

Expand Down
63 changes: 63 additions & 0 deletions src/main/java/edu/harvard/iq/dataverse/api/Datasets.java
Expand Up @@ -10,6 +10,7 @@
import edu.harvard.iq.dataverse.authorization.RoleAssignee;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just putting this here the top but this PR needs:

  • docs
  • a release note

import edu.harvard.iq.dataverse.authorization.users.ApiToken;
import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser;
import edu.harvard.iq.dataverse.authorization.users.User;
import edu.harvard.iq.dataverse.batch.jobs.importer.ImportMode;
import edu.harvard.iq.dataverse.datacapturemodule.DataCaptureModuleUtil;
Expand Down Expand Up @@ -82,6 +83,7 @@
import edu.harvard.iq.dataverse.makedatacount.MakeDataCountLoggingServiceBean.MakeDataCountEntry;
import edu.harvard.iq.dataverse.metrics.MetricsUtil;
import edu.harvard.iq.dataverse.makedatacount.MakeDataCountUtil;
import edu.harvard.iq.dataverse.privateurl.PrivateUrlServiceBean;
import edu.harvard.iq.dataverse.settings.SettingsServiceBean;
import edu.harvard.iq.dataverse.settings.SettingsServiceBean.Key;
import edu.harvard.iq.dataverse.util.ArchiverUtil;
Expand Down Expand Up @@ -236,6 +238,9 @@ public class Datasets extends AbstractApiBean {
@EJB
DatasetVersionServiceBean datasetversionService;

@Inject
PrivateUrlServiceBean privateUrlService;

/**
* Used to consolidate the way we parse and handle dataset versions.
* @param <T>
Expand Down Expand Up @@ -3849,4 +3854,62 @@ public Response getExternalToolDVParams(@Context ContainerRequestContext crc,
return wr.getResponse();
}
}

@GET
@Path("summaryFieldNames")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started a thread in Slack about being consistent in how we expose database settings: https://iqss.slack.com/archives/C010LA04BCG/p1686603802872139

This might be fine but we should make an active decision because we'll be doing this a lot for the SPA. It needs to know how the backend is configured.

public Response getDatasetSummaryFieldNames() {
String customFieldNames = settingsService.getValueForKey(SettingsServiceBean.Key.CustomDatasetSummaryFields);
String[] fieldNames = DatasetUtil.getDatasetSummaryFieldNames(customFieldNames);
JsonArrayBuilder fieldNamesArrayBuilder = Json.createArrayBuilder();
for (String fieldName : fieldNames) {
fieldNamesArrayBuilder.add(fieldName);
}
return ok(fieldNamesArrayBuilder);
}

@GET
@Path("privateUrlDatasetVersion/{privateUrlToken}")
public Response getPrivateUrlDatasetVersion(@PathParam("privateUrlToken") String privateUrlToken) {
PrivateUrlUser privateUrlUser = privateUrlService.getPrivateUrlUserFromToken(privateUrlToken);
if (privateUrlUser == null) {
return notFound("Private URL user not found");
}
boolean isAnonymizedAccess = privateUrlUser.hasAnonymizedAccess();
String anonymizedFieldTypeNames = settingsSvc.getValueForKey(SettingsServiceBean.Key.AnonymizedFieldTypeNames);
if(isAnonymizedAccess && anonymizedFieldTypeNames == null) {
throw new NotAcceptableException("Anonymized Access not enabled");
}
DatasetVersion dsv = privateUrlService.getDraftDatasetVersionFromToken(privateUrlToken);
if (dsv == null || dsv.getId() == null) {
return notFound("Dataset version not found");
}
JsonObjectBuilder responseJson;
if (isAnonymizedAccess) {
List<String> anonymizedFieldTypeNamesList = new ArrayList<>(Arrays.asList(anonymizedFieldTypeNames.split(",\\s")));
responseJson = json(dsv, anonymizedFieldTypeNamesList);
} else {
responseJson = json(dsv);
}
return ok(responseJson);
}

@GET
@Path("privateUrlDatasetVersion/{privateUrlToken}/citation")
public Response getPrivateUrlDatasetVersionCitation(@PathParam("privateUrlToken") String privateUrlToken) {
PrivateUrlUser privateUrlUser = privateUrlService.getPrivateUrlUserFromToken(privateUrlToken);
if (privateUrlUser == null) {
return notFound("Private URL user not found");
}
DatasetVersion dsv = privateUrlService.getDraftDatasetVersionFromToken(privateUrlToken);
return (dsv == null || dsv.getId() == null) ? notFound("Dataset version not found")
: ok(dsv.getCitation(true, privateUrlUser.hasAnonymizedAccess()));
}

@GET
@AuthRequired
@Path("{id}/versions/{versionId}/citation")
public Response getDatasetVersionCitation(@Context ContainerRequestContext crc, @PathParam("id") String datasetId, @PathParam("versionId") String versionId, @Context UriInfo uriInfo, @Context HttpHeaders headers) {
return response(req -> ok(
getDatasetVersionOrDie(req, versionId, findDatasetOrDie(datasetId), uriInfo, headers).getCitation(true, false)), getRequestUser(crc));
}
}
46 changes: 24 additions & 22 deletions src/main/java/edu/harvard/iq/dataverse/dataset/DatasetUtil.java
Expand Up @@ -44,6 +44,7 @@
public class DatasetUtil {

private static final Logger logger = Logger.getLogger(DatasetUtil.class.getCanonicalName());
public static final String datasetDefaultSummaryFieldNames = "dsDescription,subject,keyword,publication,notesText";
public static String datasetLogoFilenameFinal = "dataset_logo_original";
public static String datasetLogoThumbnail = "dataset_logo";
public static String thumbExtension = ".thumb";
Expand Down Expand Up @@ -429,32 +430,33 @@ public static boolean isDatasetLogoPresent(Dataset dataset, int size) {
return false;
}

public static List<DatasetField> getDatasetSummaryFields(DatasetVersion datasetVersion, String customFields) {

List<DatasetField> datasetFields = new ArrayList<>();

//if customFields are empty, go with default fields.
if(customFields==null || customFields.isEmpty()){
customFields="dsDescription,subject,keyword,publication,notesText";
}

String[] customFieldList= customFields.split(",");
Map<String,DatasetField> DatasetFieldsSet=new HashMap<>();

public static List<DatasetField> getDatasetSummaryFields(DatasetVersion datasetVersion, String customFieldNames) {
Map<String, DatasetField> datasetFieldsSet = new HashMap<>();
for (DatasetField dsf : datasetVersion.getFlatDatasetFields()) {
DatasetFieldsSet.put(dsf.getDatasetFieldType().getName(),dsf);
datasetFieldsSet.put(dsf.getDatasetFieldType().getName(), dsf);
}
String[] summaryFieldNames = getDatasetSummaryFieldNames(customFieldNames);
List<DatasetField> datasetSummaryFields = new ArrayList<>();
for (String summaryFieldName : summaryFieldNames) {
DatasetField df = datasetFieldsSet.get(summaryFieldName);
if (df != null) {
datasetSummaryFields.add(df);
}
}

for(String cfl : customFieldList)
{
DatasetField df = DatasetFieldsSet.get(cfl);
if(df!=null)
datasetFields.add(df);
return datasetSummaryFields;
}

public static String[] getDatasetSummaryFieldNames(String customFieldNames) {
String summaryFieldNames;
// If the custom fields are empty, go with the default fields.
if(customFieldNames == null || customFieldNames.isEmpty()){
summaryFieldNames = datasetDefaultSummaryFieldNames;
} else {
summaryFieldNames = customFieldNames;
}

return datasetFields;
return summaryFieldNames.split(",");
}

public static boolean isRsyncAppropriateStorageDriver(Dataset dataset){
// ToDo - rsync was written before multiple store support and currently is hardcoded to use the DataAccess.S3 store.
// When those restrictions are lifted/rsync can be configured per store, this test should check that setting
Expand Down
Expand Up @@ -2,6 +2,7 @@

import edu.harvard.iq.dataverse.Dataset;
import edu.harvard.iq.dataverse.DatasetServiceBean;
import edu.harvard.iq.dataverse.DatasetVersion;
import edu.harvard.iq.dataverse.RoleAssignment;
import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser;
import edu.harvard.iq.dataverse.util.SystemConfig;
Expand Down Expand Up @@ -61,6 +62,13 @@ public PrivateUrlRedirectData getPrivateUrlRedirectDataFromToken(String token) {
return PrivateUrlUtil.getPrivateUrlRedirectData(getRoleAssignmentFromPrivateUrlToken(token));
}

/**
* @return DatasetVersion if it can be found using the token or null.
*/
public DatasetVersion getDraftDatasetVersionFromToken(String token) {
return PrivateUrlUtil.getDraftDatasetVersionFromRoleAssignment(getRoleAssignmentFromPrivateUrlToken(token));
}

/**
* @return A RoleAssignment or null.
*
Expand Down