Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset files API extension for filters #9783

Merged
merged 55 commits into from Oct 4, 2023

Conversation

GPortas
Copy link
Contributor

@GPortas GPortas commented Aug 16, 2023

What this PR does / why we need it:

We need this API extension to support dataset files tab filters in the new frontend, as well as extend the capabilities of the API for other use cases.

In particular, this API extension includes:

  • New optional query parameter accessStatus to getVersionFiles endpoint.
  • New optional query parameter contentType to getVersionFiles endpoint.
  • New optional query parameter categoryName to getVersionFiles endpoint.

Which issue(s) this PR closes:

Special notes for your reviewer:

The QueryDSL library has been added to Dataverse, which makes it much easier to build queries dynamically. This library integrates perfectly on top of the existing JPA + Jakarta setup. This library can be used from now on for other different use cases within Dataverse.

Other similar libraries have been studied but have been discarded in favor of QueryDSL, due to its simplicity and size.

Suggestions on how to test this:

The getVersionFiles endpoint integration test has been extended to cover all possible filter values, but manual tests can be done with curl:

Example for accessStatus:

curl -H "X-Dataverse-Key: <YOUR_API_KEY>" -X GET "http://localhost:8080/api/v1/datasets/:persistentId/versions/:latest/files?persistentId=<DATASED_PID>&accessStatus=Restricted"

Will retrieve all restricted files in the specified dataset.

accessStatus query parameter supports the possible values:

  • Public
  • Restricted
  • EmbargoedThenRestricted
  • EmbargoedThenPublic

Example for contentType:

curl -H "X-Dataverse-Key: <YOUR_API_KEY>" -X GET "http://localhost:8080/api/v1/datasets/:persistentId/versions/:latest/files?persistentId=<DATASED_PID>&contentType=image/png"

Example for categoryName:

curl -H "X-Dataverse-Key: <YOUR_API_KEY>" -X GET "http://localhost:8080/api/v1/datasets/:persistentId/versions/:latest/files?persistentId=<DATASED_PID>&categoryName=Data"

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

No

Is there a release notes update needed for this change?:

Yes

Additional documentation:

@GPortas GPortas added this to Ready for Review ⏩ in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) via automation Aug 16, 2023
@github-actions

This comment has been minimized.

@GPortas GPortas added Size: 3 A percentage of a sprint. 2.1 hours. Size: 10 A percentage of a sprint. 7 hours. and removed Size: 3 A percentage of a sprint. 2.1 hours. labels Aug 16, 2023
@github-actions

This comment has been minimized.

Copy link
Contributor

@landreev landreev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, based on a very quick look. I will take a closer look and add a more formal review.
I like the idea of adding a dedicated service bean (DataverseVersionFilesServiceBean). I'm wondering if we want to also add a dedicated command - ListVersionFilesCommand? - for this, mirroring the existing ListVersionsCommand.

@scolapasta scolapasta added this to the 6.1 milestone Aug 22, 2023
@scolapasta scolapasta added the Status: Waiting issues in need of input from someone currently unavailable label Aug 22, 2023
@scolapasta scolapasta moved this from Ready for Review ⏩ to ▶ SPRINT READY in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) Aug 22, 2023
@GPortas GPortas force-pushed the 9714-files-api-extension-filters branch from 8ce462c to 7153e54 Compare August 24, 2023 11:39
@github-actions

This comment has been minimized.

…SS/dataverse into 9714-files-api-extension-filters
…taverse into 9785-files-api-extension-search
@github-actions

This comment has been minimized.

…taverse into 9785-files-api-extension-search
@github-actions

This comment has been minimized.

IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) automation moved this from In Review 🔎 to Ready for QA ⏩ Sep 27, 2023
@landreev
Copy link
Contributor

landreev commented Sep 27, 2023

I moved the PR into ready for QA.
I hope it can be merged soon-ish. Once that's done, I will need to resolve further merge conflicts in my #9883, but that was expected.

@kcondon
Copy link
Contributor

kcondon commented Sep 28, 2023

@GPortas I've tested this and it is ready to merge but has merged conflicts. Would you take a look? Thanks.

Extend DataFile API payload and new endpoints for Files (getHasBeenDeleted) and Access (userFileAccessRequested)
Dataset files API extension for search text filtering
@github-actions

This comment has been minimized.

@GPortas
Copy link
Contributor Author

GPortas commented Oct 4, 2023

@kcondon

I'm analyzing the behavior of JSF following your feedback on #9820 (comment) about the search text filtering.

I have some considerations on this part of your comment: "It also supports file extension but couldn't get it to support file type as displayed on the page".

I have tested the files tab search in JSF for a published dataset and was unable to filter by the file type as displayed. Same occurs for the draft version. Maybe I'm missing something, but I don't think it is supported. I have recorded the behavior for a published dataset:

searchfiledisplay

I assume the reason this search criteria (file type as displayed) is not supported may be that a filter already exists for the file type. (?)

@github-actions
Copy link

github-actions bot commented Oct 4, 2023

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:9714-files-api-extension-filters
ghcr.io/gdcc/configbaker:9714-files-api-extension-filters

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Size: 10 A percentage of a sprint. 7 hours. SPA These changes are required for the Dataverse SPA
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

4 participants