Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3322 Empty OAI sets should not be listed in ListSets #10228

Merged
merged 6 commits into from
Jan 18, 2024

Conversation

landreev
Copy link
Contributor

What this PR does / why we need it:

Another OAI issue that's 7 years old, but still valid.
Pretty straightforward: when an OAI set is created, it should not be shown in ListSets until it is successfully exported and contains 1 or more dataset records.

Which issue(s) this PR closes:

Closes #3322

Special notes for your reviewer:

Slightly out of scope, but while I was at it, I noticed that the harvesting server page had some calls requiring potentially non-trivial database lookups in the rendering logic rules. This is likely making our own prod. harvesting server page take 7 times longer to load.
Made the page cache these numbers (7 times = the number of times rendering logic rules are evaluated)

Suggestions on how to test this:

Create a set or two, via the API or the UI, experiment.
When creating a set on the harvesting server dashboard page, it is exported right away. So the easiest way to test this would be to create a set that doesn't match any records. With a query like dsPersistentId:NOSUCHID. Confirm that the set isn't shown in the output of the OAI ListSets.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Additional documentation:

@coveralls
Copy link

coveralls commented Jan 11, 2024

Coverage Status

coverage: 20.147% (-0.007%) from 20.154%
when pulling 867b7dc on 3322-oai-list-sets
into 39794d5 on develop.

This comment has been minimized.

@sekmiller sekmiller self-assigned this Jan 12, 2024
@cmbz cmbz added the Size: 10 A percentage of a sprint. 7 hours. label Jan 16, 2024
@landreev
Copy link
Contributor Author

Please note that the HarvestingServer API tests require the following jvm options:

<jvm-options>-Ddataverse.oai.server.maxidentifiers=2</jvm-options>
<jvm-options>-Ddataverse.oai.server.maxrecords=2</jvm-options>

This info is in the guide, but may be a bit buried.

@landreev
Copy link
Contributor Author

(I just kicked off a Jenkins run manually)

@stevenwinship stevenwinship self-assigned this Jan 18, 2024
@landreev
Copy link
Contributor Author

Checked in a better definition for the test in the PR. Test is passing locally, Jenkins run is in progress.

This comment has been minimized.

Copy link

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:3322-oai-list-sets
ghcr.io/gdcc/configbaker:3322-oai-list-sets

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

@stevenwinship stevenwinship merged commit be93513 into develop Jan 18, 2024
18 of 19 checks passed
@stevenwinship stevenwinship deleted the 3322-oai-list-sets branch January 18, 2024 22:00
@stevenwinship stevenwinship removed their assignment Jan 19, 2024
@pdurbin pdurbin added this to the 6.2 milestone Jan 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature: Harvesting Size: 10 A percentage of a sprint. 7 hours.
Projects
Status: Done 🧹
Development

Successfully merging this pull request may close these issues.

Harvest: Do not list a set as available in ListSets until it has been successfully exported at least once.
6 participants