Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test failure in develop: HarvestingServerIT.testOaiFunctionality #8937

Closed
pdurbin opened this issue Aug 25, 2022 · 1 comment · Fixed by #8939 or #9026
Closed

test failure in develop: HarvestingServerIT.testOaiFunctionality #8937

pdurbin opened this issue Aug 25, 2022 · 1 comment · Fixed by #8939 or #9026
Projects
IQSS/dataverse (TO BE RETIRED / DELET...
Milestone
5.12

Comments

@pdurbin
Copy link
Member

pdurbin commented Aug 25, 2022

What steps does it take to reproduce the issue?

  • Open HarvestingServerIT and run testOaiFunctionality.
  • Observe that the test fails

The output from the test is something like the XML below.

<OAI-PMH schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2022-08-25T14:00:57Z</responseDate>
  <request verb="ListIdentifiers" metadataPrefix="oai_dc" set="YZOIGQ">http://localhost:8080/oai</request>
  <ListIdentifiers>
    <header>
      <identifier>doi:10.5072/FK2/YZOIGQ</identifier>
      <datestamp>2022-08-25T14:00:52Z</datestamp>
      <setSpec>YZOIGQ</setSpec>
    </header>
  </ListIdentifiers>
</OAI-PMH>

The test is looking for a list of size 1:

    // Run ListIdentifiers on this newly-created set:
    Response listIdentifiersResponse = UtilIT.getOaiListIdentifiers(setName, "oai_dc");
    List ret = listIdentifiersResponse.getBody().xmlPath().getList("OAI-PMH.ListIdentifiers.header");

    assertEquals(OK.getStatusCode(), listIdentifiersResponse.getStatusCode());
    assertNotNull(ret);
    logger.info("setName: " + setName);
    logger.info("listIdentifiersResponse.prettyPrint:..... ");
    // There should be 1 and only 1 record in the response:
    assertEquals(1, ret.size());

Which version of Dataverse are you using?

The first observed failure on Jenkins on the develop branch was for 1ef73b0 (the tip of develop as of this writing).

We're also seeing the failure in PR #8880 where the tip of develop was merged.

Locally I'm on 6ca1c9d (I haven't pulled from develop so I haven't picked up the merge from PR #8901 yet) and @sekmiller also hasn't pulled and we are both seeing the failure.
@landreev
Copy link
Contributor

landreev commented Aug 25, 2022 •
edited

Quick status update - after much poking/discussing it on slack, it appears to be a) not obviously connected to any recent changes and b) intermittent. And seems like another instance of the overall test suite becoming more resource-intensive and slowing down to the point where an individual test that relies on a hard-coded sleep to ensure the completion of previous steps starts failing intermittently.
In this specific case there's a 5 sec. sleep in this test meant to let the OAI set re-export to finish. I'll make a PR doubling that interval to 10 sec.

Granted, there are smarter ways to go about it; but not sure it’s worth it at this point. I have opened the issue #8843 recently for expanding the number and scope of our Harvesting/OAI tests that I very much want to prioritize. Anything “smarter” should be done as part of that effort.

Edit: actually, the test was failing not on account of the set create taking too long, but because we were trying to create the set before the publish command had a chance to complete on the dataset there. Or, to be more specific, while the dataset was still being indexed and imported - the tasks that the publish command executes asynchronously. So it wasn't the 5 sec. sleep that needed to be increased; it was another short sleep that needed to be added before the set create step.

@pdurbin pdurbin mentioned this issue Aug 25, 2022
Merged
landreev added a commit that referenced this issue Aug 26, 2022
@landreev
doubles the sleep interval in a harvesting test. #8937
f07b824
@landreev landreev mentioned this issue Aug 26, 2022
Merged
landreev added a commit that referenced this issue Aug 26, 2022
@landreev
sleep added in the right place (#8937)
781ca74
IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) automation moved this from This Sprint 🏃‍♀️ to Done 🚀 Aug 26, 2022
pdurbin added a commit that referenced this issue Aug 26, 2022
@pdurbin
Merge pull request #8939 from IQSS/8937-harvest-test-failure
eed4775 
Quick fix for a failing harvesting test  #8937
@pdurbin pdurbin added this to the 5.12 milestone Aug 26, 2022
@qqmyers qqmyers mentioned this issue Oct 5, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
No open projects
IQSS/dataverse (TO BE RETIRED / DELET...
Done 🚀
Milestone
5.12
2 participants
@pdurbin @landreev