-
Notifications
You must be signed in to change notification settings - Fork 476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
7159 fix restassured etc #7165
7159 fix restassured etc #7165
Changes from all commits
a0d94a4
1c95927
d15fa77
32756fc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1020,7 +1020,7 @@ public Response publishDataset(@PathParam("id") String id, @QueryParam("type") S | |
PublishDatasetResult res = execCommand(new PublishDatasetCommand(ds, | ||
createDataverseRequest(user), | ||
isMinor)); | ||
return res.isCompleted() ? ok(json(res.getDataset())) : accepted(json(res.getDataset())); | ||
return res.isWorkflow() ? accepted(json(res.getDataset())) : ok(json(res.getDataset())); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. does this result in a 200 when the dataset is still inprogress/publishing not yet finalized? Seems like 202 is the right code for that (as it was) and the test should be watching for a 202? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With the async publication in progress it returns a 200; the presence of the "finalizing" lock is the indication of it being in progress. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I understand the practicality and don't think it's a big issue in terms of any effect, but I think 202 is the right code for anything asynchronous that could fail after you get your response. If its not worth addressing now, I'd suggest a ToDo/note in the code. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So maybe the same code 202, but something in the body to differentiate from the "workflow" state. As it was implemented, it was returning the code 202 and |
||
} | ||
} catch (WrappedResponse ex) { | ||
return ex.getResponse(); | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,6 +21,7 @@ | |
import java.util.Optional; | ||
import java.util.logging.Logger; | ||
import static java.util.stream.Collectors.joining; | ||
import static edu.harvard.iq.dataverse.engine.command.impl.PublishDatasetResult.Status; | ||
|
||
/** | ||
* Kick-off a dataset publication process. The process may complete immediately, | ||
|
@@ -91,7 +92,7 @@ public PublishDatasetResult execute(CommandContext ctxt) throws CommandException | |
theDataset = ctxt.em().merge(theDataset); | ||
ctxt.em().flush(); | ||
ctxt.workflows().start(prePubWf.get(), buildContext(theDataset, TriggerType.PrePublishDataset, datasetExternallyReleased)); | ||
return new PublishDatasetResult(theDataset, false); | ||
return new PublishDatasetResult(theDataset, Status.Workflow); | ||
|
||
} else{ | ||
// We will skip trying to register the global identifiers for datafiles | ||
|
@@ -135,16 +136,18 @@ public PublishDatasetResult execute(CommandContext ctxt) throws CommandException | |
lock.setInfo(info); | ||
ctxt.datasets().addDatasetLock(theDataset, lock); | ||
theDataset = ctxt.em().merge(theDataset); | ||
ctxt.datasets().callFinalizePublishCommandAsynchronously(theDataset.getId(), ctxt, request, datasetExternallyReleased); | ||
return new PublishDatasetResult(theDataset, false); | ||
// The call to FinalizePublicationCommand has been moved to the new @onSuccess() | ||
// method: | ||
//ctxt.datasets().callFinalizePublishCommandAsynchronously(theDataset.getId(), ctxt, request, datasetExternallyReleased); | ||
return new PublishDatasetResult(theDataset, Status.Inprogress); | ||
|
||
/** | ||
* Code for for "synchronous" (while-you-wait) publishing | ||
* is preserved below, commented out: | ||
} else { | ||
// Synchronous publishing (no workflow involved) | ||
theDataset = ctxt.engine().submit(new FinalizeDatasetPublicationCommand(theDataset, getRequest(),datasetExternallyReleased)); | ||
return new PublishDatasetResult(theDataset, true); | ||
return new PublishDatasetResult(theDataset, Status.Completed); | ||
} */ | ||
} | ||
} | ||
|
@@ -196,6 +199,24 @@ private void verifyCommandArguments() throws IllegalCommandException { | |
throw new IllegalCommandException("Cannot release as minor version. Re-try as major release.", this); | ||
} | ||
} | ||
} | ||
} | ||
|
||
|
||
@Override | ||
public boolean onSuccess(CommandContext ctxt, Object r) { | ||
Dataset dataset = null; | ||
try{ | ||
dataset = (Dataset) r; | ||
} catch (ClassCastException e){ | ||
dataset = ((PublishDatasetResult) r).getDataset(); | ||
} | ||
|
||
if (dataset != null) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess this is just to be extra safe, but would this ever be false? (since we're already in OnSuccess, that means the execute succeeded, no? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess in case something else still goes wrong, in some other way? - idk. |
||
ctxt.datasets().callFinalizePublishCommandAsynchronously(dataset.getId(), ctxt, request, datasetExternallyReleased); | ||
return true; | ||
} | ||
|
||
return false; | ||
} | ||
|
||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -51,7 +51,8 @@ public class UtilIT { | |
private static final String BUILTIN_USER_KEY = "burrito"; | ||
private static final String EMPTY_STRING = ""; | ||
public static final int MAXIMUM_INGEST_LOCK_DURATION = 3; | ||
|
||
public static final int MAXIMUM_PUBLISH_LOCK_DURATION = 15; | ||
|
||
private static SwordConfigurationImpl swordConfiguration = new SwordConfigurationImpl(); | ||
|
||
static Matcher<String> equalToCI( String value ) { | ||
|
@@ -1017,10 +1018,14 @@ static Response deleteFile(Integer fileId, String apiToken) { | |
} | ||
|
||
static Response publishDatasetViaSword(String persistentId, String apiToken) { | ||
return given() | ||
Response publishResponse = given() | ||
.auth().basic(apiToken, EMPTY_STRING) | ||
.header("In-Progress", "false") | ||
.post(swordConfiguration.getBaseUrlPathCurrent() + "/edit/study/" + persistentId); | ||
|
||
// Wait for the dataset to get unlocked, if/as needed: | ||
sleepForLock(persistentId, "finalizePublication", apiToken, UtilIT.MAXIMUM_INGEST_LOCK_DURATION); | ||
return publishResponse; | ||
} | ||
|
||
static Response publishDatasetViaNativeApi(String idOrPersistentId, String majorOrMinor, String apiToken) { | ||
|
@@ -1036,18 +1041,36 @@ static Response publishDatasetViaNativeApi(String idOrPersistentId, String major | |
.urlEncodingEnabled(false) | ||
.header(UtilIT.API_TOKEN_HTTP_HEADER, apiToken); | ||
} | ||
return requestSpecification.post("/api/datasets/" + idInPath + "/actions/:publish?type=" + majorOrMinor + optionalQueryParam); | ||
Response publishResponse = requestSpecification.post("/api/datasets/" + idInPath + "/actions/:publish?type=" + majorOrMinor + optionalQueryParam); | ||
|
||
// Wait for the dataset to get unlocked, if/as needed: | ||
sleepForLock(idOrPersistentId, "finalizePublication", apiToken, UtilIT.MAXIMUM_INGEST_LOCK_DURATION); | ||
|
||
return publishResponse; | ||
} | ||
|
||
static Response publishDatasetViaNativeApiDeprecated(String persistentId, String majorOrMinor, String apiToken) { | ||
/** | ||
* @todo This should be a POST rather than a GET: | ||
* https://github.com/IQSS/dataverse/issues/2431 | ||
*/ | ||
return given() | ||
Response publishResponse = given() | ||
.header(API_TOKEN_HTTP_HEADER, apiToken) | ||
.urlEncodingEnabled(false) | ||
.get("/api/datasets/:persistentId/actions/:publish?type=" + majorOrMinor + "&persistentId=" + persistentId); | ||
|
||
// Wait for the dataset to get unlocked, if/as needed: | ||
sleepForLock(persistentId, "finalizePublication", apiToken, UtilIT.MAXIMUM_INGEST_LOCK_DURATION); | ||
|
||
return publishResponse; | ||
} | ||
|
||
// Compatibility method wrapper (takes Integer for the dataset id) | ||
// It used to use the GET version of the publish API; I'm switching it to | ||
// use the new POST variant. The explicitly marked @deprecated method above | ||
// should be sufficient for testing the deprecated API call. | ||
static Response publishDatasetViaNativeApi(Integer datasetId, String majorOrMinor, String apiToken) { | ||
return publishDatasetViaNativeApi(datasetId.toString(), majorOrMinor, apiToken); | ||
} | ||
|
||
static Response modifyDatasetPIDMetadataViaApi(String persistentId, String apiToken) { | ||
|
@@ -1061,17 +1084,6 @@ static Response modifyDatasetPIDMetadataViaApi(String persistentId, String apiT | |
.get("/api/datasets/:persistentId/&persistentId=" + persistentId); | ||
} | ||
|
||
static Response publishDatasetViaNativeApi(Integer datasetId, String majorOrMinor, String apiToken) { | ||
/** | ||
* @todo This should be a POST rather than a GET: | ||
* https://github.com/IQSS/dataverse/issues/2431 | ||
*/ | ||
return given() | ||
.header(API_TOKEN_HTTP_HEADER, apiToken) | ||
.urlEncodingEnabled(false) | ||
.get("/api/datasets/" + datasetId + "/actions/:publish?type=" + majorOrMinor); | ||
} | ||
|
||
static Response publishDataverseViaSword(String alias, String apiToken) { | ||
return given() | ||
.auth().basic(apiToken, EMPTY_STRING) | ||
|
@@ -2187,13 +2199,20 @@ public void testSwordStatementWithFiles() { | |
|
||
//Helper function that returns true if a given dataset locked for a given reason is unlocked within | ||
// a given duration returns false if still locked after given duration | ||
// (the version of the method that takes a long for the dataset id is | ||
// for backward compatibility with how the method is called for the | ||
// Ingest lock throughout the test code) | ||
static Boolean sleepForLock(long datasetId, String lockType, String apiToken, int duration) { | ||
String datasetIdAsString = String.valueOf(datasetId); | ||
return sleepForLock(datasetIdAsString, lockType, apiToken, duration); | ||
} | ||
|
||
Response lockedForIngest = UtilIT.checkDatasetLocks(datasetId, lockType, apiToken); | ||
static Boolean sleepForLock(String idOrPersistentId, String lockType, String apiToken, int duration) { | ||
Response lockedForIngest = UtilIT.checkDatasetLocks(idOrPersistentId, lockType, apiToken); | ||
int i = 0; | ||
do { | ||
try { | ||
lockedForIngest = UtilIT.checkDatasetLocks(datasetId, lockType, apiToken); | ||
lockedForIngest = UtilIT.checkDatasetLocks(idOrPersistentId, lockType, apiToken); | ||
Thread.sleep(1000); | ||
i++; | ||
if (i > duration) { | ||
|
@@ -2232,12 +2251,27 @@ static Boolean sleepForSearch(String searchPart, String apiToken, String subTre | |
|
||
} | ||
|
||
|
||
|
||
// backward compatibility version of the method that takes long for the id: | ||
static Response checkDatasetLocks(long datasetId, String lockType, String apiToken) { | ||
String datasetIdAsString = String.valueOf(datasetId); | ||
return checkDatasetLocks(datasetIdAsString, lockType, apiToken); | ||
} | ||
|
||
static Response checkDatasetLocks(String idOrPersistentId, String lockType, String apiToken) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Broad question - now that finalize is always asynchronous, it looks like these tests check that the publish call succeeds and the lock goes away, but is there anything to check that the async publish succeeded rather than failed? (E.g. test that a new version exists after the lock?). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That would be the proper way to handle it, yes. |
||
String idInPath = idOrPersistentId; // Assume it's a number. | ||
String queryParams = ""; // If idOrPersistentId is a number we'll just put it in the path. | ||
if (!NumberUtils.isNumber(idOrPersistentId)) { | ||
idInPath = ":persistentId"; | ||
queryParams = "?persistentId=" + idOrPersistentId; | ||
} | ||
|
||
if (lockType != null) { | ||
queryParams = "".equals(queryParams) ? "?type="+lockType : queryParams+"&type="+lockType; | ||
} | ||
|
||
Response response = given() | ||
.header(API_TOKEN_HTTP_HEADER, apiToken) | ||
.get("api/datasets/" + datasetId + "/locks" + (lockType == null ? "" : "?type="+lockType)); | ||
.get("api/datasets/" + idInPath + "/locks" + queryParams); | ||
return response; | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you try it without this? I'd love to get this removed, if we don't actually need it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I leave it in place?
I initially removed it completely; but I'm just afraid to take it out. Publishing appears to be working (at least I haven't seen it fail yet). But I believe if it happens instantly/very fast, on small datasets w/ fast registration, etc., it can confuse the autorefresh on the page.