Skip to content
This repository has been archived by the owner on Jul 10, 2024. It is now read-only.

SUBMARINE-527. [SDK] Abstract client API #315

Closed
wants to merge 6 commits into from

Conversation

pingsutw
Copy link
Member

@pingsutw pingsutw commented Jun 18, 2020

What is this PR for?

Try to simplify client API that generated by swagger, we could add an abstract layer on swagger API, and let users more easily submit experiments
Also, prettify output in the notebook

Before: Jupyter example
After: Jupyter example

What type of PR is it?

[Improvement]

Todos

  • - e2e test
  • - unit test

What is the Jira issue?

https://issues.apache.org/jira/browse/SUBMARINE-527?jql=project%20%3D%20SUBMARINE#

How should this be tested?

https://travis-ci.org/github/pingsutw/hadoop-submarine/builds/699648185

Screenshots (if appropriate)

Questions:

  • Does the licenses files need update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

@pingsutw
Copy link
Member Author

I found that list_log only output podName, and podLog always is NULL.
Should we change to list_podsName?

@Path("/logs")
@Operation(summary = "List experiment's log",
tags = {"experiment"},
responses = {
@ApiResponse(description = "successful operation", content = @Content(
schema = @Schema(implementation = JsonResponse.class)))})
public Response listLog(@QueryParam("status") String status) {
try {
List<ExperimentLog> experimentLogList = experimentManager.listExperimentLogsByStatus(status);
return new JsonResponse.Builder<List<ExperimentLog>>(Response.Status.OK).success(true)
.result(experimentLogList).build();
} catch (SubmarineRuntimeException e) {
return parseExperimentServiceException(e);
}
}

http://submarine:8080/api/v1/experiment/logs
output:

{
  "status": "OK",
  "code": 200,
  "success": true,
  "message": null,
  "result": [
    {
      "experimentId": "experiment_1592474530204_0001",
      "logContent": [
        {
          "podName": "tf-mnist-json-ps-0",
          "podLog": []
        },
        {
          "podName": "tf-mnist-json-worker-0",
          "podLog": []
        }
      ]
    }
  ],
  "attributes": {}
}

/cc @jiwq @tangzhankun

@tangzhankun
Copy link
Contributor

@pingsutw Thanks for the update! The "list experiment log by status" might have bugs in it? Need confirmation from @jiwq

For the API, can we add a wait_for_finish(id) method which gets logs periodically until the experiment finish?

@pingsutw
Copy link
Member Author

@tangzhankun I've add wait_for_finish

@tangzhankun
Copy link
Contributor

@pingsutw , thanks for the update! LGTM. Does the Travis failure relate to the PR?

@pingsutw
Copy link
Member Author

it's unrelated to the PR, I've moved all testing to GitHub-action

@asfgit asfgit closed this in c7bcdc0 Jun 19, 2020
@tangzhankun
Copy link
Contributor

tangzhankun commented Jun 19, 2020

@pingsutw Thanks for the contribution. Committed it.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants