Skip to content

Commit

Permalink
[SPARK-13675][UI] Fix wrong historyserver url link for application ru…
Browse files Browse the repository at this point in the history
…nning in yarn cluster mode

## What changes were proposed in this pull request?

Current URL for each application to access history UI is like:
http://localhost:18080/history/application_1457058760338_0016/1/jobs/ or http://localhost:18080/history/application_1457058760338_0016/2/jobs/

Here **1** or **2** represents the number of attempts in `historypage.js`, but it will parse to attempt id in `HistoryServer`, while the correct attempt id should be like "appattempt_1457058760338_0016_000002", so it will fail to parse to a correct attempt id in HistoryServer.

This is OK in yarn client mode, since we don't need this attempt id to fetch out the app cache, but it is failed in yarn cluster mode, where attempt id "1" or "2" is actually wrong.

So here we should fix this url to parse the correct application id and attempt id. Also the suffix "jobs/" is not needed.

Here is the screenshot:

![screen shot 2016-02-29 at 3 57 32 pm](https://cloud.githubusercontent.com/assets/850797/13524377/d4b44348-e235-11e5-8b3e-bc06de306e87.png)

## How was this patch tested?

This patch is tested manually, with different master and deploy mode.

![image](https://cloud.githubusercontent.com/assets/850797/13524419/118be5a0-e236-11e5-8022-3ff613ccde46.png)

Author: jerryshao <sshao@hortonworks.com>

Closes #11518 from jerryshao/SPARK-13675.
  • Loading branch information
jerryshao authored and Tom Graves committed Mar 8, 2016
1 parent 9bf76dd commit 9e86e6e
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@
<tbody>
{{#applications}}
<tr>
<td class="rowGroupColumn"><span title="{{id}}"><a href="/history/{{id}}/{{num}}/jobs/">{{id}}</a></span></td>
<td class="rowGroupColumn"><span title="{{id}}"><a href="{{url}}">{{id}}</a></span></td>
<td class="rowGroupColumn">{{name}}</td>
{{#attempts}}
<td class="attemptIDSpan"><a href="/history/{{id}}/{{attemptId}}/jobs/">{{attemptId}}</a></td>
<td class="attemptIDSpan"><a href="/history/{{id}}/{{attemptId}}/">{{attemptId}}</a></td>
<td>{{startTime}}</td>
<td>{{endTime}}</td>
<td><span title="{{duration}}" class="durationClass">{{duration}}</span></td>
Expand Down
19 changes: 17 additions & 2 deletions core/src/main/resources/org/apache/spark/ui/static/historypage.js
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,28 @@ $(document).ready(function() {
if (app["attempts"].length > 1) {
hasMultipleAttempts = true;
}
var num = app["attempts"].length;

var maxAttemptId = null
for (j in app["attempts"]) {
var attempt = app["attempts"][j];
if (attempt['attemptId'] != null) {
if (maxAttemptId == null || attempt['attemptId'] > maxAttemptId) {
maxAttemptId = attempt['attemptId']
}
}

attempt["startTime"] = formatDate(attempt["startTime"]);
attempt["endTime"] = formatDate(attempt["endTime"]);
attempt["lastUpdated"] = formatDate(attempt["lastUpdated"]);
var app_clone = {"id" : id, "name" : name, "num" : num, "attempts" : [attempt]};

var url = null
if (maxAttemptId == null) {
url = "/history/" + id + "/"
} else {
url = "/history/" + id + "/" + maxAttemptId + "/"
}

var app_clone = {"id" : id, "name" : name, "url" : url, "attempts" : [attempt]};
array.push(app_clone);
}
}
Expand Down

0 comments on commit 9e86e6e

Please sign in to comment.