Skip to content

Commit

Permalink
[SPARK-29053][WEBUI] Sort does not work on some columns
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
Setting custom sort key for duration and execution time column.

### Why are the changes needed?
Sorting on duration and execution time columns consider time as a string after converting into readable form which is the reason for wrong sort results as mentioned in [SPARK-29053](https://issues.apache.org/jira/browse/SPARK-29053).

### Does this PR introduce any user-facing change?
No

### How was this patch tested?
Test manually. Screenshots are attached.

After patch:
**Duration**
![Duration](https://user-images.githubusercontent.com/40591404/65339861-93cc9800-dbea-11e9-95e6-63b107a5a372.png)
**Execution time**
![Execution Time](https://user-images.githubusercontent.com/40591404/65339870-97601f00-dbea-11e9-9d1d-690c59bc1bde.png)

Closes #25855 from amanomer/SPARK29053.

Authored-by: aman_omer <amanomer1996@gmail.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
  • Loading branch information
amanomer authored and srowen committed Sep 21, 2019
1 parent 076186e commit 93ac4e1
Showing 1 changed file with 6 additions and 3 deletions.
Expand Up @@ -91,8 +91,10 @@ private[ui] class ThriftServerPage(parent: ThriftServerTab) extends WebUIPage(""
<td>{formatDate(info.startTimestamp)}</td>
<td>{if (info.finishTimestamp > 0) formatDate(info.finishTimestamp)}</td>
<td>{if (info.closeTimestamp > 0) formatDate(info.closeTimestamp)}</td>
<td>{formatDurationOption(Some(info.totalTime(info.finishTimestamp)))}</td>
<td>{formatDurationOption(Some(info.totalTime(info.closeTimestamp)))}</td>
<td sorttable_customkey={info.totalTime(info.finishTimestamp).toString}>
{formatDurationOption(Some(info.totalTime(info.finishTimestamp)))} </td>
<td sorttable_customkey={info.totalTime(info.closeTimestamp).toString}>
{formatDurationOption(Some(info.totalTime(info.closeTimestamp)))} </td>
<td>{info.statement}</td>
<td>{info.state}</td>
{errorMessageCell(detail)}
Expand Down Expand Up @@ -157,7 +159,8 @@ private[ui] class ThriftServerPage(parent: ThriftServerTab) extends WebUIPage(""
<td> <a href={sessionLink}> {session.sessionId} </a> </td>
<td> {formatDate(session.startTimestamp)} </td>
<td> {if (session.finishTimestamp > 0) formatDate(session.finishTimestamp)} </td>
<td> {formatDurationOption(Some(session.totalTime))} </td>
<td sorttable_customkey={session.totalTime.toString}>
{formatDurationOption(Some(session.totalTime))} </td>
<td> {session.totalExecution.toString} </td>
</tr>
}
Expand Down

0 comments on commit 93ac4e1

Please sign in to comment.