Skip to content

Commit

Permalink
[SPARK-46704][CORE][UI] Fix MasterPage to sort Running Drivers ta…
Browse files Browse the repository at this point in the history
…ble by `Duration` column correctly

### What changes were proposed in this pull request?

This PR aims to fix `MasterPage` to sort `Running Drivers` table by `Duration` column correctly.

### Why are the changes needed?

Since Apache Spark 3.0.0, `MasterPage` shows `Duration` column of `Running Drivers`.

**BEFORE**
<img width="111" src="https://github.com/apache/spark/assets/9700541/50276e34-01be-4474-803d-79066e06cb2c">

**AFTER**
<img width="111" src="https://github.com/apache/spark/assets/9700541/a427b2e6-eab0-4d73-9114-1d8ff9d052c2">

### Does this PR introduce _any_ user-facing change?

Yes, this is a bug fix of UI.

### How was this patch tested?

Manual.

Run a Spark standalone cluster.
```
$ SPARK_MASTER_OPTS="-Dspark.master.rest.enabled=true -Dspark.deploy.maxDrivers=2" sbin/start-master.sh
$ sbin/start-worker.sh spark://$(hostname):7077
```

Submit multiple jobs via REST API.
```
$ curl -s -k -XPOST http://localhost:6066/v1/submissions/create \
    --header "Content-Type:application/json;charset=UTF-8" \
    --data '{
      "appResource": "",
      "sparkProperties": {
        "spark.master": "spark://localhost:7077",
        "spark.app.name": "Test 1",
        "spark.submit.deployMode": "cluster",
        "spark.jars": "/Users/dongjoon/APACHE/spark-merge/examples/target/scala-2.13/jars/spark-examples_2.13-4.0.0-SNAPSHOT.jar"
      },
      "clientSparkVersion": "",
      "mainClass": "org.apache.spark.examples.SparkPi",
      "environmentVariables": {},
      "action": "CreateSubmissionRequest",
      "appArgs": [ "10000" ]
    }'
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #44711 from dongjoon-hyun/SPARK-46704.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed Jan 12, 2024
1 parent 6e54315 commit 25c680c
Showing 1 changed file with 3 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -375,7 +375,9 @@ private[ui] class MasterPage(parent: MasterWebUI) extends WebUIPage("") {
<td>{formatResourcesAddresses(driver.resources)}</td>
<td>{driver.desc.command.arguments(2)}</td>
{if (showDuration) {
<td>{UIUtils.formatDuration(System.currentTimeMillis() - driver.startTime)}</td>
<td sorttable_customkey={(-driver.startTime).toString}>
{UIUtils.formatDuration(System.currentTimeMillis() - driver.startTime)}
</td>
}}
</tr>
}
Expand Down

0 comments on commit 25c680c

Please sign in to comment.