Skip to content

Commit

Permalink
[SPARK-32823][WEB UI] Fix the master ui resources reporting
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

Fixes the master UI for properly summing the resources total across multiple workers.
field:
Resources in use: 0 / 8 gpu

The bug here is that it was creating MutableResourceInfo and then reducing using the + operator.  the + operator in MutableResourceInfo simple adds the address from one to the addresses of the other.  But its using a HashSet so if the addresses are the same then you lose the correct amount.  ie worker1 has gpu addresses 0,1,2,3 and worker2 has addresses 0,1,2,3 then you only see 4 total GPUs when there are 8.

In this case we don't really need to create the MutableResourceInfo at all because we just want the sums for used and total so just remove the use of it.  The other uses of it are per Worker so those should be ok.

### Why are the changes needed?

fix UI

### Does this PR introduce _any_ user-facing change?

UI

### How was this patch tested?

tested manually on standalone cluster with multiple workers and multiple GPUs and multiple fpgas

Closes apache#29683 from tgravescs/SPARK-32823.

Lead-authored-by: Thomas Graves <tgraves@nvidia.com>
Co-authored-by: Thomas Graves <tgraves@apache.org>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
  • Loading branch information
2 people authored and HyukjinKwon committed Sep 9, 2020
1 parent adc8d68 commit 514bf56
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 11 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,11 @@ private[spark] object StandaloneResourceUtils extends Logging {

// used for UI
def formatResourcesUsed(
resourcesTotal: Map[String, ResourceInformation],
resourcesUsed: Map[String, ResourceInformation]): String = {
resourcesTotal.map { case (rName, rInfo) =>
val used = resourcesUsed(rName).addresses.length
val total = rInfo.addresses.length
resourcesTotal: Map[String, Int],
resourcesUsed: Map[String, Int]): String = {
resourcesTotal.map { case (rName, totalSize) =>
val used = resourcesUsed(rName)
val total = totalSize
s"$used / $total $rName"
}.mkString(", ")
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,19 +76,17 @@ private[ui] class MasterPage(parent: MasterWebUI) extends WebUIPage("") {

private def formatMasterResourcesInUse(aliveWorkers: Array[WorkerInfo]): String = {
val totalInfo = aliveWorkers.map(_.resourcesInfo)
.map(resources => toMutable(resources))
.flatMap(_.toIterator)
.groupBy(_._1) // group by resource name
.map { case (rName, rInfoArr) =>
rName -> rInfoArr.map(_._2).reduce(_ + _)
}.map { case (k, v) => (k, v.toResourceInformation) }
rName -> rInfoArr.map(_._2.addresses.size).sum
}
val usedInfo = aliveWorkers.map(_.resourcesInfoUsed)
.map (resources => toMutable(resources))
.flatMap(_.toIterator)
.groupBy(_._1) // group by resource name
.map { case (rName, rInfoArr) =>
rName -> rInfoArr.map(_._2).reduce(_ + _)
}.map { case (k, v) => (k, v.toResourceInformation) }
rName -> rInfoArr.map(_._2.addresses.size).sum
}
formatResourcesUsed(totalInfo, usedInfo)
}

Expand Down

0 comments on commit 514bf56

Please sign in to comment.