Skip to content

Commit

Permalink
[SPARK-4158] Fix for missing resources.
Browse files Browse the repository at this point in the history
Mesos offers may not contain all resources, and Spark needs to check to
ensure they are present and sufficient.  Spark may throw an erroneous
exception when resources aren't present.

Author: Brenden Matthews <brenden@diddyinc.com>

Closes #3024 from brndnmtthws/fix-mesos-resource-misuse and squashes the following commits:

e5f9580 [Brenden Matthews] [SPARK-4158] Fix for missing resources.

(cherry picked from commit cb0eae3)
Signed-off-by: Andrew Or <andrew@databricks.com>
  • Loading branch information
brndnmtthws authored and Andrew Or committed Nov 6, 2014
1 parent 590a943 commit c58c1bb
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 4 deletions.
Expand Up @@ -235,8 +235,7 @@ private[spark] class CoarseMesosSchedulerBackend(
for (r <- res if r.getName == name) {
return r.getScalar.getValue
}
// If we reached here, no resource with the required name was present
throw new IllegalArgumentException("No resource called " + name + " in " + res)
0
}

/** Build a Mesos resource protobuf object */
Expand Down
Expand Up @@ -273,8 +273,7 @@ private[spark] class MesosSchedulerBackend(
for (r <- res if r.getName == name) {
return r.getScalar.getValue
}
// If we reached here, no resource with the required name was present
throw new IllegalArgumentException("No resource called " + name + " in " + res)
0
}

/** Turn a Spark TaskDescription into a Mesos task */
Expand Down

0 comments on commit c58c1bb

Please sign in to comment.