Skip to content

Commit

Permalink
make isolationscheduler kill workers for all non-isolated topologies …
Browse files Browse the repository at this point in the history
…in case of squeezed cluster to make room for remaining isolated topology workers
  • Loading branch information
nathanmarz committed Feb 1, 2013
1 parent 8935c38 commit 4859f0d
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 1 deletion.
8 changes: 7 additions & 1 deletion src/clj/backtype/storm/scheduler/IsolationScheduler.clj
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,13 @@
allocated-topologies
(leftover-topologies topologies <>)
(DefaultScheduler/default-schedule <> cluster))
(log-warn "Unable to isolate topologies " (pr-str failed-iso-topologies) ". Will wait for enough resources for isolated topologies before allocating any other resources.")
(do
(log-warn "Unable to isolate topologies " (pr-str failed-iso-topologies) ". No machine had enough worker slots to run the remaining workers for these topologies. Clearing all other resources and will wait for enough resources for isolated topologies before allocating any other resources.")
;; clear workers off all hosts that are not blacklisted
(doseq [[host slots] (host->used-slots cluster)]
(if-not (.isBlacklistedHost cluster host)
(.freeSlots cluster slots)
)))
))
(.setBlacklistedHosts cluster orig-blacklist)
))
4 changes: 4 additions & 0 deletions src/jvm/backtype/storm/scheduler/Cluster.java
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,10 @@ public void blacklistHost(String host) {
public boolean isBlackListed(String supervisorId) {
return blackListedHosts != null && blackListedHosts.contains(getHost(supervisorId));
}

public boolean isBlacklistedHost(String host) {
return blackListedHosts != null && blackListedHosts.contains(host);
}

public String getHost(String supervisorId) {
return inimbus.getHostName(supervisors, supervisorId);
Expand Down

0 comments on commit 4859f0d

Please sign in to comment.