From 52494b5c1a0d3ddbefe13eea5157f4d41b68dea5 Mon Sep 17 00:00:00 2001 From: "Robert (Bobby) Evans" Date: Thu, 12 Nov 2015 10:19:37 -0600 Subject: [PATCH 1/4] Added in Empty File --- STORM_JSTORM_FEATURE_DIFF.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 STORM_JSTORM_FEATURE_DIFF.md diff --git a/STORM_JSTORM_FEATURE_DIFF.md b/STORM_JSTORM_FEATURE_DIFF.md new file mode 100644 index 00000000000..e69de29bb2d From 99442c4168ed727d4b608a4e39e7de5a0437c68c Mon Sep 17 00:00:00 2001 From: "Robert (Bobby) Evans" Date: Thu, 12 Nov 2015 10:58:19 -0600 Subject: [PATCH 2/4] Update STORM_JSTORM_FEATURE_DIFF.md --- STORM_JSTORM_FEATURE_DIFF.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/STORM_JSTORM_FEATURE_DIFF.md b/STORM_JSTORM_FEATURE_DIFF.md index e69de29bb2d..c453eabbad3 100644 --- a/STORM_JSTORM_FEATURE_DIFF.md +++ b/STORM_JSTORM_FEATURE_DIFF.md @@ -0,0 +1,30 @@ + feature | Storm-0.11.0-SNAPSHOT (and expected pull requests) | JStorm-2.1.0 | Notes | JIRA | +----------|-----------------------|--------------|-------|----| + Security | Basic Authentication/Authorization | N/A | The migration is not high risk. But JStorm daeons/connections need to be evaluated +Scheduler | Resource aware scheduling (Work In Progress) |
  1. Make the tasks of same component distributed evenly across the nodes in cluster.
  2. Make number of tasks in a worker is balanced.
  3. Try to assign two tasks which are transferring message directly into a same worker to reduce network cost.
  4. Support user defined assignment and to use the result of last assignment Different solution between Storm and JStorm.
Actually, JStorm has ever used cpu/mem/disk/network demond for scheduling. But we faced the problem of low utilization of cluster. For example, because the configuration of topologys and resources of hardwares in a cluster are different. After running a period, it is possible that no topology will be assigned to the node with enough other resources but just lacking one resource. | Scheduler interface is pluggable so we should be able to support boht schedulers for the time being | +Nimbus HA | Support to configure more than one spare nimbus. When the master nimbus is down, a most appropriate spare nimbus(topologys in disk mostly matches the recoreds in zk) will be chosen to be promoted. | Different solution. | | +Topology structure | worker/executor/task | worker/task | It might be the biggest diff between Storm and JStorm. So, not sure if there are some features related to "executor" can not be merged. (Rebalancing allows us to combine multiple tasks on a single executor) This may go away in the future once RAS is stable | +Topology Master | (Heartbeat server is similar for scaling on a pull request) | New system bolt "topology master" was added, which is responsible for collecting task heartbeat info of all tasks and reports to nimbus. Besides task hb info, it also can be used to dispatch control messages in topology. Topology master significantly reduce the amout of read/write to zk. Before this change, zk is the bottleneck to depoly big cluster and topology. +Backpressure | Store backpressure status in zk which will trigger the source spout to start flow control. When flow control, relative souce spout will stop sending tuples. |
  1. Implement backpressure based on topology master. TM is responsible to process the trigger message and send flow control request to relative spouts. When flow control, spout will not stop to send tuples, but just slow down the sending.
  2. Enable user to update the configuration of backpressure dynamically without restarting topology, e.g. enable/disable backpressue, high/low water mark... | These two solution is totally different. We need to evaluate which one is better. | +Monitor of task execute thread | N/A | Monitor the status of the execute thread of task. It is effective to find the slow bolt in a topology, and potential dead lock. | +Message processing | Configurable multi receiving threads of worker (But likely to change to have deserialization happen in Netty) |
    1. Add receiving and transferring queue/thread for each task to make deserialization and serialization asynchronously
    2. Remove receiving and transferring thread on worker level to avoid unnecessary locks and to shorten the message processing phase | Low risk for merging to JStorm (Both implementations are similar) | +Batch tuples | Batching in DisruptorQueue | Do batch before sending tuple to transfer queue and support to adjust the batch size dynamically according to samples of actual batch size sent out for past intervals. | Should evaluate both implementations, and see which is better, or if we can pull in some of the dynamic batching to the disruptor level | +Grouping | Load aware balancing shuffle grouping |
      1. Add localfrist grouping that tuples are sent to the tasks in the same worker firstly. If the load of all local tasks is high, the tuples will be sent out to remote tasks.
      2. Improve localOrShuffle grouping that tuples are only sent to the tasks in the same worker or in same node. The shuffle of JStorm checks the load of network connection of local worker. | The risk for merging of checking remote load which was implemented in "Load aware balancing shuffle grouping" is low. +Rest API | YES | NA |The porting of rest API to JStorm is in progress +web-UI | |
        1. Redesign Web-UI, the new Web-UI code is clear and clean.
        2. Data display is much more beautiful and better user experience, especially topology graph and 30 minutes metric trend. url: http://storm.taobao.org/ +metric system | |
          1. All levels of metrics, including stream metrics, task metrics, component metrics, topology metrics, even cluster metrics, will be sampled & calculated. Some metrics, e.g. ""tuple life cycle"", are very useful for debug and finding hot spot of a topology.
          2. Support full metrics data. previous metric system can only display mean value of meters/histograms, the new metric system can display m1, m5, m15 of meters, and common percentiles of histograms.
          3. Use new metrics windows, the min metric window is 1min, thus we can see the metircs data every single minute.
          4. Supplies a metric uploader interface, thirty-party companies can easily build their own metric systems based on all historic metric data. | JStorm metric system is completely redesigned. +jar command | Support progress bar when submitting topology | Does not support progress bar | Low risk for merging to JStorm +rebalance command | Basic functionality of rebalance | Besides rebalance, scale-out/in by updating the number of worker, acker, spout & bolt dynamically without stopping topology | How is grouping consistency maintined when changing the number or bolt/spout? | +list command | List information of topologys| List information of all topologys, all supervisors and jstorm version | +zktool command | N/A | Support user to do some zk operation in remote node, e.g. "list", "read"… | will need to be avaluated for security | +metricsMonitor command | monitor is similar, but read only | Switch on/off some metrics which may impact topology performance | +restart command | N/A | Restart a topology. Besides restart, this commond can also update the configuration of topology. | +update_topology command | N/A | Update jars and configuration dynamically for a topology, without stopping topology | +shell command | | N/A | Low risk for merging to JStorm +ui command | | N/A | Low risk for merging to JStorm +logviewer command | | N/A | Low risk for merging to JStorm +repl command | | N/A | Not need to merge this for java core +cgroup | N/A | Support the upper limit control of cpu core usage for a worker by cgroup | +Log |
            1. Support user difined log configuration
            2. support dynamic changes to logging of running topology
            |
            1. Support user defined configuration of log
            2. Support both logback and log4j
            | +Worker classloader | N/A uses shading for most dependencies | Avoid problem of re-loading classes | +blobstore (comming soon) | Distributed cache like interface with dynamic updating without bringing down a topology | N/A | From 71bd0b9ca554ba70aa56634d74f258d0fdf392c8 Mon Sep 17 00:00:00 2001 From: "Robert (Bobby) Evans" Date: Fri, 13 Nov 2015 08:27:47 -0600 Subject: [PATCH 3/4] Addressed some review comments and feedback --- STORM_JSTORM_FEATURE_DIFF.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/STORM_JSTORM_FEATURE_DIFF.md b/STORM_JSTORM_FEATURE_DIFF.md index c453eabbad3..5b8da9d9ec0 100644 --- a/STORM_JSTORM_FEATURE_DIFF.md +++ b/STORM_JSTORM_FEATURE_DIFF.md @@ -2,7 +2,7 @@ ----------|-----------------------|--------------|-------|----| Security | Basic Authentication/Authorization | N/A | The migration is not high risk. But JStorm daeons/connections need to be evaluated Scheduler | Resource aware scheduling (Work In Progress) |
            1. Make the tasks of same component distributed evenly across the nodes in cluster.
            2. Make number of tasks in a worker is balanced.
            3. Try to assign two tasks which are transferring message directly into a same worker to reduce network cost.
            4. Support user defined assignment and to use the result of last assignment Different solution between Storm and JStorm.
            Actually, JStorm has ever used cpu/mem/disk/network demond for scheduling. But we faced the problem of low utilization of cluster. For example, because the configuration of topologys and resources of hardwares in a cluster are different. After running a period, it is possible that no topology will be assigned to the node with enough other resources but just lacking one resource. | Scheduler interface is pluggable so we should be able to support boht schedulers for the time being | -Nimbus HA | Support to configure more than one spare nimbus. When the master nimbus is down, a most appropriate spare nimbus(topologys in disk mostly matches the recoreds in zk) will be chosen to be promoted. | Different solution. | | +Nimbus HA | Support for a pool of nimbus servers. Once Blobstore is merged in leader election and state storage will be separate. | Support to configure more than one spare nimbus. When the master nimbus is down, a most appropriate spare nimbus(topologys in disk mostly matches the recoreds in zk) will be chosen to be promoted. | Different solution. | | Topology structure | worker/executor/task | worker/task | It might be the biggest diff between Storm and JStorm. So, not sure if there are some features related to "executor" can not be merged. (Rebalancing allows us to combine multiple tasks on a single executor) This may go away in the future once RAS is stable | Topology Master | (Heartbeat server is similar for scaling on a pull request) | New system bolt "topology master" was added, which is responsible for collecting task heartbeat info of all tasks and reports to nimbus. Besides task hb info, it also can be used to dispatch control messages in topology. Topology master significantly reduce the amout of read/write to zk. Before this change, zk is the bottleneck to depoly big cluster and topology. Backpressure | Store backpressure status in zk which will trigger the source spout to start flow control. When flow control, relative souce spout will stop sending tuples. |
            1. Implement backpressure based on topology master. TM is responsible to process the trigger message and send flow control request to relative spouts. When flow control, spout will not stop to send tuples, but just slow down the sending.
            2. Enable user to update the configuration of backpressure dynamically without restarting topology, e.g. enable/disable backpressue, high/low water mark... | These two solution is totally different. We need to evaluate which one is better. | @@ -14,7 +14,7 @@ Rest API | YES | NA |The porting of rest API to JStorm is in progress web-UI | |
              1. Redesign Web-UI, the new Web-UI code is clear and clean.
              2. Data display is much more beautiful and better user experience, especially topology graph and 30 minutes metric trend. url: http://storm.taobao.org/ metric system | |
                1. All levels of metrics, including stream metrics, task metrics, component metrics, topology metrics, even cluster metrics, will be sampled & calculated. Some metrics, e.g. ""tuple life cycle"", are very useful for debug and finding hot spot of a topology.
                2. Support full metrics data. previous metric system can only display mean value of meters/histograms, the new metric system can display m1, m5, m15 of meters, and common percentiles of histograms.
                3. Use new metrics windows, the min metric window is 1min, thus we can see the metircs data every single minute.
                4. Supplies a metric uploader interface, thirty-party companies can easily build their own metric systems based on all historic metric data. | JStorm metric system is completely redesigned. jar command | Support progress bar when submitting topology | Does not support progress bar | Low risk for merging to JStorm -rebalance command | Basic functionality of rebalance | Besides rebalance, scale-out/in by updating the number of worker, acker, spout & bolt dynamically without stopping topology | How is grouping consistency maintined when changing the number or bolt/spout? | +rebalance command | Basic functionality of rebalance | Besides rebalance, scale-out/in by updating the number of worker, acker, spout & bolt dynamically without stopping topology. Routing is updated dynamically within upstream components. | dynamic routing with some groupings is difficult to get right with state, we need to be sure this is well documented, and might want to disallow it for some groupings without a force flag. | list command | List information of topologys| List information of all topologys, all supervisors and jstorm version | zktool command | N/A | Support user to do some zk operation in remote node, e.g. "list", "read"… | will need to be avaluated for security | metricsMonitor command | monitor is similar, but read only | Switch on/off some metrics which may impact topology performance | From 3ad9de5347c598bdcfd7fb03c1029dc89e692b55 Mon Sep 17 00:00:00 2001 From: Erik Weathers Date: Sat, 14 Nov 2015 14:56:34 -0800 Subject: [PATCH 4/4] fix spelling & grammar mistakes in Storm/JStorm diff table --- STORM_JSTORM_FEATURE_DIFF.md | 60 ++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/STORM_JSTORM_FEATURE_DIFF.md b/STORM_JSTORM_FEATURE_DIFF.md index 5b8da9d9ec0..4bbe2161f4f 100644 --- a/STORM_JSTORM_FEATURE_DIFF.md +++ b/STORM_JSTORM_FEATURE_DIFF.md @@ -1,30 +1,30 @@ - feature | Storm-0.11.0-SNAPSHOT (and expected pull requests) | JStorm-2.1.0 | Notes | JIRA | -----------|-----------------------|--------------|-------|----| - Security | Basic Authentication/Authorization | N/A | The migration is not high risk. But JStorm daeons/connections need to be evaluated -Scheduler | Resource aware scheduling (Work In Progress) |
                  1. Make the tasks of same component distributed evenly across the nodes in cluster.
                  2. Make number of tasks in a worker is balanced.
                  3. Try to assign two tasks which are transferring message directly into a same worker to reduce network cost.
                  4. Support user defined assignment and to use the result of last assignment Different solution between Storm and JStorm.
                  Actually, JStorm has ever used cpu/mem/disk/network demond for scheduling. But we faced the problem of low utilization of cluster. For example, because the configuration of topologys and resources of hardwares in a cluster are different. After running a period, it is possible that no topology will be assigned to the node with enough other resources but just lacking one resource. | Scheduler interface is pluggable so we should be able to support boht schedulers for the time being | -Nimbus HA | Support for a pool of nimbus servers. Once Blobstore is merged in leader election and state storage will be separate. | Support to configure more than one spare nimbus. When the master nimbus is down, a most appropriate spare nimbus(topologys in disk mostly matches the recoreds in zk) will be chosen to be promoted. | Different solution. | | -Topology structure | worker/executor/task | worker/task | It might be the biggest diff between Storm and JStorm. So, not sure if there are some features related to "executor" can not be merged. (Rebalancing allows us to combine multiple tasks on a single executor) This may go away in the future once RAS is stable | -Topology Master | (Heartbeat server is similar for scaling on a pull request) | New system bolt "topology master" was added, which is responsible for collecting task heartbeat info of all tasks and reports to nimbus. Besides task hb info, it also can be used to dispatch control messages in topology. Topology master significantly reduce the amout of read/write to zk. Before this change, zk is the bottleneck to depoly big cluster and topology. -Backpressure | Store backpressure status in zk which will trigger the source spout to start flow control. When flow control, relative souce spout will stop sending tuples. |
                  1. Implement backpressure based on topology master. TM is responsible to process the trigger message and send flow control request to relative spouts. When flow control, spout will not stop to send tuples, but just slow down the sending.
                  2. Enable user to update the configuration of backpressure dynamically without restarting topology, e.g. enable/disable backpressue, high/low water mark... | These two solution is totally different. We need to evaluate which one is better. | -Monitor of task execute thread | N/A | Monitor the status of the execute thread of task. It is effective to find the slow bolt in a topology, and potential dead lock. | -Message processing | Configurable multi receiving threads of worker (But likely to change to have deserialization happen in Netty) |
                    1. Add receiving and transferring queue/thread for each task to make deserialization and serialization asynchronously
                    2. Remove receiving and transferring thread on worker level to avoid unnecessary locks and to shorten the message processing phase | Low risk for merging to JStorm (Both implementations are similar) | -Batch tuples | Batching in DisruptorQueue | Do batch before sending tuple to transfer queue and support to adjust the batch size dynamically according to samples of actual batch size sent out for past intervals. | Should evaluate both implementations, and see which is better, or if we can pull in some of the dynamic batching to the disruptor level | -Grouping | Load aware balancing shuffle grouping |
                      1. Add localfrist grouping that tuples are sent to the tasks in the same worker firstly. If the load of all local tasks is high, the tuples will be sent out to remote tasks.
                      2. Improve localOrShuffle grouping that tuples are only sent to the tasks in the same worker or in same node. The shuffle of JStorm checks the load of network connection of local worker. | The risk for merging of checking remote load which was implemented in "Load aware balancing shuffle grouping" is low. -Rest API | YES | NA |The porting of rest API to JStorm is in progress -web-UI | |
                        1. Redesign Web-UI, the new Web-UI code is clear and clean.
                        2. Data display is much more beautiful and better user experience, especially topology graph and 30 minutes metric trend. url: http://storm.taobao.org/ -metric system | |
                          1. All levels of metrics, including stream metrics, task metrics, component metrics, topology metrics, even cluster metrics, will be sampled & calculated. Some metrics, e.g. ""tuple life cycle"", are very useful for debug and finding hot spot of a topology.
                          2. Support full metrics data. previous metric system can only display mean value of meters/histograms, the new metric system can display m1, m5, m15 of meters, and common percentiles of histograms.
                          3. Use new metrics windows, the min metric window is 1min, thus we can see the metircs data every single minute.
                          4. Supplies a metric uploader interface, thirty-party companies can easily build their own metric systems based on all historic metric data. | JStorm metric system is completely redesigned. -jar command | Support progress bar when submitting topology | Does not support progress bar | Low risk for merging to JStorm -rebalance command | Basic functionality of rebalance | Besides rebalance, scale-out/in by updating the number of worker, acker, spout & bolt dynamically without stopping topology. Routing is updated dynamically within upstream components. | dynamic routing with some groupings is difficult to get right with state, we need to be sure this is well documented, and might want to disallow it for some groupings without a force flag. | -list command | List information of topologys| List information of all topologys, all supervisors and jstorm version | -zktool command | N/A | Support user to do some zk operation in remote node, e.g. "list", "read"… | will need to be avaluated for security | -metricsMonitor command | monitor is similar, but read only | Switch on/off some metrics which may impact topology performance | -restart command | N/A | Restart a topology. Besides restart, this commond can also update the configuration of topology. | -update_topology command | N/A | Update jars and configuration dynamically for a topology, without stopping topology | -shell command | | N/A | Low risk for merging to JStorm -ui command | | N/A | Low risk for merging to JStorm -logviewer command | | N/A | Low risk for merging to JStorm -repl command | | N/A | Not need to merge this for java core -cgroup | N/A | Support the upper limit control of cpu core usage for a worker by cgroup | -Log |
                            1. Support user difined log configuration
                            2. support dynamic changes to logging of running topology
                            |
                            1. Support user defined configuration of log
                            2. Support both logback and log4j
                            | -Worker classloader | N/A uses shading for most dependencies | Avoid problem of re-loading classes | -blobstore (comming soon) | Distributed cache like interface with dynamic updating without bringing down a topology | N/A | + feature | Storm-0.11.0-SNAPSHOT (and expected pull requests) | JStorm-2.1.0 | Notes | JIRA | +----------|-----------------------|--------------|-------|----| + Security | Basic Authentication/Authorization | N/A | The migration is not high risk. But JStorm daemons/connections need to be evaluated +Scheduler | Resource aware scheduling (RAS) (Work In Progress) |
                            1. Evenly distribute a component's tasks across the nodes in the cluster.
                            2. Balanace the number of tasks in a worker.
                            3. Try to assign two tasks which are transferring messages directly into the same worker to reduce network cost.
                            4. Support user-defined assignment and using the result of the last assignment. Different solution between Storm and JStorm.
                            Actually, JStorm has always used the cpu/mem/disk/network demand for scheduling. But we faced problems with low utilization of cluster resources, arising from the configuration of topologies differing from the hardware resources in a cluster. After running for a period, it is possible that a node will satisfy almost all of the resource demands, but lack 1 resource (e.g., having 4 CPUs but the topology needing 5 CPUs). | The scheduler interface is pluggable, so we should be able to support both schedulers for the time being. | +Nimbus HA | Support for a pool of nimbus servers. Once Blobstore is merged in, "leader election" and "state storage" will be separate. | Support for configuring more than one backup nimbus. When the master nimbus is down, the most appropriate spare nimbus (topologies on disk most closely match the records in ZooKeeper) will be chosen to be promoted. | Different solution.| | +Topology structure | worker/executor/task | worker/task | This might be the biggest difference between Storm and JStorm. Not sure if there are some features related to "executor" that cannot be merged. (Rebalancing allows JStorm to combine multiple tasks on a single executor) This may go away in the future once RAS is stable. | +Topology Master | [The Heartbeat Server (Pacemaker)](https://issues.apache.org/jira/browse/STORM-885) is similar for scaling. [The pacemaker change is under review](https://github.com/apache/storm/pull/838) | New system bolt "topology master" was added, which is responsible for collecting task heartbeat info of all tasks and reporting the info to nimbus. Besides task heartbeat info, it also can be used to dispatch control messages within the topology. Topology master significantly reduces the amout of read/write to ZooKeeper. Before this change, ZooKeeper was the bottleneck for deploying big clusters and topologies. +Backpressure | Store backpressure status in ZooKeeper, which can trigger the source spout to start "flow control", meaning it will stop sending tuples. |
                            1. Implement backpressure using "topology master" (TM). TM is responsible for processing the trigger message and sending the flow control request to relevant spouts. "flow control" in JStorm doesn't complete stop the spout from emitting tuples, but instead just slows down the tuple sending.
                            2. User can update the configuration of backpressure dynamically without restarting topology, e.g. enable/disable backpressue, high/low watermark, etc. | These two solutions are totally different, we need to evaluate which one is better. | +Monitoring of task execute thread | N/A | Monitors the status of the execute thread of tasks. It is effective to find the slow bolt in a topology, and potentially uncovers deadlock as well. | +Message processing | Configurable multi-receiving threads of worker (but likely to change to have deserialization happen in Netty) |
                              1. Add receiving and transferring queue/thread for each task to make deserialization and serialization asynchronously
                              2. Remove receiving and transferring thread on worker level to avoid unnecessary locks and to shorten the message processing phase | Low risk for merging to JStorm (Both implementations are similar) | +Batch tuples | Batching in DisruptorQueue | Do batch before sending tuple to transfer queue and support for adjusting the batch size dynamically according to samples of actual batch size sent out for past intervals. | Should evaluate both implementations, and see which is better, or whether we can move some of the dynamic batching logic into the disruptor level | +Grouping | Load aware balancing shuffle grouping |
                                1. Has a "localfirst" grouping that causes tuples to be sent to the tasks in the same worker by default. But if the load of all local tasks is high, the tuples will be sent out to remote tasks.
                                2. Improve localOrShuffle grouping from Storm. In Storm's localOrShuffle grouping the definition of "local" is local within the same worker process. i.e., if there is a bolt that the component can send to in the current worker process it will send the tuples there. If there is not one, it will do round robin between *all* of the instances of that bolt no matter which hosts they are on. JStorm has extended that so that other workers/JVMs on the same host are considered "local" as well, taking into account the load of the network connections on the local worker. | The risk for merging of checking remote load which was implemented in "Load aware balancing shuffle grouping" is low. +REST API | Yes | NA |The porting of REST API to JStorm is in progress +web-UI | Yes |
                                  1. Has a redesigned Web-UI with clear and clean code.
                                  2. Data display is much more beautiful and provides a better user experience, such as the topology graph and 30 minute metric trend. Example URL: http://storm.taobao.org/ +metric system | |
                                    1. All levels of metrics, including stream metrics, task metrics, component metrics, topology metrics, even cluster metrics, are sampled & calculated. Some metrics, e.g. ""tuple life cycle"", are very useful for debugging and finding the hotspots of a topology.
                                    2. Support full metrics data. Previous metric system can only display mean value of meters/histograms, the new metric system can display m1, m5, m15 of meters, and common percentiles of histograms.
                                    3. Use new metrics windows, the mininum metric window is 1 minute, thus we can see the metrics data every single minute.
                                    4. Supplies a metric uploader interface, third-party companies can easily build their own metric systems based on the historic metric data. | JStorm metric system is completely redesigned. +jar command | Supports progress bar when submitting topology | Does not support progress bar | Low risk for merging to JStorm +rebalance command | Basic functionality of rebalance | Besides rebalance, scale-out/in by updating the number of workers, ackers, spouts & bolts dynamically without stopping topology. Routing is updated dynamically within upstream components. | dynamic routing with some groupings is difficult to get right with state, we need to be sure this is well documented, and might want to disallow it for some groupings without a force flag. | +list command | List information of topologies| List information of all topologies, all supervisors, and JStorm version | +zktool command | N/A | Supports some ZooKeeper operations, e.g. "list", "read"… | will need to be evaluated for security | +metricsMonitor command | monitor is similar, but read-only | Allows toggling on/off some metrics which may impact topology performance | +restart command | N/A | Restart a topology. Besides restart, this commond can also update the topology configuration. | +update_topology command | N/A | Update jars and configuration dynamically for a topology, without stopping the topology. | +shell command | | N/A | Low risk for merging to JStorm +ui command | | N/A | Low risk for merging to JStorm +logviewer command | | N/A | Low risk for merging to JStorm +repl command | | N/A | No need to merge this for java core +cgroup | with storm-on-mesos framework, Storm can restrict the CPU and Memory available to a worker process | Supports controlling the upper limit of CPU core usage for a worker using cgroups | +Log |
                                      1. Supports user-defined log configuration via Log4j 2
                                      2. Supports dynamic changes to logging of a running topology
                                      |
                                      1. Supports user-defined configuration of log
                                      2. Supports both logback and log4j
                                      | +Worker classloader | N/A uses shading for most dependencies | The "worker classloader" avoids problem of re-loading classes | +blobstore (coming soon) | Distributed cache-like interface with dynamic updating without bringing down a topology | N/A |