Provide a way to override the size of the executor thread pool

[eric]

Right now, it appears that the thread pool size is set to the number of cores.

I have experienced situations where I am seeing timeouts from clients but are not maxing the CPU on the system and would like to experiment with increasing the size of the thread pool, but I couldn't find any obvious way to do it.

[pyr]

Hi @eric,

You can override the netty initializer by providing an :initializer key when starting a tcp-server.
In this initializer you can set an event-executor of your choosing.

This would look like:

(def my-event-executor
    (DefaultEventExecutorGroup. number-of-threads-i-want))

(defn my-initializer []
  (channel-initializer
               int32-frame-decoder  (int32-frame-decoder)
               ^:shared int32-frame-encoder  (int32-frame-encoder)
               ^:shared protobuf-decoder     (protobuf-decoder)
               ^:shared protobuf-encoder     (protobuf-encoder)
               ^:shared msg-decoder          (msg-decoder)
               ^:shared msg-encoder          (msg-encoder)
               ^{:shared true :executor my-event-executor} handler
       (gen-tcp-handler core stats channel-group tcp-handler))))

If this proves to be improving your use case we can add utility functions to build initializers without so much hurdles.

[eric]

Unfortunately this has gone beyond my ability to reason out how to use this in my riemann.config.

Would you be able to give me an example of how I would actually use this in my riemann.config to make riemann use the number of threads I want?

[pyr]

This would be the full config bits:

(import 'io.netty.util.concurrent.DefaultEventExecutorGroup)
(use 'riemann.transport.tcp)
(use 'riemann.transport)

(def my-event-executor
    (DefaultEventExecutorGroup. number-of-threads-i-want))

(defn my-initializer []
  (channel-initializer
               int32-frame-decoder  (int32-frame-decoder)
               ^:shared int32-frame-encoder  (int32-frame-encoder)
               ^:shared protobuf-decoder     (protobuf-decoder)
               ^:shared protobuf-encoder     (protobuf-encoder)
               ^:shared msg-decoder          (msg-decoder)
               ^:shared msg-encoder          (msg-encoder)
               ^{:shared true :executor my-event-executor} handler
       (gen-tcp-handler core stats channel-group tcp-handler))))

(tcp-server {:intializer (my-initializer)})