Accessing HelixManager ExecutorService

[9aman]

Context
This is about the use of Helix in Apache Pinot. I want to infer the queue size of the executor service that handles state transitions.

2024/05/09 03:46:55.385 INFO [HelixTaskExecutor] [ZkClient-EventThread-137-pinot-zookeeper-headless:2181] Submit task: 755de162-49a1-4092-8e11-945c8e5e7674 to pool: java.util.concurrent.ThreadPoolExecutor@18623363[Running, pool size = 40, active threads = 40, queued tasks = 160, completed tasks = 33065]

Above are the logs emitted by Helix. These logs contain the details of the executor service used for the state transition messages.

On analysis of the code, I found that the following line in the HelixTaskExecutor class provides the above details:
LOG.info("Submit task: " + taskId + " to pool: " + exeSvc);

Is your feature request related to a problem? Please describe.

1. I wanted to access the helix task executor queue size.
2. The _executorMap inside HelixTaskExecutor cannot be accessed.
3. This limits our access to the executor service.

Is there a way to get the queue size?

Describe the solution you'd like
Ability to fetch the queue size of the executor service that handles state transition requests inside HelixTaskExecutor.

[junkaixue]

@9aman is this for programming purpose? The log already contains the info.

[9aman]

@9aman is this for programming purpose? The log already contains the info.

Yes, I want to access the queue size. I couldn't find a way to see the queue size to gauge the current load or state transition requests waiting to be processed.

[9aman]

@junkaixue is there a way to access this information?

[junkaixue]

Yeah. Unfortunately, we did not expose these info. We can have this ticket to track the work to expose the stats. But not feel comfortable to give user accessing of the map.

@9aman

[9aman]

@junkaixue what other ways are there for accessing such information if we don't want to expose this field to the user?
The queue size information can help us decipher whether we are running into any deadlocks (during state transition) or too many messages being generated and blocked.

Is there a better way of achieving this?

[junkaixue]

Yeah. There is another way that Helix accept user passed in executor service. If you would like to have the fully control, you can define your own executor service as set the corresponding state transition for the executor service registered in Helix. Then you can get the stats or anything at any time. @9aman