Should we remove the listener thread pool?

[jasontedor]

Previously this thread pool was primarily used in the transport client to ensure that listeners are not called back on network threads.

We also adopted the use of this thread pool for global checkpoint listeners, which are callbacks that are executed at the shard level when the global checkpoint for the shard increases above a per-listener defined value.

With the removal of the transport client, this leaves global checkpoint listeners as the only use of the listener thread pool.

I would love to remove another thread pool from Elasticsearch.

Should we remove the listener thread pool from Elasticsearch? This would be a breaking change for any user that has manually configured the size of this thread pool. That would be unexpected, tuning this thread pool should not be necessary:

• its only use in the server is for global checkpoint listeners
• global checkpoint listeners are only registered when CCR is in use
• its size should not be a limiting factor in any functionality on CCR since typically there would only be one listener outstanding per shard, and these listeners timeout after 30s

Therefore, while this would be a breaking change, we expect the impact on the user base to be extremely minimal.

• remove the use of the listener thread pool for refresh listeners, use the calling thread Notify refresh listeners on the calling thread #53259
• use a listener supplied thread pool for global checkpoint listeners Use given executor for global checkpoint listener #53260
• switch CCR to use the CCR thread pool instead of the listener thread pool for global checkpoint listeners Fork CCR checkpoint listeners on CCR thread pool #53265
• remove dispatching listenable action future that forks to the listener thread pool Drop action future that forks on listener executor #53261
• deprecate the listener thread pool in 7.x Deprecate the listener thread pool #53266
• remove the listener thread pool in master Remove the listener thread pool #53314

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Core)

[jasontedor]

I missed an additional use of this thread pool which is that refresh listeners are called back on this thread pool.

We discussed this during the distributed team sync and agreed that we should aim for reducing the number of thread pools in Elasticsearch and settled on the following plan:

• use the CCR thread pool for global checkpoint listeners that are queued from CCR
• use the calling thread for refresh listeners, as those callbacks are lightweight

[jaymode]

Should we remove the listener thread pool from Elasticsearch?

+1

Since this issue is still labeled team discuss and it was discussed during the distributed sync, is there something specific that needs further discussion?

[jasontedor]

Since this issue is still labeled team discuss and it was discussed during the distributed sync, is there something specific that needs further discussion?

I've removed the label. 🙂

[jasontedor]

@original-brownbear Thanks for being my review buddy on this one, and your helpful comments on the PRs!