KAFKA-20210: Add group.coordinator.background.threads config by squah-confluent · Pull Request #21555 · apache/kafka

squah-confluent · 2026-02-23T19:00:07Z

Add the group.coordinator.background.threads config option to control
the number of threads in the group coordinator's CoordinatorExecutor
thread pool.

Reviewers: Dongnuo Lyu dlyu@confluent.io, Chia-Ping Tsai
chia7712@gmail.com, PoAn Yang payang@apache.org, David Jacot
djacot@confluent.io

Add the group.coordinator.executor.threads config option to control the number of threads in the group coordinator's CoordinatorExecutor thread pool.

dajac · 2026-02-23T19:32:33Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorConfig.java

    public static final int GROUP_COORDINATOR_NUM_THREADS_DEFAULT = 4;

+    public static final String GROUP_COORDINATOR_NUM_EXECUTOR_THREADS_CONFIG = "group.coordinator.executor.threads";
+    public static final String GROUP_COORDINATOR_NUM_EXECUTOR_THREADS_DOC = "The number of executor threads used by the group coordinator.";


Should we explain what they are used for? We may also want to extend the doc of the runtime threads.

I updated the config docs. Let me know what you think!

dajac

Thanks for the patch, @squah-confluent. I left a few more comments for consideration.

dajac · 2026-02-24T07:44:09Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorConfig.java

+    public static final String GROUP_COORDINATOR_NUM_THREADS_DOC = "The number of threads used by the group coordinator for processing requests.";
    public static final int GROUP_COORDINATOR_NUM_THREADS_DEFAULT = 4;

+    public static final String GROUP_COORDINATOR_NUM_EXECUTOR_THREADS_CONFIG = "group.coordinator.executor.threads";


Thinking about it, I find the name of the config not very good, mainly because it is not clear why it does based on the name and it is not clear how it differs from group.coordinator.threads too. I wonder whether we should use something like group.coordinator.backgroup.threads. @squah-confluent @dongnuo123 Thoughts?

Should it be group.coordinator.background.threads?

I renamed the config.

dajac · 2026-02-24T07:46:49Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorConfig.java

+    public static final String GROUP_COORDINATOR_NUM_EXECUTOR_THREADS_DOC = "The number of executor threads used by the group coordinator for " +
+        "updating the list of topics for regex subscriptions and metadata changes and offloaded assignments.";


The number of executor threads used by the group coordinator for processing background/asynchronous tasks such as resolving regular expressions and computing assignments.

For the computing assignment part, we could mention the related config too when we add it.

dajac · 2026-02-24T07:47:43Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java

                    .withCompression(Compression.of(config.offsetTopicCompressionType()).build())
                    .withAppendLingerMs(config.appendLingerMs())
-                    .withExecutorService(Executors.newSingleThreadExecutor())
+                    .withExecutorService(Executors.newFixedThreadPool(config.numExecutorThreads()))


While we are here, would it be possible to give a proper name to the threads used here? We could follow the name we use for the config so administrators can connect the dots.

I've named the threads group-coordinator-background-<n>

dajac · 2026-02-26T11:13:41Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java

-                    .withExecutorService(Executors.newSingleThreadExecutor())
+                    .withExecutorService(Executors.newFixedThreadPool(
+                        config.numBackgroundThreads(),
+                        runnable -> new KafkaThread(


nit: It seems that we can now use Thread.ofPlatform().name("group-coordinator-background-", 0).factory() since Java 21. Does it work here?

I don't think we can, since our min supported Java version is 17

kafka/build.gradle

Line 48 in 3fd6017

minNonClientJavaVersion = 17

17 is for clients. We use a newer one on the server.

I thought 11 was for clients? I can't get it to compile locally unless I also bump the minNonClientJavaVersion to 21.

We can't use Thread.ofPlatform() because the server-side code targets JDK 17, and the client-side code is still on JDK 11.

maybe we could use ThreadUtils.createThreadFactory("group-coordinator-background-%d", false) instead?

My bad... I was sure that we were already on 21. Using ThreadUtils.createThreadFactory makes sense.

Thanks, I've switched to ThreadUtils.createThreadFactory.

dajac · 2026-02-26T11:16:11Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorConfig.java

+    public static final String GROUP_COORDINATOR_NUM_BACKGROUND_THREADS_DOC = "The number of background threads used by the group coordinator for " +
+        "updating the list of topics for regex subscriptions and metadata changes and offloaded assignments.";


nit: The number of threads used by the group coordinator for processing background tasks (e.g. regular expressions or assignments).? I don't get the metadata changes part in the current one.

That's a typo! I meant to write "on metadata changes". Updated the description.

chia7712 · 2026-03-02T08:29:03Z

group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorConfig.java

+    public static final String GROUP_COORDINATOR_NUM_THREADS_DOC = "The number of threads used by the group coordinator for processing requests.";
    public static final int GROUP_COORDINATOR_NUM_THREADS_DEFAULT = 4;

+    public static final String GROUP_COORDINATOR_NUM_BACKGROUND_THREADS_CONFIG = "group.coordinator.background.threads";


Should we add this new config to upgrade.md?

I am +1 for adding something but I would do it for the KIP, not for the config. Hence, I suggest to defer it to another PR. @squah-confluent Could you please file a ticket so we don't forget about it.

Filed as https://issues.apache.org/jira/browse/KAFKA-20240

This reverts commit 5c72195.

FrankYang0529

LGTM. Thanks for the fix.

dajac

lgtm

…21555) Add the group.coordinator.background.threads config option to control the number of threads in the group coordinator's CoordinatorExecutor thread pool. Reviewers: Dongnuo Lyu <dlyu@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>, PoAn Yang <payang@apache.org>, David Jacot <djacot@confluent.io>

KAFKA-20210: Add group.coordinator.executor.threads config

589f979

Add the group.coordinator.executor.threads config option to control the number of threads in the group coordinator's CoordinatorExecutor thread pool.

github-actions bot added triage PRs from the community core Kafka Broker group-coordinator small Small PRs labels Feb 23, 2026

dongnuo123 approved these changes Feb 23, 2026

View reviewed changes

dajac added ci-approved and removed triage PRs from the community labels Feb 23, 2026

dajac reviewed Feb 23, 2026

View reviewed changes

fixup: Update num threads config descriptions

382de31

dajac reviewed Feb 24, 2026

View reviewed changes

squah-confluent added 2 commits February 25, 2026 20:18

fixup: Rename executor.threads to background.threads

959eec6

fixup: Name background threads

b4ab21e

squah-confluent changed the title ~~KAFKA-20210: Add group.coordinator.executor.threads config~~ KAFKA-20210: Add group.coordinator.background.threads config Feb 25, 2026

dajac reviewed Feb 26, 2026

View reviewed changes

squah-confluent added 2 commits March 1, 2026 17:17

fixup: fix config description

8a9506f

fixup: use Thread.ofPlatform

5c72195

squah-confluent mentioned this pull request Mar 2, 2026

KAFKA-20211 [1/2]: Change CoordinatorRuntime executor type from ExecutorService to ThreadPoolExecutor #21610

Closed

chia7712 reviewed Mar 2, 2026

View reviewed changes

squah-confluent added 2 commits March 2, 2026 13:27

Revert "fixup: use Thread.ofPlatform"

3d466da

This reverts commit 5c72195.

fixup: use ThreadUtils.createThreadFactory

6e1cc38

FrankYang0529 approved these changes Mar 2, 2026

View reviewed changes

dajac approved these changes Mar 2, 2026

View reviewed changes

chia7712 approved these changes Mar 2, 2026

View reviewed changes

dajac merged commit 0653154 into apache:trunk Mar 3, 2026
36 of 39 checks passed

dajac deleted the squah-kip-1263-add-executor-threads branch March 3, 2026 06:48

		public static final String GROUP_COORDINATOR_NUM_EXECUTOR_THREADS_DOC = "The number of executor threads used by the group coordinator for " +
		"updating the list of topics for regex subscriptions and metadata changes and offloaded assignments.";

Conversation

squah-confluent commented Feb 23, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dajac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

squah-confluent Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FrankYang0529 left a comment

Choose a reason for hiding this comment

Uh oh!

dajac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

squah-confluent commented Feb 23, 2026 •

edited by github-actions bot

Loading

squah-confluent Feb 25, 2026 •

edited

Loading