YARN-11982: Automatically stage weight capacity when dynamic queue creation on root queue is enabled

[Hean-Chhinling]

Contains content generated by Cursor.

Description of PR

Automatically stage 1w capacity for root queue when Dynamic Queue Creation is enabled.
There is no way to set capacity for root queue because the "Edit Capacity" button is disable, therefore it is not possible to enable the Dynamic Queue Creation on root queue.

The main change in this PR is to pass to 1w and 100% capacity when Dynamic Queue Creation is enabled or disable, respectively. This is implemented at rootFlexibleAutoCreation.ts.

Notes: the automatically stage weight capacity only works when yarn.scheduler.capacity.legacy-queue-mode.enabled=true. If non-legacy queue mode, then there is no need to stage weight capacity.

How was this patch tested?

Tested manually and works as expected.

Automatically stage 1w capacity for root queue when Dynamic Queue Creation is enabled:

If Dynamic Queue Creation is disable, then the root queue 100% will automatically be staged:

For code changes:

• Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: Have the integration tests been executed and the endpoint
  declared according to the connector-specific documentation? Note: Automated CI
  testing doesn't cover all cases so manual testing with cloud storage is still
  required.
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

• The PR includes the phrase "Contains content generated by "
  where is the name of the AI tool used.
• My use of AI contributions follows the ASF legal policy
  https://www.apache.org/legal/generative-tooling.html

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 23s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+1 💚	mvninstall	48m 52s		trunk passed
+1 💚	shadedclient	83m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 28s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	shadedclient	34m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 48s		The patch does not generate ASF License warnings.
		123m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/1/artifact/out/Dockerfile
GITHUB PR	#8675
Optional Tests	dupname asflicense shadedclient codespell detsecrets
uname	Linux 8d4e5a6aef22 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 1040d54
Max. process+thread count	571 (vs. ulimit of 10000)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/1/console
versions	git=2.43.0 maven=3.9.15
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+1 💚	mvninstall	48m 20s		trunk passed
+1 💚	shadedclient	84m 22s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	33m 46s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		123m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/2/artifact/out/Dockerfile
GITHUB PR	#8675
Optional Tests	dupname asflicense shadedclient codespell detsecrets
uname	Linux ad95ae979a5d 5.15.0-185-generic #195-Ubuntu SMP Fri Jun 19 17:11:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / e9dbf83
Max. process+thread count	577 (vs. ulimit of 10000)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/2/console
versions	git=2.43.0 maven=3.9.15
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

[K0K0V0K]

The capacity for the root queue cannot be configured, it is fixed to 100% (percentage mode).
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Based on the docs seems like this is not allowed. Or do i miss some thing?

[K0K0V0K]

Why this validation was not running during your test?

hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java

Line 607 in 7f49190

"Cannot set capacity, root queue has a fixed capacity of 100.0f");

[K0K0V0K]

I checked

<property>
  <name>yarn.scheduler.capacity.root.capacity</name>
  <value>3w</value>
</property>

on a cluster and seems working.

[Hean-Chhinling]

The capacity for the root queue cannot be configured, it is fixed to 100% (percentage mode).
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Based on the docs seems like this is not allowed. Or do i miss some thing?

Of course, the root queue capacity should be 100%. And if the user tries to set it to be different than 100% than the /validate endpoint will fail as I stated in the PR's description.

But we allowing this mostly for the weight and also for absolute mode. So that user could enable the Dynamic Child Queue creation for "root" queue.
For more details, please check the jira description:
https://issues.apache.org/jira/browse/YARN-11982

[Hean-Chhinling]

Why this validation was not running during your test?

hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java

Line 607 in 7f49190

"Cannot set capacity, root queue has a fixed capacity of 100.0f");

Hmm..I don't think we are hitting this exception. The root queue is always 100% by itself.
This is to editCapacity for root queue in weight and absolute mode.

[brumi1024]

@Hean-Chhinling thanks for the patch! In general root will always have 100% capacity (regardless of if it's in weight, absolute resources or capacity), hence I wouldn't enable the edit capacity button, it could just cause confusion. Instead I think a more user-friendly solution would be to when Flexible Queue Auto-Creation is enabled on root, a 100%->1w change on root's capacity is auto-staged. It should be simpler in code as well.

[Hean-Chhinling]

@Hean-Chhinling thanks for the patch! In general root will always have 100% capacity (regardless of if it's in weight, absolute resources or capacity), hence I wouldn't enable the edit capacity button, it could just cause confusion. Instead I think a more user-friendly solution would be to when Flexible Queue Auto-Creation is enabled on root, a 100%->1w change on root's capacity is auto-staged. It should be simpler in code as well.

Thanks you for review @brumi1024. In that case, I will implement the following:

• Enable the Dynamic Queue Creation for root queue
• Automatically stage root queue from 100% -> 1w

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	18m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+1 💚	mvninstall	50m 6s		trunk passed
+1 💚	shadedclient	85m 8s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 38s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	33m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		141m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/3/artifact/out/Dockerfile
GITHUB PR	#8675
Optional Tests	dupname asflicense shadedclient codespell detsecrets
uname	Linux 44eec9e5dc7e 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 7ab4e55
Max. process+thread count	575 (vs. ulimit of 10000)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/3/console
versions	git=2.43.0 maven=3.9.15
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	18m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 30s		trunk passed
+1 💚	shadedclient	73m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	27m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		123m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/4/artifact/out/Dockerfile
GITHUB PR	#8675
Optional Tests	dupname asflicense shadedclient codespell detsecrets
uname	Linux 0338693c592d 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / b9176b1
Max. process+thread count	641 (vs. ulimit of 10000)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8675/4/console
versions	git=2.43.0 maven=3.9.15
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.