CP-54473 Configure max-cstate by XAPI #6681

changlei-li · 2025-09-22T08:29:29Z

C-states are power management states for CPUs where higher numbered states represent deeper sleep modes with lower power consumption but higher wake-up latency. The max_cstate parameter controls the deepest C-state that CPUs are allowed to enter.

Common C-state values:

C0: CPU is active (not a sleep state)
C1: CPU is halted but can wake up almost instantly
C2: CPU caches are flushed, slightly longer wake-up time
C3+: Deeper sleep states with progressively longer wake-up times

To set max_cstate on dom0 host, two commands are used: xenpm to set at runtime and xen-cmdline to set it in grub conf file to take effect after reboot.
xenpm examples:

   # xenpm set-max-cstate 0 0
   max C-state set to C0
   max C-substate set to 0 succeeded
   # xenpm set-max-cstate 0
   max C-state set to C0
   max C-substate set to unlimited succeeded
   # xenpm set-max-cstate unlimited
   max C-state set to unlimited
   # xenpm set-max-cstate -1
   Missing, excess, or invalid argument(s)

xen-command-line examples:

/opt/xensource/libexec/xen-cmdline --get-xen max_cstate
     "" -> unlimited
     "max_cstate=N" -> max cstate N
     "max_cstate=N,M" -> max cstate N, max c-sub-state M *)
/opt/xensource/libexec/xen-cmdline --set-xen max_cstate=1
/opt/xensource/libexec/xen-cmdline --set-xen max_cstate=1,0
/opt/xensource/libexec/xen-cmdline --delete-xen max_cstate

xen-command-line.max_cstate.

This PR adds a new field host.max_cstate to manage host's max_cstate. host.set_max_cstate use the two commands mentioned above to configure. While dbsync on xapi start, the filed will be synced by xen-cmdline --get-xen max_cstate

changlei-li · 2025-09-22T08:35:36Z

Test:
Initial state:

# /opt/xensource/libexec/xen-cmdline --get-xen max_cstate
                                                                                                                           
# xe host-param-get uuid=a66aed30-2241-4c5e-b644-7c0874eb287f param-name=max-cstate

host.set_max_cstate

# xe host-param-set uuid=a66aed30-2241-4c5e-b644-7c0874eb287f max-cstate="0"                                
# xe host-param-get uuid=a66aed30-2241-4c5e-b644-7c0874eb287f param-name=max-cstate                       
0                                                                                                                          
# /opt/xensource/libexec/xen-cmdline --get-xen max_cstate                                                 
max_cstate=0                                                                                                             


# xe host-param-set uuid=a66aed30-2241-4c5e-b644-7c0874eb287f max-cstate=""                                
# xe host-param-get uuid=a66aed30-2241-4c5e-b644-7c0874eb287f param-name=max-cstate                       
                                                 
# /opt/xensource/libexec/xen-cmdline --get-xen max_cstate                                                 
      

# xe host-param-set uuid=a66aed30-2241-4c5e-b644-7c0874eb287f max-cstate="1,0"
# xe host-param-get uuid=a66aed30-2241-4c5e-b644-7c0874eb287f param-name=max-cstate
1,0
# /opt/xensource/libexec/xen-cmdline --get-xen max_cstate
max_cstate=1,0

dbsync

# /opt/xensource/libexec/xen-cmdline --delete-xen max_cstate                                              
# xe host-param-get uuid=a66aed30-2241-4c5e-b644-7c0874eb287f param-name=max-cstate                       
1,0                                                                                                                          
# xe-toolstack-restart                                                                                    
Executing xe-toolstack-restart                                                                                             
done.                                                                                                                      
# xe host-param-get uuid=a66aed30-2241-4c5e-b644-7c0874eb287f param-name=max-cstate

error message:

# xe host-param-set uuid=a66aed30-2241-4c5e-b644-7c0874eb287f max-cstate="-1"
The value given is invalid
field: max_cstate
value: -1

# xe host-param-set uuid=a66aed30-2241-4c5e-b644-7c0874eb287f max-cstate="3"
You attempted to set a value that is not supported by this implementation. The fully-qualified field name and the value that you tried to set are returned. Also returned is a developer-only diagnostic reason.
field: max_cstate
value: 3
reason: Only C0, C1 and unlimited are supported for max-cstate

# xe host-param-set uuid=a66aed30-2241-4c5e-b644-7c0874eb287f max-cstate="C0"
The value given is invalid
field: max_cstate
value: C0

python script example:

    session.xenapi.host.set_max_cstate(host_ref, "1,0")
    print(session.xenapi.host.get_max_cstate(host_ref))
    try:
        session.xenapi.host.set_max_cstate(host_ref, "5")
    except Exception as e:
        print(e)
    try:
        session.xenapi.host.set_max_cstate(host_ref, "C1")
    except Exception as e:
        print(e)
----
1,0
['VALUE_NOT_SUPPORTED', 'max_cstate', '5', 'Only C0, C1 and unlimited are supported for max-cstate']
['INVALID_VALUE', 'max_cstate', 'C1']

last-genius · 2025-09-22T09:06:58Z

ocaml/xapi/xapi_host.ml

+      Xapi_host_max_cstate.xenpm_set value 0L ;
+      Xapi_host_max_cstate.xen_cmdline_set value 0L ;


This means that currently the only value allowed for the substate is 0 - so the C1E enhanced halt state will not be legal (given that max cstate is currently 1). is this going to change in the future?

Yes. The requirement limits the value to C0, C1 or default. And C1E itself should not be configured by the integer substate.

Probably best to make the API itself more generic and allow configuring the sub-states too with the same API call, i.e. make it take 2 parameters (it is fine to make it stricter in the UI).

The requirement are for the high-level user experience (maybe the person who wrote it thought that keeping the list smaller is easier to implement, but that is usually not the case), and shouldn't be taken too literally.
The requirements aren't for the low-level/internal API details (where we shouldn't block things unless we have a good reason to), although of course the API needs to allow at least what the requirements asked for (it is OK to allow more).

One of the documented design principles of XAPI is:

The API is for developers and power users. If you ask it to blow your foot off, it should blow your foot off, not ask if you're sure. The GUI and CLI are where you query whether your user is sane.

If you want to keep the GUI and CLI simple by exposing just C0, C1 and unlimited that is probably OK (although in this case restricting the CLI would be more work for no good reason, it might be easier to just make it allow whatever the API allows)

I add the support to set max-sub-cstate now. See the updated test result in the first comment. The field now is string, just the same rule as https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html#max_cstate-x86

last-genius · 2025-09-22T09:10:12Z

I'm also not completely sure I understand how this fits into the "config-ntp-timezone-maxcstate" feature, could the bigger picture be clarified?

changlei-li · 2025-09-22T10:36:00Z

I'm also not completely sure I understand how this fits into the "config-ntp-timezone-maxcstate" feature, could the bigger picture be clarified?

@last-genius As it shows, in this branch, some apis will be added to configure ntp, timezone, max_cstate things. Currently, users have to ssh to dom0 to configure them. Now they can use xenapi to replace the ssh based workflows. Especially we provided the feature that can disable ssh on dom0 recently.

lindig

Generally solid. Some minor comments.

ocaml/idl/datamodel_host.ml

lindig · 2025-09-22T14:07:39Z

ocaml/xapi/xapi_host.ml

+    in
+    raise
+      Api_errors.(
+        Server_error


I believe the list should not be used for arbitrary strings. Our API errors are not ergonomic :(

yes. the message is like

You attempted to set a value that is not supported by this implementation. The fully-qualified field name and the value that you tried to set are returned. Also returned is a developer-only diagnostic reason. field: max_cstate value: 3 reason: Only C0, C1 and unlimited are supported for max-cstate

The values in list are filled to field, value, reason orderly.

lindig · 2025-09-22T14:09:05Z

ocaml/xapi/xapi_host.ml

+        Printf.sprintf "Failed to update max_cstate: %s" (Printexc.to_string e)
+      in
+      error "%s" err_msg ;
+      Helpers.internal_error "%s" err_msg


internal_error accepts a printf-style format. Why not use it directly?

I find the internal error has a parameter log_err and default set tp false. I want to add the log. Then I didn't want to write the format string two times.

edwintorok · 2025-09-22T14:15:26Z

ocaml/xapi/xapi_host_max_cstate.ml

+    | v, sub_v when v >= 0L && sub_v >= 0L ->
+        ["--set-xen"; Printf.sprintf "max_cstate=%Ld,%Ld" v sub_v]
+    | v, -1L when v >= 0L ->
+        ["--set-xen"; Printf.sprintf "max_cstate=%Ld" v]


We have something like this already for setting sched-gran.

It'd be good to refactor that API call (in a separate commit prior to this one), and introduce some set-xen and delete-xen helpers, that both of these API calls could reuse. That'll also help if in the future we want to add more tweakable parameters.

The refactored functions should probably take a list of key-value pairs, where the value is optional, so that we only need to declare the key once, and we wouldn't risk typoing delete-xen vs set-xen.
(the value needs to be optional to allow for arguments that are not key-value pairs, like some kernel cmdline options are)

After a quick try, the functions are like Xen_cmdline_helper.set_xen kvs, Xen_cmdline_helper.delete_xen keys. I find it's every tiny different to add one more abstract layer for the xen_cmdline script. And I am not very clear about the xen_cmdline_helper behavior in different cases (like set a list of key-values). So, I am not going to refactor the functions now.

changlei-li · 2025-09-24T07:54:37Z

I would add support to set both max_ctate and max_sub_cstate in this API. Convert to draft now.

Done. The test result is updated in the first comment.

Two commands are used to set max_cstate: xenpm to set at runtime and xen-cmdline to set it in grub conf file to take effect after reboot. Signed-off-by: Changlei Li <changlei.li@cloud.com>

String is used to represent the max_cstate and max_sub_cstate. "" -> unlimited "N" -> max cstate CN "N,M" -> max cstate CN with max sub state M Just follow the xen-cmdline cstate, see https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html#max_cstate-x86 Signed-off-by: Changlei Li <changlei.li@cloud.com>

Signed-off-by: Changlei Li <changlei.li@cloud.com>

last-genius reviewed Sep 22, 2025

View reviewed changes

lindig approved these changes Sep 22, 2025

View reviewed changes

edwintorok reviewed Sep 22, 2025

View reviewed changes

changlei-li marked this pull request as draft September 24, 2025 07:52

changlei-li added 3 commits September 24, 2025 16:46

CP-54473 Implement max-state setting

788fd74

Two commands are used to set max_cstate: xenpm to set at runtime and xen-cmdline to set it in grub conf file to take effect after reboot. Signed-off-by: Changlei Li <changlei.li@cloud.com>

update last_known_schema_hash

182a553

Signed-off-by: Changlei Li <changlei.li@cloud.com>

changlei-li force-pushed the private/changleli/config-max-cstate branch from 74e5aa9 to 182a553 Compare September 24, 2025 08:49

changlei-li marked this pull request as ready for review September 24, 2025 10:29

		Xapi_host_max_cstate.xenpm_set value 0L ;
		Xapi_host_max_cstate.xen_cmdline_set value 0L ;

CP-54473 Configure max-cstate by XAPI #6681

Are you sure you want to change the base?

CP-54473 Configure max-cstate by XAPI #6681

Uh oh!

Conversation

changlei-li commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changlei-li commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

edwintorok Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

last-genius commented Sep 22, 2025

Uh oh!

changlei-li commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lindig left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

edwintorok Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

edwintorok Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

changlei-li commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

changlei-li commented Sep 22, 2025 •

edited

Loading

changlei-li commented Sep 22, 2025 •

edited

Loading

edwintorok Sep 22, 2025 •

edited

Loading

changlei-li commented Sep 22, 2025 •

edited

Loading

edwintorok Sep 22, 2025 •

edited

Loading

edwintorok Sep 22, 2025 •

edited

Loading

changlei-li commented Sep 24, 2025 •

edited

Loading