Is there any low power mode for DirectML

[liyuming1978]

hi, now I have a quick enough model (120fps) and will run at 20fps, what i need is use as low as possible gpu power. but i find the gpu frequency jump to 1150mhz too many times. as compare to "https://voovmeeting.com/download-center.html?from=1001" tencent meeting , I found when I enable human segmentation , in a 8xxx laptop, the gpu frequency hold below 400mhz , but GPU load over 75%, that is strange for frequency policy.
so I guess , maybe directx12 or dx11 has some low power mode ? or some other ways, for ex. add some wait in each OP (for ex. convolution op）

[jstoecker]

It's definitely a good idea to have your app limit the FPS manually: space out the recording and dispatching of work to hit your FPS target. However, you can't guarantee that the hardware driver won't boost clock speed to execute the work in each frame; these policies vary and must factor in potential demands from many simultaneous applications.

My suggestion would also be to have your application to select the "minimum power" adapter if more than one is available. Check out IDXCoreAdapterList::Sort and the DXCoreAdapterPreference::MinimumPower enum. Generally this means using the integrated GPU when possible, and these adapters generally tend to be fairly sophisticated in sipping as little power as necessary. Of course, you'll want to make sure that this device is capable of providing enough FPS for your application.

[liyuming1978]

the fps has been limited to 20fps (even it can run 120fps). I have only one embeded gpu card , so DXCoreAdapterPreference::MinimumPower has not use for this issue :) i have also try this before, no use...
I will keep to find a good solution . thx

[liyuming1978]

@jstoecker : I find the reason! but I have no idea how to resolve it ...
I test as below:
use MicrosoftMLRunner.exe to run mnist.onnx : MicrosoftMLRunner.exe -model \mnist.onnx -perf -gpu -Iterations 100000
I use two cmd to perf at the same time (make the workload big enough ), and then I find
the gpu load is over 70% , but frequency is only 300mhz (it has a quick boost at start , but hold 300mhz later) , and the gpu power is less than 1.0w (see the data with gpu-z)

so : conclusion is if model is small enough (or there are some sleep in a "big" model) , this will lead a powersaving mode if gpu is a balance power mode (default value for most computer).

the question is : how to add some sleep or flush function in a big model. to stop long running for gpu kernel.

[jstoecker]

Ah, I didn't realize you were running an ONNX model. @smk2007 , @fdwr : do either of you know of a good way to slow down a large ONNX model somehow? Is this even possible?

Perhaps you can break up the ONNX model graph into smaller models and run them with sleeps between, but I honestly have no idea if the language or various runtimes have some way of doing this for you.

[liyuming1978]

split into samll model is too hard for engineering... I need split one model to > 50 models ...

[jstoecker]

I understand. With higher level APIs/libraries this is one of the tradeoffs: less control over scheduling work.

It's an interesting requirement though. It might be worthwhile asking on onnxruntime GitHub to see if they have thoughts on this (energy efficient inference mode). I'll also try to ping some more folks to see if they have ideas to help you out.

[liyuming1978]

Microsoft.AI.DirectML.1.5.1 fix this issues