Consider using WMI for Windows cpu::frequency

[svartalf]

CallNtPowerInformation and PROCESSOR_POWER_INFORMATION combination returns some nominal frequency values, which are not really useful and on some systems could produce really weird values (ex. current frequency = 0, see #229).

It seems that WMI query for Win32_Processor is more reliable, it should be checked if heim should use WMI to load that info.
Points to check:

• Does that WMI data is more reliable?
• Is WMI available in all Windows versions?
• Are there crates available and maintained? wmi crate?

[zenhuw]

i found that in both cases (my amd and intel machine) WMI returns only base speed value, not realtime value.

i tried systeminformation lib in node, turns out it use wmi too, and it's only showing base speed.

[Lomanic]

It looks like the oshi Java library had the same issue as #229 in oshi/oshi#966, they improved or fixed the situation in oshi/oshi#989 with a Performance counter and a WMI call getting Win32_PerfRawData_Counters_ProcessorInformation

[apennamen]

Hello,

I ran a quick and dirty experiment with wmi crate.

I found that WMI has Pre-formatted counters that are easier to use than rawdata counter. Especially this property on performance counter called Win32_perfformatteddata_counters_processorinformation :
https://wutils.com/wmi/root/cimv2/win32_perfformatteddata_counters_processorinformation/#percentprocessorutility_properties

I think this is the counter used in Perfmon windows utility, also I was not able to check this :

Here is a small code, that runs with only the wmi dependency in Cargo.toml

use std::time::Duration;
use std::thread::sleep;
use std::collections::HashMap;
use wmi::Variant;
use wmi::{COMLibrary, WMIConnection};

fn main() {
    let com_con = COMLibrary::new().unwrap();
    let wmi_con = WMIConnection::new(com_con.into()).unwrap();

    for _ in 0..10 {
        let results: Vec<HashMap<String, Variant>> = wmi_con
            .raw_query("SELECT Name, PercentProcessorUtility FROM  Win32_PerfFormattedData_Counters_ProcessorInformation WHERE NOT Name LIKE \"%_Total\"").unwrap();
        for procs in results {
            println!("{:?}", procs);
        }
        sleep(Duration::new(1, 0));
    }
}

On an Octocore CPU this is the result of the first and last iteration of the loop :

{"Name": String("0,0"), "PercentProcessorUtility": String("8")}
{"Name": String("0,1"), "PercentProcessorUtility": String("10")}
{"Name": String("0,2"), "PercentProcessorUtility": String("3")}
{"PercentProcessorUtility": String("4"), "Name": String("0,3")}
{"PercentProcessorUtility": String("6"), "Name": String("0,4")}
{"PercentProcessorUtility": String("1"), "Name": String("0,5")}
{"PercentProcessorUtility": String("6"), "Name": String("0,6")}
{"Name": String("0,7"), "PercentProcessorUtility": String("7")}

...

{"PercentProcessorUtility": String("10"), "Name": String("0,0")}
{"Name": String("0,1"), "PercentProcessorUtility": String("2")}
{"PercentProcessorUtility": String("20"), "Name": String("0,2")}
{"Name": String("0,3"), "PercentProcessorUtility": String("4")}
{"PercentProcessorUtility": String("11"), "Name": String("0,4")}
{"PercentProcessorUtility": String("2"), "Name": String("0,5")}
{"PercentProcessorUtility": String("11"), "Name": String("0,6")}
{"PercentProcessorUtility": String("7"), "Name": String("0,7")}

You have 8 entries, one for each proc (from "0,0" to "0,7") and a string representing the current utility percent for the proc.

Tell me if you are interested in further investigations. I'm new to Rust but it could lead to a first PR

Regards,
AP

EDIT : I'm a bit off-topic as the subject here is the frequency, but I think there is a way to compute frequency from the various other informations available in the counter, like this for example in each proc :

"PercentofMaximumFrequency": I8(
42,
),

[apennamen]

Hi again :)

I think it could be worth it to integrate WMI in heim for windows, as you could use it for more than just CPU Frequency.

If we look back to your criterias :
- Does that WMI data is more reliable?
"Definitely" for real time data
In the particular case of CPU Frequency, we have 2 good candidates with the following query :

SELECT PercentofMaximumFrequency, PercentProcessorPerformance FROM Win32_PerfFormattedData_Counters_ProcessorInformation WHERE Name = '_Total'

The PercentProcessorPerformance is mentionned in this issue in oshi : oshi/oshi#966 (comment)

- Is WMI available in all Windows versions?
Not much details available in MS doc (https://docs.microsoft.com/en-us/windows/win32/wmisdk/operating-system-availability-of-wmi-components), but Tiers 1 is supported
Found on Wikipedia that WMI comes preinstalled in : Windows Me, Windows 2000, Windows XP, Windows Vista, Windows 7, Windows 8, Windows 10, Windows Server 2003, Windows Server 2008, Windows Server 2012.

- Are there crates available and maintained? wmi crate?
As you already mentionned in this issue , the wmi crate seems like an excellent candidate, despite its absence of async support. Not

I'll try to look into async WMI, and if an implementation is possible in Rust

Regards

[svartalf]

Hi, @apennamen! Amazing investigation, thank you!

Even if it will take time and effort to introduce async support for wmi crate, we can start with offloading queries to a blocking thread pool, same to how disk IO is executed now.
I'm still not quite sure if WMI is always installed and enabled on all used nowadays Windows versions, but since we don't have any other option to fetch this data, why shouldn't we use WMI?

Let me know if you need any help with your work!

[apennamen]

Hello there!
Happy new year ;) All the best for 2021!

I'm struggling on a AddRef/Release problem when adding async support to WMI crate. I think I won't wait until this is resolved (and hopefully merged!) before working on adding a Proof of Concept.
Maybe we can test if WMI is enabled, and if not fallback to the actual solution ;)

See you soon!
Regards,
AP.

[apennamen]

Hello,

As the work for adding async queries is reaching the end in the wmi crate (waiting for final reviews and minor adjustments), I started looking into the conception for how to integrate WMI with heim for frequencies, before creating a pull request.

WMI Benefits

We can take advantage of WMI to query general CPU frequency information, as well as per-CPU information. So we could provide a heim::cpu::os::windows::frequencies

CPU total information:
SELECT Name, PercentofMaximumFrequency, PercentProcessorPerformance FROM Win32_PerfFormattedData_Counters_ProcessorInformation WHERE Name = '_Total'

per-CPU information:
SELECT Name, PercentofMaximumFrequency, PercentProcessorPerformance FROM Win32_PerfFormattedData_Counters_ProcessorInformation WHERE NOT Name LIKE '%_Total\'

For the max and min frequency, some small researchs are needed to query them with WMI.

Tradeoff

There is adding complexity if we want to offer WMI + fallback to Winternl. I think it can be handle quite "elegantly" with a design pattern, but of course it would be easier to just stick with WMI for frequency.

General Design

Strategy Design Pattern

If WMI can't be used, we switch back to Winternl System Services. As we'll only know this at runtime, one way to implement this is to use a Strategy in heim-cpu::sys::windows::freq::CpuFrequency :

trait FreqStrategy {
  fn current(&self) -> Frequency;
  fn max(&self) -> Option<Frequency>;
  fn min(&self) -> Option<Frequency> {
    None
  }
}

struct WmiStrategy;
impl FreqStrategy for WmiFreqStrategy {
// -snip- use WMI to retrieve info
}

struct WinternlStrategy;
impl FreqStrategy for WinternlStrategy { 
// -snip- use winternl to retrieve info
}

pub struct CpuFrequency {
  // We use trait object to represent the abstract strategy
  strategy: Box<dyn FreqStrategy>,
}

impl CpuFrequency {
    pub fn current(&self) -> Frequency {
        self.strategy.current()
    }

    pub fn max(&self) -> Option<Frequency> {
        self.strategy.max()
    }

    pub fn min(&self) -> Option<Frequency> {
        self.strategy.min()
    }
}

pub async fn frequency() -> Result<CpuFrequency> {
// -snip- chose strategy: if WMI Connection fail, switch to Winternl
}

Pitfall: I was not expecting frequency() to be async, but rather current(), min() and max(). The implementation I propose here does not work yet giving this API, but I think the workaround can be found with some reflexion.

WMI Connection

For better performance, it is better to initialize the WMI Connection once, and then use it to query for frequency or other information. I guess one use case for heim-cpu is to get "real-time" CPU frequency update, and therefore to poll regularly the connection to get the info.

The WMI Connection could be a member of the WMIStrategy struct. That way, as long as the CpuFrequency object lives, the WMI Connection stays opened. I don't know if it's necessary, but WMI Connection could be lazy_static .

New Bindings

Both strategies would be moved to the binding module if it's appropriate (i do not get yet the signification of the binding module, if you can help me).

Per-CPU Frequencies

There is some reflexion to have here, as the information is available only with WMI and not with Winternl.
If we want to provide a frequencies() -> impl Stream<Item = Result<CpuFrequency>> API for windows, we need to provide an implementation with Winternl to, even if it only sends zeroed data.

Regards,
AP

[svartalf]

Hey, @apennamen, sorry for not getting back to you earlier.

There is adding complexity if we want to offer WMI + fallback to Winternl. I think it can be handle quite "elegantly" with a design pattern, but of course it would be easier to just stick with WMI for frequency.

I don't know much about Windows, but what are chances that WMI will be unaccessible on user machine?
It would be nice to get rid of all winapi calls due to safety reasons, so if WMI is generally available and running on all Windows hosts nowadays, it will be a good thing to do. Yet, I don't know if it is true, maybe you have some insights regarding this? MSDN says that WMI is installed by default everywhere now, but can we confirm that?

I was not expecting frequency() to be async, but rather current(), min() and max(). The implementation I propose here does not work yet giving this API, but I think the workaround can be found with some reflexion.

All top-level functions are marked as async, because when they are executed, they might do some blocking IO. In case of CPU frequency working via WMI, I would expect this function to execute WMI query and return some struct with pre-filled information; current(), min() and max() are acting as a getters to information in that struct here.

As I mentioned before, all functions in heim are following the same approach, so suggested approach does not fit in it that much. Why do you want to make all these "former" getters async? Am I understanding correctly that intent is to do like SELECT CurrentFrequency FROM WMI_CpuFrequencies (pseudo-query) each time when .current() is called?

I don't know if it's necessary, but WMI Connection could be lazy_static.

That sounds like a better approach: if it will be painless to integrate, I would prefer to replace whole Windows implementation with WMI.

Let me know what you think about it!

[apennamen]

Hello @svartalf !

It's okay I tested an implementation with the strategy, we can have both winternl and WMI.

I don't know much about Windows, but what are chances that WMI will be unaccessible on user machine?
It would be nice to get rid of all winapi calls due to safety reasons, so if WMI is generally available and running on all Windows hosts nowadays, it will be a good thing to do. Yet, I don't know if it is true, maybe you have some insights regarding this? MSDN says that WMI is installed by default everywhere now, but can we confirm that?

I have absolutely no previous experience with WMI, but I'll ask on the windows-dev channel of the discord if we can take for word the info in the MSDN documentation ^^

All top-level functions are marked as async, because when they are executed, they might do some blocking IO. In case of CPU frequency working via WMI, I would expect this function to execute WMI query and return some struct with pre-filled information; current(), min() and max() are acting as a getters to information in that struct here.

As I mentioned before, all functions in heim are following the same approach, so suggested approach does not fit in it that much. Why do you want to make all these "former" getters async? Am I understanding correctly that intent is to do like SELECT CurrentFrequency FROM WMI_CpuFrequencies (pseudo-query) each time when .current() is called?

Yes in fact it's only for the "current" getter that could have been interesting to trigger the wmi query.
We can keep the same API, and trigger 2 async WMI queries when calling frequency() : one for the min and max freq, and the other one for the current freq.

In your opinion, what is the best place to write the lazy_static WMI Connection ? I'll have to try different options and see which is best :)

Regards,
AP

[svartalf]

Hey!

I have absolutely no previous experience with WMI, but I'll ask on the windows-dev channel of the discord if we can take for word the info in the MSDN documentation ^^

Any news on that?

Yes in fact it's only for the "current" getter that could have been interesting to trigger the wmi query.
We can keep the same API, and trigger 2 async WMI queries when calling frequency() : one for the min and max freq, and the other one for the current freq.

I would prefer to stick with the current flow, when we load everything at once and getters are actually getters. I imagine that fetching all that info in one query is way cheaper than doing multiple queries for each getter.

In your opinion, what is the best place to write the lazy_static WMI Connection ? I'll have to try different options and see which is best :)

I guess if we will use it everywhere, putting it somewhere in the heim-common/src/sys/windows/* will make it accessible for all other sub-crates!