OS-level thread::Builder priority and affinity extensions

[ian-h-chamberlain]

Proposal

Supersedes #71. This proposal is based on the discussion there and on Zulip about that proposal.

cc @AzureMarker @joshtriplett as they were involved in the earlier discussion / proposal as well.

Problem statement

Exposing OS thread scheduling options has been requested for quite a long time. Some third-party crates have been created to supplement the std functionality, but they seem to mostly work by either duplicating std APIs or only work for the current thread after spawning.

This proposal aims to enable std::os extension traits that can modify thread::Builder to set some of these properties before a thread is spawned, without committing to a higher-level cross-platform API (but ideally leaving room for one to be designed and implemented in the future).

Motivation, use-cases

Setting thread affinity and priority has a variety of motivations. For platforms that use a cooperative thread scheduler, setting CPU affinity and priority may be necessary to ensure that threads are not starved or cause deadlocks. High-performance / realtime applications may need fine grained control over their threads to meet performance requirements.

In principle, I believe this proposal should enable implementation of the necessary APIs to exert low-level OS-specific control over threads, which should pave the way for a more general-purpose cross-platform solution for these use cases.

Solution sketches

I have taken some of the changes from previous attempts and implemented them in a fork here, trying to show the minimum viable API surface:

• std::os::linux allows for setting affinity with libc::cpu_set_t (OS-specific structure).
• std::os::horizon allows for setting priority with libc::c_int (OS-specific meaning).
• Other platforms are unaffected. It should be possible to expand the extension traits as needed.

The new public API surface area is fairly small but can be expanded on a per-platform basis as support is added / desired:

std::os::horizon::thread

/// Horizon-specific extension trait for [`thread::Builder`](crate::thread::Builder).
pub trait BuilderExt {
    /// Set the priority of the thread to be spawned.
    ///
    /// See <https://www.3dbrew.org/wiki/Multi-threading#Threads> for details
    /// about the meaning / valid values of the `priority` parameter.
    fn priority(self, priority: libc::c_int) -> Self;
}

impl BuilderExt for crate::thread::Builder {}

std::os::linux::thread

/// Linux-specific extensions for [`thread::Builder`](crate::thread::Builder).
pub trait BuilderExt {
    /// Set the CPU affinity (which cores to run on) for the thread to be spawned.
    ///
    /// See <https://man7.org/linux/man-pages/man3/CPU_SET.3.html> for more details
    /// about how to construct the `cpu_set` parameter.
    fn affinity(self, cpu_set: libc::cpu_set_t) -> Self;
}

impl BuilderExt for crate::thread::Builder {}

Draft of an implementation here:

rust-lang/rust@master...ian-h-chamberlain:rust:feature/thread-schedule-os-ext

Links and related work

• Long-standing RFC Expose thread/task priorities rfcs#819
• thread-priority crate (duplicates std::thread::Builder API to allow setting before spawn)
• core_affinity crate (only allows set/get of current thread's affinity)
• Initial PR for armv6k-nintendo-3ds extension traits: std::thread support for the Nintendo 3DS rust#98514
• Previous proposal motivated by discussion of the above: Add Special Thread Builder Functions for the Nintendo 3DS #71
• First attempt implementing the above proposal: Partially implement thread scheduling attributes API proposal rust#101222
• Zulip discussion of the above proposal https://rust-lang.zulipchat.com/#narrow/stream/327149-t-libs-api.2Fapi-changes/topic/Reviews.20.2F.20feedback.20on.20thread.20priority.20proposal.3F

[the8472]

Why are those on the thread-builder though? Thread priorities and affinities can be set after creation on most platforms.

A more general and portable mechanism would be providing a closure that can wrap the Builder::spawn closure, similar to CommandExt::pre_exec (but safer).

And the mix of proposed horizon and linux APIs is weird.

[ChrisDenton]

See Processor Groups and Scheduling Priorities for Windows considerations.

Since (I assume) most libstd platforms support some form of setting priorities and/or affinities, surely there most be some hope of cross-platform APIs? Within the standard library I personally really prefer cross-platform APIs if at all possible because it provides the most value. Users get cross-platform support "for free" just by using normal APIs.

[ian-h-chamberlain]

Why are those on the thread-builder though? Thread priorities and affinities can be set after creation on most platforms.

That's true, this proposal is specifically limited to the Builder (setting these attributes before spawning) as I didn't want to overextend the scope. I could see potential for a similar extension trait on std::thread::Thread as well in the future.

A more general and portable mechanism would be providing a closure that can wrap the Builder::spawn closure, similar to CommandExt::pre_exec (but safer).

This is a fair point and something I hadn't considered. Even with this mechanism though, I think OS-specific helpers would need to be provided to expose the underlying structures that control the spawning of the thread. Maybe it would end up looking something like this?

use std::os::linux::thread::affinity;

let cpus: libc::cpu_set_t = todo!();

let t = std::thread::Builder::new()
    .with(affinity(cpus))
    .spawn(|| println!("hi"))
    .unwrap();

t.join().unwrap();

And the mix of proposed horizon and linux APIs is weird.

The main reason I did both of these was to show examples - in the last iteration of this proposal, @joshtriplett asked for a proof of concept that this could be applied to multiple (including Tier 1) platforms. I am not as familiar with macOS/Windows APIs for this so I chose Linux as the proof of concept target.

I think one advantage to using OS-specific trait extensions like this is that it would in theory allow platform experts for those particular platforms to add APIs as they saw fit, rather than shoehorning all platforms into a paradigm that doesn't make sense for them (e.g. enum-like priority values vs integer priority values).

---

See Processor Groups and Scheduling Priorities for Windows considerations.

Since (I assume) most libstd platforms support some form of setting priorities and/or affinities, surely there most be some hope of cross-platform APIs? Within the standard library I personally really prefer cross-platform APIs if at all possible because it provides the most value. Users get cross-platform support "for free" just by using normal APIs.

Thanks for the reference! This is interesting, actually:

All threads are created using THREAD_PRIORITY_NORMAL.

That seems to suggest to me that Windows doesn't actually provide a way to set priority before spawning a thread, which would mean that a true cross-platform API for Builder wouldn't be possible (maybe it could be a Result that always returns Err on Windows).

I definitely agree that cross-platform is ideal, but the scope of design for a fully cross-platform API for something like this seems fairly large. The idea with this proposal is to form the building blocks that could later be used to implement a cross platform API. I think by exposing these lower-level knobs we could enable third party crates like thread-priority to better experiment with APIs that could eventually be upstreamed into std, without committing to a specific design up front.

[the8472]

Even with this mechanism though, I think OS-specific helpers would need to be provided to expose the underlying structures that control the spawning of the thread. Maybe it would end up looking something like this?

It means stuff doesn't have to live in std. E.g. if you have these 3 building blocks

1. a thread-pool crate
2. std's thread builder
3. a crate to set priorities and affinities on running threads

Currently you can glue 1 and 2 together to give the thread pool named threads and custom stack sizes. But to set the pool's affinities you either have to hope for direct support from the pool crate or do some clunky dance that involves submitting a bunch of jobs to the pool which then set the affinity, those are difficult to coordinate.

If the builder gains closure-wrapping then these types suddenly compose better and you can use that crate to set affinities as soon as the thread starts running and you don't need affinity stuff in std.

It also avoids the need to have separate thread-create and running-thread APIs. Some of the former would have to be emulated with the latter anyway.

The only case where this doesn't work are platforms where these things must be set before thread startup.

[joshtriplett]

These extension traits should be sealed, so that we can extend them further in the future.

I do think we're going to need target-specific mechanisms here, because the same abstractions will not work for all targets. That said, I do wonder if we can provide some abstractions that work here. For instance, we could have the concept of "thread priority" in the form of:

• An opaque type that exists on all platforms
• Portable functions in Builder that accept that type.
• Target-specific ways to construct that type.
• Portable abstractions that construct that type, which can start out in crates outside of the standard library.

[pitaj]

Agreed, I'm thinking we can at least have

impl OpaquePriority {
    /// Returns a new instance of the highest 
    /// possible thread priority. This may be equal 
    /// to default on platforms without thread priority.
    fn highest() -> Self;
    /// Returns a new instance of the 
    /// normal / default thread priority.
    // Maybe this should be a Default impl
    fn default() -> Self;
    /// Returns a new insurance of the lowest 
    /// possible thread priority. This may be equal 
    /// to default on platforms without thread priority.
    fn lowest() -> Self;
}

I also agree think that the target specific extension traits should come first, then the target-independent abstraction can be built upon it.

[AzureMarker]

Why are those on the thread-builder though? Thread priorities and affinities can be set after creation on most platforms.

At least for Horizon, the thread affinity can only be set before the thread is spawned.

[ian-h-chamberlain]

If the builder gains closure-wrapping then these types suddenly compose better and you can use that crate to set affinities as soon as the thread starts running and you don't need affinity stuff in std.

It also avoids the need to have separate thread-create and running-thread APIs. Some of the former would have to be emulated with the latter anyway.

The only case where this doesn't work are platforms where these things must be set before thread startup.

As @AzureMarker mentioned, one of the motivating platforms for this proposal indeed cannot set affinity after thread creation, which is why the proposed API operates on the Builder itself.

The closure-wrapping idea is interesting, but the more I think about it the more I wonder -- is there any advantage to this kind of API compared to what you can already accomplish with spawn()? Example:

std::thread::Builder::new()
    .spawn(|| {
        // some OS-specific API, possibly from 3rd party crate
        set_current_thread_priority(123);

        println!("hi");
	})
    .unwrap()
    .join()
    .unwrap();

// vs

std::thread::Builder::new()
    // possible new API:
    .pre_spawn(|| {
        // some OS-specific API, possibly from 3rd party crate
        set_current_thread_priority(123);
    })
    .spawn(|| {
        println!("hi");
	})
    .unwrap()
    .join()
    .unwrap();

The only thing I could think of offhand would be that the pre_spawn closure in this example could be FnMut() -> io::Result<()> or something like that, which could propagate errors from the pre-spawn closure to spawn() itself, perhaps? Or maybe I'm missing something in how I'm thinking about this kind of API?

[the8472]

is there any advantage to this kind of API compared to what you can already accomplish with spawn()? Example:

Your example omits the interaction between different crates. If you have a threadpool crate which can be configured then one possible configuration item is the threadbuilder. Once you start submitting closures to the pool you have no control over on which thread the tasks will execute or how many closures will be executed on which thread. So a pool.submit() API wouldn't be the right place to set thread priorities. But a pool.init(builder) would work by passing a builder with a pre_spawn closure.

As @AzureMarker mentioned, one of the motivating platforms for this proposal indeed cannot set affinity after thread creation, which is why the proposed API operates on the Builder itself.

Sure, for those platforms it makes sense to implement extra methods because thread-spawning is encapsulated in std and not accessible to other crates. But for all other cases we can leave it to 3rd party code which can iterate on useful portable abstractions.