Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add task::consume_budget util for cooperative scheduling #4498

Merged
merged 2 commits into from May 30, 2022

Conversation

psarna
Copy link
Contributor

@psarna psarna commented Feb 14, 2022

Motivation

For cpu-only computations that do not use any Tokio resources, budgeting does not really kick in in order to yield and prevent other tasks from starvation. One way to take part in budgeting is to use a Tokio resource, but some workloads simply don't need any, as they just need to compute a lot and occasionally yield in order to keep latencies low (e.g. a Wasm engine).

Solution

The new mechanism - task::consume_budget, performs a budget check and consumes a unit of it as if a Tokio resource was used (except it wasn't, so it's cheaper) and yields only if the task exceeded the budget. That allows cpu-intenstive computations to define yield points that yield only conditionally, since unconditional yielding to runtime is a potentially heavy operation.

@carllerche
Copy link
Member

I think this is probably a good feature to add. I would probably not name it "maybe_yield_now()". Instead, name it something to reflect that it represents conceptually tracking work.

task::did_work().await
task::consume_work_budget().await

^^ Don't love either of those names, but yeah.

@psarna
Copy link
Contributor Author

psarna commented Feb 14, 2022

Right, the name should reflect that there's a side effect - burning budget. I'll figure out a new name and send it in v2, along with the tests

@Darksonn Darksonn added A-tokio Area: The main tokio crate M-coop Module: tokio/coop labels Feb 15, 2022
@psarna psarna changed the title Add maybe_yield_now util for cooperative scheduling Add task::consume_budget util for cooperative scheduling Feb 15, 2022
@psarna
Copy link
Contributor Author

psarna commented Feb 15, 2022

v2:

  • renamed the utility to task::consume_budget
  • added a test case
  • simplified the code by using Poll:map instead of an explicit match; possible, because coop::poll_proceed already wakes up the task, so it doesn't need to be done twice

@psarna psarna marked this pull request as ready for review February 15, 2022 11:41
@psarna psarna force-pushed the maybe_yield_now branch 3 times, most recently from 7ae1285 to 9f0e842 Compare February 15, 2022 13:28
@carllerche
Copy link
Member

@jonhoo I would love to get your thoughts on naming and the feature in general.

Copy link
Member

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I think something like this is probably a good idea. I had some minor nitpicks and suggestions, but they're not hard blockers.

If we want to avoid a very long bikeshed over this API, it occurs to me that we could just go ahead and ship it as an unstable feature, so people can start experimenting with it, and then we can come back and stabilize the API separately?

/// Consumes a unit of budget and returns the execution back to the Tokio
/// runtime *if* the task went out ouf budget.
///
/// The task will only yield if it ran out of its coop budget.
/// It can be used in order to insert optional yield points into long
/// computations that do not use Tokio resources like sockets or semaphores,
/// without redundantly yielding to runtime each time.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one note about the documentation for this is that this is the first actual mention of the cooperative scheduling budget in the API documentation. I think it is probably worth adding a high-level summary of the task budget somewhere in the tokio::task documentation and having this API documentation reference that? Currently, the task budget is basically only documented in this blog post which seems maybe not ideal...

/// computations that do not use Tokio resources like sockets or semaphores,
/// without redundantly yielding to runtime each time.
#[cfg_attr(docsrs, doc(cfg(feature = "rt")))]
pub async fn consume_budget() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder if we should pre-emptively mark this as an unstable feature, so that we can ship it sooner in the next release while reserving the ability to bikeshed the API for a bit longer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you point me to a code example showing how to mark a feature as unstable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm, just found #4499, I'll follow it as a guideline

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I view the generated docs, then it is not listed as unstable. You should add that to the doc(cfg(..)).

#[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = "rt"))))]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thanks!

struct ConsumeBudget {
status: Poll<()>,
}

impl Future for ConsumeBudget {
type Output = ();

fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
if self.status.is_ready() {
return self.status;
}
self.status = crate::coop::poll_proceed(cx).map(|restore| {
restore.made_progress();
});
self.status
}
}

ConsumeBudget { status: Poll::Pending }.await
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, take it or leave it: i think we could probably implement this a bit more simply using poll_fn:

Suggested change
struct ConsumeBudget {
status: Poll<()>,
}
impl Future for ConsumeBudget {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
if self.status.is_ready() {
return self.status;
}
self.status = crate::coop::poll_proceed(cx).map(|restore| {
restore.made_progress();
});
self.status
}
}
ConsumeBudget { status: Poll::Pending }.await
let mut status = Poll::Pending;
crate::future::poll_fn(move |cx| {
if status.is_ready() {
return status;
}
status = crate::coop::poll_proceed(cx).map(|restore| {
restore.made_progress();
});
status
}).await

i believe this should work identically, and avoids a little bit of boilerplate by not having to impl Future for a type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat! Looks much better, thanks

use std::task::{Context, Poll};

/// Consumes a unit of budget and returns the execution back to the Tokio
/// runtime *if* the task went out ouf budget.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: how about

Suggested change
/// runtime *if* the task went out ouf budget.
/// runtime *if* the task's coop budget was exhausted.

/// The task will only yield if it ran out of its coop budget.
/// It can be used in order to insert optional yield points into long
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe

Suggested change
/// The task will only yield if it ran out of its coop budget.
/// It can be used in order to insert optional yield points into long
/// The task will only yield if its entire coop budget has been exhausted.
/// This function can be used in order to insert optional yield points into long

/// The task will only yield if it ran out of its coop budget.
/// It can be used in order to insert optional yield points into long
/// computations that do not use Tokio resources like sockets or semaphores,
/// without redundantly yielding to runtime each time.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
/// without redundantly yielding to runtime each time.
/// without redundantly yielding to the runtime each time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, it could be nice to have an example in the documentation, maybe showing this in a long-running loop or something?

@jonhoo
Copy link
Sponsor Contributor

jonhoo commented Feb 15, 2022

We can always make this a no-op down the line should we decide that coop isn't a thing we want, and make it forward to an external lib if that's where coop ends up living, so I'm 👍 on this in general. In fact, a method like this existed in the original implementation in #2160, initially called cooperate, and later renamed to coop::proceed. It was never made publicly available though, and was removed as part of the coop simplification in #2498. I'm happy to see it coming back. I do think we should likely also expose poll_proceed, since coop is low enough level that it's fairly likely a pollable API will be needed.

@psarna psarna force-pushed the maybe_yield_now branch 3 times, most recently from 399968e to 613d125 Compare February 16, 2022 10:02
@psarna
Copy link
Contributor Author

psarna commented Feb 16, 2022

v3:

  • marked the feature as unstable
  • applied the wording suggestions in comments (thanks!)
  • added a simple example to the doc
  • refactored the implementation to use poll_fn instead, as recommended (thanks again!)

I also retested everything with the additional RUSTFLAGS='--cfg tokio_unstable' flag, so that my new test would also get executed. Is it the correct way to go?

@psarna
Copy link
Contributor Author

psarna commented Mar 3, 2022

This failure in CI/test tokio full run looks unrelated to my patch - it errors out on some macro tests that I haven't touched and I'm able to reproduce the same failures on current master branch.

@Darksonn
Copy link
Contributor

Darksonn commented Mar 3, 2022

Those errors are due to the previous release of rustc. Please rebase or merge master into your branch to fix them.

@psarna
Copy link
Contributor Author

psarna commented Mar 3, 2022

got it, rebased, thanks

@psarna
Copy link
Contributor Author

psarna commented Apr 14, 2022

(ping)

@psarna
Copy link
Contributor Author

psarna commented May 5, 2022

(rebased on top of newest master)

@psarna
Copy link
Contributor Author

psarna commented May 30, 2022

(monthly rebase)

For cpu-only computations that do not use any Tokio resources,
budgeting does not really kick in in order to yield and prevent
other tasks from starvation. The new mechanism - consume_budget,
performs a budget check, consumes a unit of it, and yields only
if the task exceeded the budget. That allows cpu-intenstive
computations to define points in the program which indicate that
some significant work was performed. It will yield only if the budget
is gone, which is a much better alternative to unconditional yielding,
which is a potentially heavy operation.
The test case ensures that the task::consume_budget utility
actually burns budget and makes the task yield once the whole
budget is gone.
Copy link
Contributor

@Darksonn Darksonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some other coop APIs that I wonder if they are more important to add. (Mainly, a public version of our poll_proceed.) However, this function does seem useful too, and it is marked as unstable, so it seems fine to just go ahead and merge this.

@Darksonn Darksonn enabled auto-merge (squash) May 30, 2022 08:58
@Darksonn
Copy link
Contributor

Sorry for taking so long.

@Darksonn Darksonn merged commit 88e8c62 into tokio-rs:master May 30, 2022
41 checks passed
@psarna
Copy link
Contributor Author

psarna commented May 30, 2022

Thanks! No worries, I often take active part in review processes that take years to finish, this one was actually quite painless and swift (:

crapStone pushed a commit to Calciumdibromid/CaBr2 that referenced this pull request Jun 11, 2022
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.18.2` -> `1.19.1` |
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.18.2` -> `1.19.1` |

---

### Release Notes

<details>
<summary>tokio-rs/tokio</summary>

### [`v1.19.1`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.19.1)

[Compare Source](tokio-rs/tokio@tokio-1.19.0...tokio-1.19.1)

##### 1.19.1 (June 5, 2022)

This release fixes a bug in `Notified::enable`. ([#&#8203;4747])

[#&#8203;4747]: tokio-rs/tokio#4747

### [`v1.19.0`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.19.0)

[Compare Source](tokio-rs/tokio@tokio-1.18.2...tokio-1.19.0)

##### 1.19.0 (June 3, 2022)

##### Added

-   runtime: add `is_finished` method for `JoinHandle` and `AbortHandle` ([#&#8203;4709])
-   runtime: make global queue and event polling intervals configurable ([#&#8203;4671])
-   sync: add `Notified::enable` ([#&#8203;4705])
-   sync: add `watch::Sender::send_if_modified` ([#&#8203;4591])
-   sync: add resubscribe method to broadcast::Receiver ([#&#8203;4607])
-   net: add `take_error` to `TcpSocket` and `TcpStream` ([#&#8203;4739])

##### Changed

-   io: refactor out usage of Weak in the io handle ([#&#8203;4656])

##### Fixed

-   macros: avoid starvation in `join!` and `try_join!` ([#&#8203;4624])

##### Documented

-   runtime: clarify semantics of tasks outliving `block_on` ([#&#8203;4729])
-   time: fix example for `MissedTickBehavior::Burst` ([#&#8203;4713])

##### Unstable

-   metrics: correctly update atomics in `IoDriverMetrics` ([#&#8203;4725])
-   metrics: fix compilation with unstable, process, and rt, but without net ([#&#8203;4682])
-   task: add `#[track_caller]` to `JoinSet`/`JoinMap` ([#&#8203;4697])
-   task: add `Builder::{spawn_on, spawn_local_on, spawn_blocking_on}` ([#&#8203;4683])
-   task: add `consume_budget` for cooperative scheduling ([#&#8203;4498])
-   task: add `join_set::Builder` for configuring `JoinSet` tasks ([#&#8203;4687])
-   task: update return value of `JoinSet::join_one` ([#&#8203;4726])

[#&#8203;4498]: tokio-rs/tokio#4498

[#&#8203;4591]: tokio-rs/tokio#4591

[#&#8203;4607]: tokio-rs/tokio#4607

[#&#8203;4624]: tokio-rs/tokio#4624

[#&#8203;4656]: tokio-rs/tokio#4656

[#&#8203;4671]: tokio-rs/tokio#4671

[#&#8203;4682]: tokio-rs/tokio#4682

[#&#8203;4683]: tokio-rs/tokio#4683

[#&#8203;4687]: tokio-rs/tokio#4687

[#&#8203;4697]: tokio-rs/tokio#4697

[#&#8203;4705]: tokio-rs/tokio#4705

[#&#8203;4709]: tokio-rs/tokio#4709

[#&#8203;4713]: tokio-rs/tokio#4713

[#&#8203;4725]: tokio-rs/tokio#4725

[#&#8203;4726]: tokio-rs/tokio#4726

[#&#8203;4729]: tokio-rs/tokio#4729

[#&#8203;4739]: tokio-rs/tokio#4739

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

 **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [x] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Co-authored-by: cabr2-bot <cabr2.help@gmail.com>
Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1394
Reviewed-by: crapStone <crapstone@noreply.codeberg.org>
Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-coop Module: tokio/coop
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants