Running tasks via WaitForTask(NULL)

[baldurk]

I want to be able to say "wait up to N microseconds on the current thread for a task to be executable then run at most one task", and optionally repeat that afterwards with a smaller N, as a way of better using time than a usleep(N) on my main thread in between doing other high priority work. The docs for TaskScheduler::WaitForTask say:

if called with 0 it will try to run tasks, and return if none available.

And assuming 0 means NULL then that seems pretty much like what I want, however I'm not sure what guarantees there are on how much work this does. Will it run only one task or multiple? Will it keep going until no more are ready?

If this contractually only runs one task at most then I think that would pretty much do what I want, though I would need to implement the timeout myself with an external spinloop. I'm not sure if that's much less optimal than if you could do a timeout internally. I also don't see a way to tell if this actually ran a task, to be able to run at most one within the timeframe. That would be nice but isn't necessary.

Wall of text if desired with more context on what I'm trying to do to avoid the XY problem

---

I'm sketching a design where I would have a main thread which adds N tasks, and then be solely responsible for picking up special work from the tasks that needs to go to and from the GPU via a single controlling thread, running it and reading back the results, and adding follow-up tasks to process the results. Normally this would just have a sleep while waiting for results as there's a fair amount of latency in going to the GPU and getting results back, so I want to be able to run tasks on the CPU in the meantime on the main thread without being late to pick up results from the GPU.

My ideal design then would be for the main thread to have a loop whereby it checks to see if there's GPU work to process, and then if there's nothing to do it runs CPU tasks for a bit before checking again. The key is I would really only want it to run a bounded amount of CPU work before returning to check on the GPU again, to have guarantees on how frequently I will check for GPU work again.

From what I can see there's a few options:

1. Keep all of the GPU work out of enkiTS entirely, have my outer loop that looks for GPU work but instead of sleeping when there's nothing to do I instead call WaitforTask(NULL). Hence the above question 😄

2. Rely on the OS scheduler instead of enkiTS's scheduler. Let the main thread do a loop and do a genuine OS sleep in between GPU work, relying on the thread being kicked off the hardware core and add one more task thread to be scheduled at the same time. I'm worried that this could impact the main thread though as now the hardware threads could be oversubscribed, and it depends on when the OS scheduler decides to schedule the main thread again.

3. Set a high priority pinned task on the main thread, and each time it completes it queues a new pinned task on the main thread to be re-run again. The main thread just does WaitForAll or similar and relies on the enkiTS scheduler to sort it out. I think this might technically work but I have a feeling it's going to boil down to a spinloop, since the GPU task will keep being higher priority and I don't think there's anything to stop it getting scheduled again as soon as a new task is added. I don't see a way to represent that sleep/delay before being scheduled again (at least without some explicit dependency on a task in between which I don't have).

4. Have the tasks submitting the GPU work add a new high priority pinned task whenever they add new work, but the challenge here is if the GPU isn't ready I don't want the pinned task to be blocking if there's more CPU work to do - so something needs to get it to check again and I'm not guaranteed any more tasks will submit new GPU work after that.

I may also be missing an obviously better way to do this!

[dougbinks]

TaskScheduler::WaitForTask will only run one task if 0/NULL/nullptr passed instead of an ICompletable, I will update the documentation on that shortly (I'm AFK a lot at the moment so responses and updates will be a little slow for a few days). So this seems like it would match your requirements.

Note that I am thinking of adding a more explicit user callable TryRunTasks function with some more control over what to run as there are a few circumstances where this would help, but it needs more thought.

One option for you might be to do the following:

1. GPU work is issued by a task which has a pinned task on thread 0 (main thread/thread which initialized enkiTS) as a dependent task (or issues it after doing work).
2. Thread 0 uses WaitForNewPinnedTasks to wait for any pinned tasks, then calls RunPinnedTasks which run the tasks and update any data structures needed by thread 0. Note that WaitForNewPinnedTasks can complete with no tasks available, but this should be handled by next step.
3. Thread 0 Checks for data if needed, if it manages to get all data from operations back it goes to 2.
4. With data not ready thread 0 performs a WaitForTask with nullptr and probably TASK_PRIORITY_HIGH or similar where all lengthy tasks are run on lower priorities.
5. If WaitForTask did not take long then sleep (this is where enkiTS could help).
6. Go back to 3.

The main thing enkiTS could add here is that the wait at 5 could be replaced by a WaitForTask with a timeout or similar, which instead of completing immediately would wait with an enkiTS spin then OS sleep but also wake if there were any new tasks/tasks completed using enkiTS internal signals.

It might be worth you first trying the above version (or similar) and then I can create an experimental branch with the above feature to test.

[baldurk]

That's perfect, thanks. Indeed knowing that it only runs one task is the key. It would be nice to have a more flexible way of doing this with a timeout, but it's not critical and at least enables the organisation I want to have.