Job dependencies?

[ib00]

Is there a way to create job dependencies and have a more complex job spawning/waiting?
(example: https://github.com/google/marl/blob/main/examples/tasks_in_tasks.cpp)

[slembcke]

So the example you posted is the easiest use case in Tina. Jobs can wait for many other jobs (1:N). What I don't have an API for yet is having multiple jobs wait for a single job (N:1). So far I've found I haven't actually needed it in my hobby game project, or the other places I've used Tina Jobs, so I haven't worried about it.

Example:

static void print_job_name(tina_job* job){
	printf("Job '%s' is done\n", tina_job_get_description(job)->name);
}

static void make_and_wait_for_jobs(tina_job* job){
	tina_group group = {};
	
	// Create a single job and add it to a group.
	tina_scheduler_enqueue(SCHED, "B", print_job_name, NULL, 0, 0, &group);
	
	// Create a batch of jobs and add them to the group too.
	tina_scheduler_enqueue_batch(SCHED, (tina_job_description[]){
		{.func = print_job_name, .name = "C"},
		{.func = print_job_name, .name = "D"},
	}, 2, &group);
	
	
	printf("Waiting for jobs B, C, and D\n");
	tina_job_wait(job, &group, 0);
	printf("Sub-jobs complete.\n");
}

If you want many jobs to wait for many jobs, the easiest modification to make would be to make a generic increment/decrement function for groups. Do a search for group->_count. There are only two places where it's changed. One for increment, and one for decrement. Lock the scheduler and do that. To make many jobs wait for a one, you'll need to make a list of groups. One group for each waiting job, and decrement them all when the waited for job finishes. Though beware, manual retain counting like this is error prone, and the resulting deadlocks or crashes will be quite hard to debug. (That's why I haven't bothered yet)

[slembcke]

Hmm. Still thinking about this:

Though beware, manual retain counting like this is error prone, and the resulting deadlocks or crashes will be quite hard to debug

I feel like I should take that back since I can't think of a reason why it would be worse than old fashioned semaphores or condition variables in that regard. Though I've wasted many hours debugging those. ;)

Though I consider Tina coroutines to be pretty solid, Tina Jobs might be more of a toy than a real tool. It's minimalism works great for my personal projects, but there are plenty of other fiber job libraries out there with more features too. The best thing Tina Jobs has going for it is that it's a tiny little hackable header lib. If it doesn't do what you need to do, feel free to hack on it or find something better.

[ib00]

Thanks!

I think simplicity is good. Tina's API is clean, code easy to follow and it already covers most use cases.

What about a task generating more jobs (in the same group)?
Lock the scheduler, enqueue new jobs, increment the counter and unlock the scheduler?

[slembcke]

Yeah, it's safe to add new jobs to a group from any job or thread. Keep in mind that you can still have race conditions if you add to a group from a job that isn't already in the group. Put another way, you want to make sure the group doesn't finish while you are adding another job to it.

Additionally, you can use a group as a throttle. Normally, the last parameter to tina_job_wait() is zero so that it doesn't wake up until all jobs have completed. So what you can do is to have a job that creates a few sub jobs, waits for some of them to finish, and then fills up the queue again. Repeat until the main job is done. The mandelbrot example does this so the queue doesn't get stuck on just expensive tiles. Then it can process more tiles at once than it has available threads.