Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of order dependencies lead to exception #146

Open
mebenstein opened this issue Apr 13, 2023 · 2 comments
Open

Out of order dependencies lead to exception #146

mebenstein opened this issue Apr 13, 2023 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@mebenstein
Copy link

mebenstein commented Apr 13, 2023

Tutorial 1 mentions:

Tasks can be spawned out of order. A tasks dependencies do not need to have been before being listed through the TaskSpace.

When I tried the following:

from parla import Parla
from parla.tasks import spawn, TaskSpace
from parla.cpu import cpu

if __name__ == "__main__":
    print("start")
    with Parla():
        @spawn(placement=cpu)
        async def main_task():
            t = TaskSpace("tasks")

            @spawn(t[1], dependencies=[t[0]],placement=cpu)
            def b():
                print("b")

            @spawn(t[0],placement=cpu)
            def a():
                print("a")


            await t

I get:

ValueError: ('dependent task %s should have been spawned', 'TaskID(tasks_1, task=None)')

Which is caused by the following assertion in task_runtime.py:
if dependent.task is None

It seems that this assertion makes no sense since ComputeTask is only invoked after all the unspawned dependencies are supposed to be added. Thus this assertion can never be true, maybe it should be removed?

@wlruys wlruys self-assigned this Apr 13, 2023
@wlruys wlruys added the bug Something isn't working label Apr 13, 2023
@wlruys
Copy link
Contributor

wlruys commented Apr 13, 2023

I agree, this seems to have been added and not caught in #112. Thanks for raising this!

I'll remove the check and run through the tests to make sure everything is okay.

Also as a heads up, we've been rewriting our runtime from scratch in C++ for Parla 0.3 where we've made these mechanisms a bit more robust than storing waiting tasks in a global dictionary :) We have a few more features to refine before release but it will hopefully be within the next month.

@nicelhc13 nicelhc13 self-assigned this Apr 13, 2023
@wlruys
Copy link
Contributor

wlruys commented Apr 13, 2023

Seems like there's a few mutex conflicts for this case since it hasn't been tested in a while. We're seeing if we can resolve it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants