It appears that Dask Futures always use pending as the default value, which seems inappropriate.
In my use case, I have a Flask service with two routes:
/submit starts a long-running (several minutes-long) Dask job. This returns a Future key (plus some additional info relating to the request)./status/<key> that, given key will check the status of the job from step 1.
The result from step 1 is always a key with a status of pending (since the job is just starting when my Flask service returns its JSON result). When I pass this key to the status request, I also always get a pending status back -- even if the job has completed (verified by finding the result file that step 1 creates).
I have two concerns here:
-
If reconstruct a Future from a known valid key, the Dask client doesn't report the status as finished - this seems like a bug to me. If there's a proper way to do this, I'd love to find out. From the Python REPL, this does seem to work; from a deployed application, it doesn't.
-
If I construct a Future from an invalid key (eg, fut = Future("this_key_doesnt_exist")), then my Future's status shouldn't default to pending but perhaps invalid. To me, pending implies the job is running - especially since, from the REPL, a Future starts as pending and moves to finished. Perhaps an interrmediate state explicitly indicating the job is running or executing?
It appears that Dask Futures always use pending as the default value, which seems inappropriate.
In my use case, I have a Flask service with two routes:
/submitstarts a long-running (several minutes-long) Dask job. This returns a Future key (plus some additional info relating to the request)./status/<key>that, givenkeywill check the status of the job from step 1.The result from step 1 is always a key with a status of pending (since the job is just starting when my Flask service returns its JSON result). When I pass this key to the status request, I also always get a pending status back -- even if the job has completed (verified by finding the result file that step 1 creates).
I have two concerns here:
If reconstruct a
Futurefrom a known valid key, the Dask client doesn't report the status asfinished- this seems like a bug to me. If there's a proper way to do this, I'd love to find out. From the Python REPL, this does seem to work; from a deployed application, it doesn't.If I construct a
Futurefrom an invalid key (eg,fut = Future("this_key_doesnt_exist")), then my Future's status shouldn't default to pending but perhaps invalid. To me, pending implies the job is running - especially since, from the REPL, a Future starts as pending and moves to finished. Perhaps an interrmediate state explicitly indicating the job is running or executing?