Stream output from jobs

[sethladd]

From customer: "when running something like PubTools.build() doesn’t output anything until the entire job is done. So when a build takes 60 seconds, and there isn’t any sort of output it’s not clear that something is going on. It’s even worse with something like pub serve that doesn’t ever finish so there is never any output."

[devoncarew]

I'll need to think about how to handle this. There are two ways to run processes - sync and async. The sync API returns all the stdout and stderr that happened during process execution, but you don't get access the the I/O as it occurs. So, no streaming.

The async way does get you access to I/O streams. But the async nature of the call would bubble up through all the callers. We'd loose the current simplification that we're doing sync programming and not async.

If I can't figure out way to make this work w/o additional API, I might add explicit named calls (performFooSync and performFooAsync).

[sethladd]

Thanks for taking a look!

[NicholasTuck]

@devoncarew from my experimentation it appears that Grinder does support returning futures from within tasks, so if pub tools returned a future for these things and the task function also returned that or a final future it seems like everything should function. Is this incorrect somewhere?

I only ask as we are still learning Grinder and we want to ensure that returning futures does work as expected/it appears to be working. There may be more complications I am overlooking though. Thanks for the info!

[devoncarew]

@NicholasTuck, you're spot on w/ how returning Futures works in grinder. The current API looks like:

void PubTools.build(...);

and the new API might look something like:

Future PubTools.buildAsync(...);

and the implementation of that task would make sure that it streamed the stdout/err as it occurred. I've preferred keeping the APIs for grinder synchronous in order to reduce the burden on clients of the API. But it makes sense to add a parallel set of APIs available for the streaming functionality, and clients can choose to use other APIs those if they prefer.

[devoncarew]

I added some new async APIs, available in 0.5.7. You can see them used here: https://github.com/google/grinder.dart/blob/master/example/ex2.dart.