-
Notifications
You must be signed in to change notification settings - Fork 396
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rethinking progress #264
Comments
Interesting use case ... the "might not happen" especially. Does anything consume those progress "states"? I guess there are a couple ways it could work. One is to just have each step emit a preset value, like 0.25, 0.5, 0.75, etc. And the last step emit 1.0. If any steps are skipped you still end up with 1.0 at the end, you just don't get intermediate values for things that were skipped. Another is to have each step add In both cases, you kinda end up with a weird code maintenance issue. If you add a new step, you have to adjust the progress values that the other steps emit. Hmmm, or maybe just change the |
Hrm, if you add |
Some time ago I was using progress to emit partial results of the long running operation so that they can be presented to user ASAP. It was by no means just pure number, but a complex object. However there was no sophisticated promise chain in the middle, thus the problem stated is valid of course. I'm just not sure if it will not kill many practical use cases. |
@maciasello partial results are definitely an interesting case ... like an array that isn't full yet, or somesuch. Yeah, need to think about that. Do you have any thoughts on how the two approaches (arbitrary opaque values, and numeric 0.0-1.0 progress) might be used together? What if progress could carry 2 pieces of information instead of 1, a numeric value, for which you get mathematical combining for "free" from the promise impl, and an opaque value that you have to deal with yourself. Does that make any sense? |
@briancavalier its definitely worth consideration. Library support for easing progress measurements seems to be good idea. The approach with 2 pieces carried is fine as well. Do you mean something like checking for existence of a well-known property ( Do you have an idea how to get things correct with such a situation: progressing_promise.then(function() {
// do something
return progressing_promise2;
}).then(null, null, function(progress) {
// we get first set of progresses from progressing_promise from 0.0 up until 1.0
// and then we get second set of progresses from progressing_promise2 starting from 0.0 until 1.0
}) |
@maciasello Honestly, I don't know yet what the best representation of the 2 pieces of data would be. An object like the one you suggested would work. Another option would be 2 parameters, although that might be a little weird in that other promise operations deal with only a single parameter. Definitely have to think carefully about that. Using 2 params, it might look something like: // emitting progress:
return new Promise(function(resolve, reject, notify) {
// do stuff
notify(.5, intermediateState);
// do more stuff
notify(1.0, finalState);
resolve(result);
}); And consuming progress: var p = doAsyncThingWithProgress();
p.progress(function(percentComplete, currentState) {
// percentComplete is a number 0 <= percentComponete <= 1.0
// currentState is any arbitrary value
// what to return here??
}); One obvious question is what the above progress handler should return! Maybe it's allowed to return a new arbitrary/transformed state, but the numbers are always handled automatically. Not sure. A side note: In some cases, if what you have is actually streaming data, then promises aren't necessarily the right fit, and an actual stream (either a data stream, or a discrete event stream) may be a better fit for those situations. In the case of your example, I'm thinking the progress values should be combined using the same averaging approach as p3 = p1.then(function() {
return p2;
});
p3.then(null, null, handleProgress); If you think about it, progress toward completion of p3 is the average of progress toward p1 and progress toward p2, that is, So, I think combining 2 promises either sequentially, or in parallel, when the goal is to fulfill both promises, can be done by applying the same averaging technique. By extension, the same is true for N promises. Does that sound right? If so, that seems pretty cool. |
A side note on data streams: if I get things right, cujo's msgs module |
@sompylasar msgs.js provides a general model for working with messages. While you certainly could model a stream inside a message bus, it's not going to be as efficient as an actual stream. That said, there is strong support in msgs.js for adapting to/from Node Streams, so it's easy to mix and match as needed. |
It feels wrong that progress events have two params when all other promise "events" have one. Maybe I'm just being picky? Anyway, here's an idea for allowing non-numeric objects to be combined: var myProgressState = {
status: "almost-done",
value: 0.95,
valueOf: function () { return this.value; }
}; If when.js's internals ensured that they cast the progress state object to Number, the math would just work. If a state object without The problem, though is what to do with the non-numeric parts of the states when combining them? Sorting to find the max seems easy enough to do. Just sort the state items and choose the one that ends up at the top. But how do you average arbitrary objects together? Finding the median seems easy enough, but I'm guessing median isn't really going to be the right thing much of the time. I know I've told Brian this, but I have convinced myself that events aren't really what promises should provide. Yes, we need two events: onFulfilled and onRejected, but we only need those because of the way that promises must interact with the language. Promises are containers for a future value; the onFulfilled and onRejected events are just an implementation detail. |
@sompylasar msgs.js is a full messaging bus, complete with enterprise integration patterns. That's really way more than was ever intended for promise progress. You can certainly use them together, though, eg putting messages on a bus to indicate progress toward some goal that involves promises, or broadcasting a message when a promise resolves. The intent of promises, as we know them today in JS, is really to be a proxy for a single value, like @unscriptable said, and to give you a programming model that is roughly analogous to synchronous--transforming values, handling errors in a sane way, etc. In that sense, progress, especially arbitrarily complex progress just seems weird. You either have an integer or you don't yet. Of course, promises complicate that analogy precisely because they have a time component. Take an Array, for example. On one hand, you could say that you either have an Array or you don't yet. On the other, though, it's easy to consider a partially filled Array as some sort of indication of "progress". I honestly think that in the "partially filled Array" case, promises are probably not the right thing. Using some sort of async streaming data type, like a most/Stream can be a better fit. If we accept that promises represent an atomic value (even if that atomic value is an Array) that might materialize later, then one obvious question we'd want to answer is: "when will that value be available?". I see a couple fairly obvious ways to answer that in a general sense. One is to allow a promise to provide (or be queried for) some sort of indication of how "close" the value is. A number between 0 and 1 handles that well. Another way is to provide an ETA--an estimate of the actual time that the value is expected to be available. Given one of those, you can compute the other. For example, the ETA of Sometimes one is easier to determine initially, though. For example, determining ETA for Is it ironic that I just used XHR progress to try to argue for simpler promise progress? Perhaps, but I don't think so :) While XHR progress is a complex object, it is not an arbitrary object. It has a known format, which, by definition, arbitrary promise progress events do not. A well-defined promise progress object could be a solution, and it might help with the "passing two parameters" weirdness. The pair of (number, arbitraryProgressThing) could easily be passed as a single argument using an object with well-defined keys, or as a 2-element array. |
Another potentially interesting aspect of using 0.0-1.0 is that progress can be synthesized for operations involving multiple promises where all input promises are expected to fulfill (eg when.join, when.all, when.map, when.reduce), even if the input promises don't provide explicit progress updates. In that case progress you can consider every input promise's progress to be binary: 0.0 for each input promise to start, and 1.0 for an input promise that fulfills. To compute the output promise's progress, just average them as usual. So, when half of the input promises have fulfilled, the output promise's progress is 0.5. I wonder if there are formulae that would work for competitive races like when.any and when.some in the case where the inputs don't provide progress as well ... it's too early and I haven't had enough coffee. |
I'm gonna try to make some time to prototype this later in the week, or maybe over the weekend. It's too late for it to go into when 3.0, and would be a pretty nasty breaking change anyway :) So, we have time to do some experiments. |
It looks like the simple math doesn't quite work out for sequential promises. For example, say you have 3 promises in sequence, like: let p2 = p1.then(...);
let p3 = p2.then(...); Let's say each promise has only binary progress: 0 while pending, 1 at the instant of fulfillment. If you're listening to progress on It's not really clear to me what to do about that. Does it even make sense to try to compute progress for sequential promises that way ... hrm. |
@briancavalier that is what I had in mind when I thought about the example above, however I failed to cover that in the description and your then-response had dulled my vigilance :) |
@maciasello Yeah, that 0.0-1.0 math doesn't work for sequential, but ETA would. With a relative time in ms, In fact, ETA may be better all around than percentage:
So, I still think this is worth exploring. I also still think the current situation is worse, since there's absolutely no way to combine parallel values or races, and a user-defined function is required to handle sequences. Arbitrary values for progress basically means the only time it's safe to use progress is when you are in full control of both producing and consuming the progress values. It's just not a great situation :( |
Progress is clearly worthless right now because, expect for isolated environments, there is no consistency about what progress means. ETA seems particularly tricky. How often do you know how long a promise is expected to take? I like the idea of averaging the progress values, however, some task are inherently more complex than others. What if progress events where a ratio of work steps completed and total work steps. That would make it possible to do a weighted average across an array of promises. In rest.js, there are many places where the promise is a no-op and just passes the value along, while other times it needs to manipulate the object. For the no-op, I can pass the progress values as is, for the transforms, I can increment the number total steps. For an operation like when.all you sum both the work completed and sum the total work for each promise. For when.any you return the progress values for the furthest complete promise, ignoring the other values. when.some is the same as when.any, but take the top x value and sum them like for when.all. |
In some cases, percentage is easier to compute for sure. In others, ETA is easier. Typically you can compute one from the other, tho, so most of the time (all of the time?), either is possible. Either way, though, an estimate is the best you can do for some situations (eg network transfers). For example, ETA is particularly easy for
Percentage and ratio are the same thing :) (after almost finishing this post, I realized you might mean something else here, see below!)
Right, this is documented upthread. But see also the examples upthread of why percentage (ratio) is impossible to compute accurately for sequential situations like |
Promise progress is problematic for several reasons. One big one is that progress values are opaque (they might be numbers, objects, arrays, etc.). A promise library can't combine them in any meaningful way. For example,
when.all
can't combine the progress values from its input promises. It must simply pass them through. This means that the promise returned bywhen.all
might receive heterogenous values: some are objects, some are numbers, etc. and the caller would have to write code to combine them.I would put money on progress being used in 95+% of cases to track I/O operations in order to show the user some sort of progress indicator. There are other uses, but I've rarely seen them. One example is wire.js, which could (but doesn't currently) track progress toward wiring completion, which involves wiring a discrete number of components. Another example is tracking the progress of UI wizard steps (I'm not convinced that's actually a good use case, but I've seen it).
With that in mind, what if progress values were required to be a number between 0 and 1, literally representing "percent complete". That seems to have some interesting properties:
when.all
,when.map
,when.reduce
, etc etc). For 2 promises,combined = (progress1 + progress2) / 2
, and by extension for N promises:combined = sum(progressValues) / progressValues.length
. I believe this works for sequential, and "parallel" tasks.progress2(a, progress2(b, c)) === progress2(progress2(a, b), c)
. which means that progress values from arbitrary operations (all, any, race) etc. can be combined easily, again via averaging:progress2(all(array1), p)
when.any
andwhen.race
->Math.max.apply(progressValues)
(it's a race, so progress can be estimated by the one that appears to be winning)when.some
-> sort progress values in descending order, take top N, combine via average.when.settle
-> same aswhen.al
l?I'm hoping that we can discuss here, and if this makes sense, talk to other promise library maintainers to get their thoughts as well.
The text was updated successfully, but these errors were encountered: