Simplified __call__ interface?

[bmcfee]

I've been using pescador a bit over the last few weeks, and find myself wondering if we could tighten up the API a little before the 1.0 release.

Specifically, streamer.generate() is a bit verbose; how about adding a __call__ method so we can instead do something like:

stream = pescador.Streamer(my_generator, my_params...)

for x in stream():
    process(x)

This wouldn't help for cycle or tuples, but it would cut down a bit of boilerplate code for the most common use cases.

[cjacoby]

I've also been using it a lot recently, and I like. 👍

What if we kept the generator() interface internally, and did something like this?

def __call__(self, cycle=False, **kwargs):
    if cycle:
        return self.cycle(**kwargs)
    else:
        return self.generate(**kwargs)

[bmcfee]

I dig it. Tuples are still an oddball, and I'd prefer to keep that one separate.

How about I implement this after merging the doc updates so that everything can be updated in one pass?

[cjacoby]

Roger roger

…

On Sat, Mar 18, 2017 at 1:08 PM Brian McFee ***@***.***> wrote: I dig it. Tuples are still an oddball, and I'd prefer to keep that one separate. How about I implement this after merging the doc updates so that everything can be updated in one pass? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#64 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4t888WA0dKMabIvsm7RYKDHlqiDeitks5rnDnRgaJpZM4Mhhkv> .

[ejhumphrey]

I realize I slept on this issue the first time around and apologize in advance, but I would be keen to revisit this discussion maybe? 😞 🙏

Specifically, I'd propose making core.Streamer directly iterable, e.g. [x for x in stream] is a thing.

Commentary

• This isn't (necessarily) mutually exclusive with [x for x in stream(max_batches=10, cycle=True)] ... though it does violate the "There should be one good way to do a thing" principle.
• My most common use case is consistent with the default args (which leads me to think that it is the common use case) ... which means everyone using pescador will be writing for stuff in stream() all over the place, and those bonus parens smell like a minor anti-pattern.
• Can the cycle and max_batches (or max_samples, re: Language conventions #75) go in the init method? I don't immediately see a reason these would change over the lifetime of a Streamer, but am happy to be educated.

[bmcfee]

Can the cycle and max_batches (or max_samples, re: #75) go in the init method? I don't immediately see a reason these would change over the lifetime of a Streamer, but am happy to be educated.

I think this might cause some difficulties with how streamers get used inside Mux. If you're using poisson-distributed sample limits, you would have a different max_batches each time the streamer is activated. Ditto for cycle, though it's a bit less obvious there.

Given the above, I'm not sure dropping the call interface (or implementing a direct iterable interface) is the best option, but I'm open to persuasion.

[ejhumphrey]

well, here's some simple(r) math: what's the more annoying anti-pattern, a high occurrence of empty parens, or two call signatures for the same functionality?

[cjacoby]

The original reason, if I recall correctly, for not allowing you to __iter__ over streamers directly is language in the Python docs about how __iter__ should behave, with regards to StopIterations. It was an intentional design decision based on those Python definitions. (I'm on my phone or I'd look up the details. I'll get to it in a bit if @bmcfee doesn't beat me to it)

…

On Mon, Apr 3, 2017, 09:57 Eric J. Humphrey ***@***.***> wrote: well, here's some simple(r) math: what's the more annoying anti-pattern, a high occurrence of empty parens, or two call signatures for the same functionality? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#64 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4t86NG3blhquqfSz4nxjwcIclR5BVyks5rsSUVgaJpZM4Mhhkv> .

[ejhumphrey]

hrm, well in that case, I would augment my proposal to:

• Iterating on a Streamer behaves as one would expect (it raises StopIteration and runs to exhaustion), and one of...
• (a) Explicitly calling the streamer (__call__) with args allows the Streamer to augment it's behavior, such as cycle itself or terminate early via max_iter, which I would assume violates PEP conventions around iterators or (b) do we need __call__ when we have both generate() and cycle()?

[cjacoby]

I would be okay with either solution. I think (a) is more elegant, and (b) is more explicit. If I may point back to the original proposal on this Issue, the objective here is to create or add a simplified API for the End-User which is less verbose than `streamer.generate()`. I think I would prefer to actually do both (a) and (b) here, and internally to `Pescador`, or in complicated use cases, prefer the explicit API, but offer a simplified/concise API as a "shortcut", which simply calls the Explicit API under the hood.

…

On Mon, Apr 3, 2017 at 10:19 AM Eric J. Humphrey ***@***.***> wrote: hrm, well in that case, I would augment my proposal to: - Iterating on a Streamer behaves as one would expect (it raises StopIteration and runs to exhaustion), and one of... - *(a)* Explicitly calling the streamer (__call__) with args allows the Streamer to augment it's behavior, such as cycle itself or terminate early via max_iter, which I would assume violates PEP conventions around iterators or *(b)* do we need __call__ when we have both generate() and cycle()? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#64 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4t83HOGl47DwR9I_pHQx5wvtUkqBR1ks5rsSoxgaJpZM4Mhhkv> .

[ejhumphrey]

having thought about it more, I agree with you on 👍 (a) ... a simplified / concise interface for doing non-standard things is nice, and I can imagine that it'd be useful to control these parameters with config-args, rather than augmenting code.

[ejhumphrey]

tiny update: I'm having a really interesting / useful day catching up to pescador. Given the mix of Streamer and generator objects, I would like to emphasize the need for anything that gets referred to as a "stream" to be directly iterable.

[ejhumphrey]

closed with #88