Generator op and auto minibatch scaling

[ferrine]

This PR implements auto minibatching with appropriate likelihood scaling. Now generator can be passed to data.

[ferrine]

There is one but significant problem with generators. Any exception within calling __next__ can will break the generator inside theano.function

[twiecki]

I wonder if population is the best name. population_size, total_size?

[ferrine]

I like both population and total_size. total size is probably more intuitive

[twiecki]

Needs good doc-string.

[twiecki]

I don't think the casting should be necessary here?

[twiecki]

Neat!

[twiecki]

This is really neat!

What happens if the generator raises an exception?

[ferrine]

@twiecki then generator dies, I need to handle that case in some way or say user that it is dangerous. I think the latter is the only way I can solve the problem.

[ferrine]

I can check if generator is picklable as such generators don't die when exception is caught

[ferrine]

the second option is here (one more dependency and not always works)

[twiecki]

"generator dies" -- raises another exception, crashes the kernel? how does it look like exactly?

[ferrine]

@twiecki after exception generator raises StopIteration when __next__ is called, it becomes exhausted

[twiecki]

I see, and in general this should never happen, the generators should be non-exhaustive. Thus, when that happens we can raise an exception that says that the generator stopped for unknown reasons and that we can't tell whether it was an exception. That seems acceptable to me.

[ferrine]

KeyboardInterrupt can be annoing

[twiecki]

Well, that's triggered by the user so it should be fairly obvious what happened in that case.

[ferrine]

he will need to recompile the whole model

[twiecki]

I don't think we need to stress over this. The generator will most likely be fast so the probability that the KeyboardInterrupt would happen there is low.

[ferrine]

I'm stressed as risk is potentially high. Imagine DNN as a model. I would be disappointed

[twiecki]

OK, how about we do not raise an exception but just display a warning. Question is what to do next, I suppose the only way to exit gracefully would be to raise a KeyboardInterrupt that will be caught by the inference algorithm which would be super odd. Or we add another Exception case to the inference to exist gracefully.

[twiecki]

Is that a general constraint with generators? That exceptions do not percolate up?

[ferrine]

These exceptions cannot be caught. If they appear inside __next__ it is fail in any case. KeyboardInterrupt is an exception that cannot be avoided

[twiecki]

Well, that's a limitation of the language then. I think we should move ahead with raising an Exception of the generator stops. If it becomes annoying later on we can think of other options. I really don't think it's all that critical.

[ferrine]

Good documentation is needed so

[twiecki]

Agreed, documentation will mention that.

Do you instead want to merge this into your VI branch so that we can test how this would work in practice?

[ferrine]

Yes, I'll add documentation and then merge