Print warnings for some bad traces

[aseyboldt]

Right now we don't report anything if a trace if obviously nonsense. This PR prints warnings if a trace does not meet some minimum requirements.
The test about the target acceptance probability is a bit hairy, I don't know any obvious choices for the threshold. The one I implemented works out to printing a warning if:

• If the target accept is 0.99, fail if mean accept is not between 0.96 and 0.9997.
• If target_accept = 0.8, fail if mean accept is not between 0.72, 0.87.
  I'm open for better ideas...

[ColCarroll]

This looks really useful. Neal (https://arxiv.org/pdf/1206.1901.pdf) has some calculations for optimal acceptance rate for Metropolis and HMC/NUTS on page 30. Not sure how wide the band should be around that...

[twiecki]

Very useful. Seems like there are some legit test errors.

[springcoil]

Awesome - useful work.

…

On Fri, Apr 7, 2017 at 8:01 AM, Thomas Wiecki ***@***.***> wrote: Very useful. Seems like there are some legit test errors. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2002 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA8DiPP6LQm7VXyLqzV5uKiU-PLXbQlFks5rtd9kgaJpZM4M2CtL> .

-- Peadar Coyle Skype: springcoilarch www.twitter.com/springcoil peadarcoyle.wordpress.com

[aseyboldt]

I guess this also means we need a way to pass parameters to nuts though pm.sample. It's a bad idea to tell users to increase target_accept, but to do this you need to manually specify your step methods and automatic initialization with advi doesn't work anymore.

[twiecki]

@aseyboldt Yeah, I had this on my mind before, should definitely pass kwargs. Maybe best way is to have step_method_kwargs and init_kwargs (or advi_kwargs?).

[junpenglao]

I guess this also means we need a way to pass parameters to nuts though pm.sample. It's a bad idea to tell users to increase target_accept, but to do this you need to manually specify your step methods and automatic initialization with advi doesn't work anymore.

@aseyboldt isn't that always an option https://github.com/pymc-devs/pymc3/blob/master/pymc3/sampling.py#L491 ?

[aseyboldt]

@junpenglao Yes, but this does not work through pm.sample.

[aseyboldt]

See #2004