Gradient exceptions for ADVI default to having no dropped gradients

[dustinvtran]

Also, even though exceptions are caught now, by default grad_samples is still 1 so a single exception causes it to halt. I think we should either make grad_samples default to 2 or to do one gradient evaluation more than whatever grad_samples is if there are exceptions.

Ben
From stan-dev.

My proposal:

Yeah I noticed that too. I didn’t want to throw down a constant threshold for the max number of dropped gradients at the time. But yeah this needs to be addressed. Thinking a bit more about it now, I’ll just submit a pull request which sets the max number to be 5*grad_samples—the intuition being that we allow 5 dropped gradients per grad_sample, and so that it still scales with the size of the mini batch.