log_sum_exp and log_diff_exp boundaries

[bob-carpenter]

Summary:

The boundary conditions are off for negative infinite values. Currently, we have

log_sum_exp(log(0), log(0)) = nan
log_diff_exp(log(0), log(0)) = nan
log_diff_exp(log(1), log(1)) = nan

and they should all return negative infinity. For example,

log_sum_exp(log(0), log(0))
  = log_sum_exp(-inf, -inf)
  = log(exp(-inf) + exp(-inf))
  = log(0 + 0)
  = -inf

Now some care is going to have to be taken to manage the derivatives. We may not be able to chain rule through the resulting negative infinities for an expression like exp(log_sum_exp(log(0), log(0))). The more doc explaining what's going on in the math lib (and Stan doc itself), the better.

Originally reported by Dalton Hance on Discourse:

http://discourse.mc-stan.org/t/log-0-and-stan-arithmetic/2171

Current Version:

v2.17.0

[aggarwalvinayak]

I would like to work on this issue. Can I be assigned this bug?

[bob-carpenter]

@aggarwalvinayak Thanks. The first thing to do is update the issue to propose a solution---it wasn't clear to me what to do with derivatives when writing the issue. Even if they blow up, it would be good to fix this to get the right values.

[aggarwalvinayak]

@bob-carpenter Sure i would look into fixing this to get the right values and will also see if i can fix the derivative part too... This is my first time i am contributing to an open source project. So could you help me get started with it. i have cloned it and found the files containing the log_sum_exp and log_diff_exp declaration. what should i now look into to begin with fixing this

[bob-carpenter]

Our process is described here: https://github.com/stan-dev/stan/wiki/Developer-process-overview It's fairly standard for a medium-sized open-source project. I'm not sure the derivatives can be fixed. That's an analytic question as much as a computational one.

…

On Jun 19, 2018, at 8:20 AM, Vinayak Aggarwal ***@***.***> wrote: @bob-carpenter Sure i would look into fixing this to get the right values and will also see if i can fix the derivative part too... This is my first time i am contributing to an open source project. So could you help me get started with it. i have cloned it and found the files containing the log_sum_exp and log_diff_exp declaration. what should i now look into to begin with fixing this — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[aggarwalvinayak]

I did look into the implementation of the function but i dont think it can be improved any further. as the problem is not of the compiler not treating log(0) as -inf instead log(0) returns a -inf value and our log_sum_exp evaluates it as expected but it is unfortunately printed as NaN for the required -inf value. But i am not completely sure about it.. Could something else be responsible for this?

[sakrejda]

The way forward is likely to check for zeros in log_* functions and immediately provide the correct result rather than letting the zeros propagate

[andrjohns]

Closed via #1290