Discontinuous Functions

[betanalpha]

What should we do with functions without proper derivatives? Stan includes them and ignores the discontinuities, but this does lead to unintended problems when people try to autodiff through them.

http://www.cplusplus.com/reference/cmath/

• ceil
• floor
• fmod
• trunc
• round
• lround
• llround
• rint
• lrint
• llrint
• nearbyint
• remainder
• remquo
• fdim
• fmax
• fmin
• fabs
• abs
• operator_equal_to
• operator_greater_than_or_equal_to
• operator_greater_than
• operator_less_than_or_equal_to
• operator_less_than
• operator_not_equal_to
• operator_unary_not

[bob-carpenter]

It's not just Stan --- what do you do with the C++
functions that operate on double values if you're
building a complete auto-diff system?

Are all the Stan developers on this list? I'd like
to include everyone in a discussion this major.

• Bob

On Jul 2, 2014, at 2:45 PM, Michael Betancourt notifications@github.com wrote:

What should we do with functions without proper derivatives? Stan includes them and ignores the discontinuities, but this does lead to unintended problems when people try to autodiff through them.

http://www.cplusplus.com/reference/cmath/

• ceil
• [ ]
• [ ]
• [ ]
—
Reply to this email directly or view it on GitHub.

[betanalpha]

I'll create a separate thread but formally we might make difference choices for the two systems so I'm keeping this one open here as well.

[syclik]

My opinion is that if Nomad is going to be standalone, we should provide as much functionality as possible. This means doing making decisions based on auto diff and not based on its applications, like HMC.

So, I'm going to suggest we keep all of the functions listed. Perhaps a compromise could be to set an error flag with "discontinuous function, auto diff not valid" flag that could be queries at the end of reach computation. The burden would then be on the user to check the flag.

[betanalpha]

My opinion is that if Nomad is going to be standalone, we should provide as much functionality as possible. This means doing making decisions based on auto diff and not based on its applications, like HMC.

So, I'm going to suggest we keep all of the functions listed. Perhaps a compromise could be to set an error flag with "discontinuous function, auto diff not valid" flag that could be queries at the end of reach computation. The burden would then be on the user to check the flag.

I am also trying to make an autodiff-only decision, which is why I’m happy not supporting functions that work fine for doubles. It is my opinion that we should not be including functions whose derivatives are not defined everywhere.

I’d also prefer to not make any checking optional. Maybe have an option to turn it off, but I don’t want to make it easy for users to mess up without thinking.

[syclik]

I would find the library extremely frustrating and not easy to use if a lot
of functions that I'm used to in normal C++ don't exist and I have to find
workarounds.

Lots of people I've met want to port old code. I understand that you want
to limit misuse, but that would really hinder the usage of the library.
On Jul 3, 2014 6:43 AM, "Michael Betancourt" notifications@github.com
wrote:

My opinion is that if Nomad is going to be standalone, we should provide
as much functionality as possible. This means doing making decisions based
on auto diff and not based on its applications, like HMC.

So, I'm going to suggest we keep all of the functions listed. Perhaps a
compromise could be to set an error flag with "discontinuous function, auto
diff not valid" flag that could be queries at the end of reach computation.
The burden would then be on the user to check the flag.

I am also trying to make an autodiff-only decision, which is why I’m happy
not supporting functions that work fine for doubles. It is my opinion that
we should not be including functions whose derivatives are not defined
everywhere.

I’d also prefer to not make any checking optional. Maybe have an option to
turn it off, but I don’t want to make it easy for users to mess up without
thinking.

—
Reply to this email directly or view it on GitHub
#2 (comment).

[bob-carpenter]

On Jul 3, 2014, at 8:11 AM, Daniel Lee notifications@github.com wrote:

I would find the library extremely frustrating and not easy to use if a lot
of functions that I'm used to in normal C++ don't exist and I have to find
workarounds.

Lots of people I've met want to port old code. I understand that you want
to limit misuse, but that would really hinder the usage of the library.

I think we all agree that this'll block some uses of the
library --- for example, most of Eigen and Boost wouldn't
compile. For example, we wouldn't have been able to build
prototype autodiff the integrator approach to solving ODEs
or any of our initial matrix functions. Users expect these
things because the autodiff literature is full of them.

[Let's just take it as read that the vast majority of the
scientific literature is written by dimwitted folk who never
properly mastered mathematics or computation.]

The difference of opinion is that Michael thinks
this is a feature whereas Daniel and I consider it a bug.

As to Daniel's suggestion, I do not like raise-an-error flag approaches.
It's problematic in multi-threaded situations and also a pain
to program against. It's a holdover from the C days
before exceptions. What I'd prefer if we want some adjustable
wrench kind of thing is a static control of what's allowed
and what's not --- something like an ENFORCE_STRICT_DIFFERENTIABILITY
flag from the outside and a singleton static config as
in the Boost error handling.

I'm sympathetic to both sides of this whole argument, but I tend
to be more of a Volvo guy, being wary of both
cobbled-together dune buggies or very strictly
toleranced Formula I cars. To complete this analogy,
I think Andrew's on the dune buggy side and Michael on
the Formula I side.

I should write a "what kind of programmer are you"
blog post. Something along the line of Neal Stephenson's
Tank / Batmobile / Ford Taurus essay on operating systems.
Now if only I could write as well as he does...

• Bob

[betanalpha]

Attended to in #11.