ReverseDiff: Do not always compile tape

[devmotion]

It's nice - and very efficient - to pre-compute and compile tapes with ReverseDiff whenever possible. Unfortunately, if there is some branching it leads to incorrect derivatives. This is even the case for the example in the tests. A simpler example that demonstrates the issue:

julia> using ReverseDiff                                   
                                                           
julia> f(x) = x < zero(x) ? zero(x) : x   
f (generic function with 1 method)   

julia> ReverseDiff.gradient(f ∘ only, [-1.23])
1-element Vector{Float64}:
 0.0

julia> ReverseDiff.gradient(f ∘ only, [1.41])
1-element Vector{Float64}:
 1.0

julia> tape = ReverseDiff.GradientTape(f, [-1.3])
typename(ReverseDiff.GradientTape)(f)

julia> ReverseDiff.gradient!(tape, [1.34])
1-element Vector{Float64}:
 0.0

julia> ReverseDiff.gradient!(tape, [-0.2])
1-element Vector{Float64}:
 0.0

julia> compiledtape = ReverseDiff.compile(tape)
typename(ReverseDiff.CompiledTape)(f)

julia> ReverseDiff.gradient!(tape, [1.34])
1-element Vector{Float64}:
 0.0

julia> ReverseDiff.gradient!(tape, [-0.2])
1-element Vector{Float64}:
 0.0

Thus I suggest to not use compiled tapes by default (alternatively, one could use something like https://github.com/SciML/SciMLSensitivity.jl/blob/4a608f15413e5fdedd9939093d97d5cd982205f6/src/hasbranching.jl to determine if the function has branches but that seems a bit too dependency-heavy for LogDensityProblems, I assume).

The PR adds a (default) option for not compiling the tape and support for ReverseDiff without tapes.

[tpapp]

Thanks for the PR and the very thorough tests. Just requesting a docstring change, then I will merge.

[tpapp]

Can you please document the compile keyword in the docstring above?

[tpapp]

Thanks for the update, I will tag in a minute.