Missing Scalar Rules

[oxinabox]

We have a lot of scalar rules
A few I know are still missing

• ldexp (WolframAlpha partials)
• clamp see https://discuss.pytorch.org/t/torch-round-gradient/28628/3
• polygamma

Zero everywhere it is differentiable

Idk if we want to close over the point gsps. (We do that for abs)

• round see https://discuss.pytorch.org/t/torch-round-gradient/28628/3
• floor
• ceil

[oxinabox]

Arguably, we want to replace anywhere the derivative is not defined with Zero() or 0.0, if 0.0 is in the subgradient at that point.
Which is the case for things like abs and step functions

[freemin7]

No guarantees:

@scalar_rule(clamp(x,low,high), 
  if x < low
     (Zero(),One(),Zero())
  elseif high < x
     (Zero(), Zero(), One())
  else
     (One(),Zero(),Zero())
  end
)

@scalar_rule(ldexp(x, y), (2^y, Ω*log(2))

Poly gamma seems to be already implemented

[willtebbutt]

@freemin7 if you wouldn't mind turning the above into a PR, that would be fantastic.

[freemin7]

I will turn it into a PR

[freemin7]

See #173
I am not happy with the testing situation. As i found no way to test ldexp as the second argument is enforced to be an Integer which then obviously has no derivative. An Integrator based test would have no problems with discontinuities like that.

Clamp's derivative is undefined at high and low and i had to do a rewrite of my code to satisfy the macro gods.

[oxinabox]

Clamp's derivative is undefined at high and low

It is zero? It is undefined at the edge but we can just chose it to be zero there which is correct enough for anything using subgradients.
Where ever gradient is undefined at a point, and either one side is zero, or sides have opposite sign then can fill the point discontinuality with zero and all is well.

and i had to do a rewrite of my code to satisfy the macro gods.

You do not have to use the macro.