Drop Test time dependency on FiniteDifferences

[oxinabox]

Conceptually it doesn't make sense that we have a test time dependency on FiniteDifferences.
We don't actually have any rules.

Having it causes problems in being able to do JuliaDiff/ChainRulesTestUtils.jl#63
Especially because FiniteDifferences has an actual dependency on ChainRulesCore.
So we have an almost circular dependency going on

[oxinabox]

I think we just need to delete the tests that do it,
and remove the dependency.
Those tests are already in ChainRules.jl itself.

[oxinabox]

closed by #228