Deprecate initialiser keyword arguments

[MikeInnes]

These don't really work, they aren't flexible enough and are no more convenient that just supplying your own weights e.g.

Dense(randn(2, 5), rand(2))

This is partly a docs issue as people don't realise that this is a supported option. Although like most things, it will of course get more convenient with #669.

[shreyas-kowshik]

I am willing to work on this.
But I need to clarify a few things beforehand :

using Flux
using Distributions:Normal

dist = Normal(0.0,1.0)
function initW(out,in)
    W = rand(dist,out,in)
end

Dense(2,3,sigmoid,initW=initW,initb=zeros)

This piece of code does work right?...As in, we can define our own functions for weight initialization right?...So why is there a need to depreciate it?...
I apologize if I'm being silly here. In case if this is not what is relevant to this issue, then can you please explain what needs to be done exactly?

[MikeInnes]

The original idea was that you could choose initilialisers for a large model with the init kwarg (e.g. ResNet50(init = xorot_uniform). But we ended up using different initialisers on a per-parameters basis (initW and initb) which defeats the purpose, since now you have to deal with every weight separately anyway. On top of that, this extra API must be explicitly added to each new layer, which makes it harder to achieve consistency (most newer layers don't have it at all).

So given those issues and the fact that you can just as easily write this without initW, we can simplify things a fair bit by just removing it.

[shreyas-kowshik]

So all that needs to be done is to add deprecation warnings to Dense(...)...?
Also for the warning, is this message fine :

Dense(in,out,act,initW,initb) is deprecated; use Dense(in,out) instead

OR is this required...? :

Dense(in,out,act,initW,initb) is deprecated; use Dense(in,out,init) instead

Accordingly I shall make changes to the API then.

Can you please clarify it a bit here @MikeInnes.
Thank You.

[johnnychen94]

Flux.jl/src/layers/basic.jl

Lines 65 to 71 in 7418a2d

	struct Dense{F,S,T}
	W::S
	b::T
	σ::F
	end
	Dense(W, b) = Dense(W, b, identity)

This is my understanding:
Dense(in,out,act,initW,initb) is deprecated; use Dense(in,out,σ) or Dense(W, b, σ) instead

[shreyas-kowshik]

@MikeInnes added the deprecation warning in #722 .