Deep GP example: input-connected network?

[finmod]

Putting autograd to the test of the many examples provided, the deep Gaussian process example is based on @duvenaud deep limits (unfortunately on inefficient matllab). More doc at the beginning of deepGP.py should clarify if the deep GP implemented is the pathology-free input-connected network or not.

In addition, it would be great to have a complete example matching with deepGPy of Hensman and Lawrence https://github.com/SheffieldML/deepGPy with the step function example pushed to two or three hidden layers with the corresponding normalized singular values. The example in autograd of "1.4 Fixing the pathology" would really be the ultimate test.

Fantastic work.

[duvenaud]

Thanks for the kind words. The deep GP example isn't input-connected, and in fact is a very rough-proof-of-concept. A student is turning it into a nicer version, which we'll put up once he's done, but neither one implements the full-fledged variational version by the Sheffield group. This demo is more like FITC at each layer.