Create Sparsennetdemo.py

[karalleyna]

Closes #210

I have implemented iterative soft-thresholding algorithm in order to obtain Figure 13.21(b) in the book.

I have just changed the data synthesis part of the original code. Could you please review it? @murphyk @mjsML

[murphyk]

It's not clear this is resulting in a sparse NN.
It seems we will need to play with lambda to get a sparse estimate that still fits reasonably.
It would be great to find a way to visualize the graph structure, perhaps using graphviz.
In the meantime, you can use this block to see if the parameter were sparse
'''
for p in params:
if len(p)==0: continue
p = p[0].flatten()
print(['{:0.2f}'.format(x) for x in p])
'''

[karalleyna]

I changed the data synthesis considering the original code and also added the code related to the diagram of sparse neural network. Figures I obtained are

[murphyk]

I cannot reproduce your example - it seems there is some non-determinism somwhere. Maybe just use jax RNG exclusively, and not mix with np.random.

[karalleyna]

I cannot reproduce your example - it seems there is some non-determinism somwhere. Maybe just use jax RNG exclusively, and not mix with np.random.

I removed all np.random in generate_data.