Restricted Boltzmann Machine with CNTK

[nuxai]

Is it possible to build a Restricted Boltzmann Machine for in CNTK?

Are there any "Autoencoders" or Deep Believe Network Models in CNTK?

[zpbappi]

You cannot currently build and train an RBM with CNTK. Reference: #646

There is an autoencoder example in the "Examples\Speech\Miscellaneous\TIMIT\config" directory. Look into "TIMIT_TrainAutoEncoder.cntk" and "ae.ndl" files. In case you are wondering how to pre-train the autoencoder with RBM CD1, it is currently not possible. Please see the issue referenced above for detail.

[arijit17]

@zpbappi Do you know how to run the TIMIT Auto-encoder in CNTK? I have no idea about how to use HTK to prepare the inputs. Do you have a simple autoencoder example that just takes in a matrix (e.g. 2D matrix) and tries to reconstruct the input?

[zpbappi]

@arijit17 The example you are referring to is marked as obsolete. There have been many changes and improvement in CNTK (or, should I say, Microsoft Cognitive Toolkit) since then. I would suggest you use the current version, wich is V2-Beta3. The good thing is, you have a fully working python API support for V2. I am no longer looking at the BrainScript model, just the python. You can have look at the python examples here. They are very thorough. I think the Sequence2Sequence example will give you an idea about how you can implement your network. Moreover, as you mentioned that you have a 2D matrix, please have a look at the NumpyInterop example there as well. That will show how you can use numpy arrays as input to a network designed using CNTK. Hope that helps.

[arijit17]

@zpbappi Thank you for useful tips. With Python, I think it should be now easier to feed in data (e.g. Audio/speech wav files) to CNTK. I don't have to deal with the CNTK "readers", correct?

[n17s]

Yes in Python you can either feed the data yourself or with readers.