2017-03-24. Important Note breaking changes There are breaking changes in master compared to beta15, click here for more information.
2017-03-15. V 2.0 Beta 15 Release
Highlights of this Release:
- Added support for TensorBoard output in BrainScript. Read more here.
- Learners can now be implemented in pure Python by means of
UserLearners. Read more here.
- New debugging helpers:
- Tensors can be indexed using advanced indexing. E.g.
x[[0,2,3]]would return a tensor that contains the first, third and fourth element of the first axis.
- Significant updates in the Layers Library of Pythin API. See Release Notes for detailed description.
- Updates and new examples in C# API.
- Various bug fixes.
2017-02-23. V 2.0 Beta 12 Release
Highlights of this Release:
- New and updated features: new activation functions, support of
Argmin, improved performance of
numpyinterop, new functionality of existing operators, and more.
- CNTK for CPU on Windows can now be installed via
pip installon Anaconda 3. Other configurations will be enabled soon.
- HTK deserializers are now exposed in Python. All deserializers are exposed in C++.
- The memory pool implementation of CNTK has been updated with a new global optimization algorithm. Hyper memory compression has been removed.
- New features in C++ API.
- New Eval examples for RNN models.
- New CNTK NuGet Packages with CNTK V2 C++ Library.
See all news.
What is The Microsoft Cognitive Toolkit
The Microsoft Cognitive Toolkit (https://www.microsoft.com/en-us/research/product/cognitive-toolkit/), is a unified deep-learning toolkit that describes neural networks as a series of computational steps via a directed graph. In this directed graph, leaf nodes represent input values or network parameters, while other nodes represent matrix operations upon their inputs. CNTK allows to easily realize and combine popular model types such as feed-forward DNNs, convolutional nets (CNNs), and recurrent networks (RNNs/LSTMs). It implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers. CNTK has been available under an open-source license since April 2015. It is our hope that the community will take advantage of CNTK to share ideas more quickly through the exchange of open source working code.
License: See LICENSE.md in the root of this repository for the full license information.
- Microsoft Computational Network Toolkit offers most efficient distributed deep learning computational performance
- Microsoft researchers win ImageNet computer vision challenge (December 2015)
Cognitive Toolkit (CNTK) provides significant performance gains compared to other toolkits click here for details. Here is a summary of findings by researchers at HKBU.
- CNTK’s LSTM performance is 5-10x faster than the other toolkits.
- For convolution (image tasks), CNTK is comparable, but note the authors were using CNTK 1.7.2, and current CNTK 2.0 beta 10 is over 30% faster than 1.7.2.
- For all networks, CTNK's performance was superior to TensorFlow performance.
CNTK was a pioneer in introducing scalability across multi-server multi-GPU systems. The figure below compares processing speed (frames processed per second) of CNTK to that of four other well-known toolkits. The configuration uses a fully connected 4-layer neural network (see our benchmark scripts) and an effective mini batch size (8192). All results were obtained on the same hardware with the respective latest public software versions as of Dec 3, 2015.
If you used this toolkit or part of it to do your research, please cite the work as:
Amit Agarwal, Eldar Akchurin, Chris Basoglu, Guoguo Chen, Scott Cyphers, Jasha Droppo, Adam Eversole, Brian Guenter, Mark Hillebrand, T. Ryan Hoens, Xuedong Huang, Zhiheng Huang, Vladimir Ivanov, Alexey Kamenev, Philipp Kranen, Oleksii Kuchaiev, Wolfgang Manousek, Avner May, Bhaskar Mitra, Olivier Nano, Gaizka Navarro, Alexey Orlov, Hari Parthasarathi, Baolin Peng, Marko Radmilac, Alexey Reznichenko, Frank Seide, Michael L. Seltzer, Malcolm Slaney, Andreas Stolcke, Huaming Wang, Yongqiang Wang, Kaisheng Yao, Dong Yu, Yu Zhang, Geoffrey Zweig (in alphabetical order), "An Introduction to Computational Networks and the Computational Network Toolkit", Microsoft Technical Report MSR-TR-2014-112, 2014.
CNTK is in active use at Microsoft and constantly evolving. There will be bugs.