Deep Learning Library: pydbm_mxnet
pydbm_mxnet is Python library based on
MXNet for building restricted boltzmann machine, deep boltzmann machine, and multi-layer neural networks.
This library is derived from pydbm (Cython version).
The function of this library is building and modeling restricted boltzmann machine, deep boltzmann machine, and multi-layer neural networks. The models are functionally equivalent to stacked auto-encoder. The main function is the same as dimensions reduction(or pre-training).
In relation to my Automatic Summarization Library, it is important for me that the models are functionally equivalent to stacked auto-encoder. The main function I observe is the same as dimensions reduction(or pre-training). But the functional reusability of the models can be not limited to this. These Python Scripts can be considered a kind of experiment result to verify effectiveness of object-oriented analysis, object-oriented design, and GoF's design pattern in designing and modeling neural network, deep learning, and reinforcement-Learning.
For instance, dbm_multi_layer_builder.py is implemented for running the deep boltzmann machine to extract so-called feature points. This script is premised on a kind of builder pattern for separating the construction of complex restricted boltzmann machines from its graph representation so that the same construction process can create different representations. Because of common design pattern and polymorphism, the stacked auto-encoder in demo_stacked_auto_encoder.py is functionally equivalent to deep boltzmann machine.
Full documentation is available on https://code.accel-brain.com/Deep-Learning-by-means-of-Design-Pattern/ . This document contains information on functionally reusability, functional scalability and functional extensibility.
Install using pip:
pip install pydbm_mxnet
The source code is currently hosted on GitHub.
Python package index(PyPI)
Installers for the latest released version are available at the Python package index.
- numpy: v1.13.3 or higher.
- mxnet: latest.
Usecase: Building the deep boltzmann machine for feature extracting.
Import Python and modules.
# The `Client` in Builder Pattern from pydbm_mxnet.dbm.deep_boltzmann_machine import DeepBoltzmannMachine # The `Concrete Builder` in Builder Pattern. from pydbm_mxnet.dbm.builders.dbm_multi_layer_builder import DBMMultiLayerBuilder # Contrastive Divergence for function approximation. from pydbm_mxnet.approximation.contrastive_divergence import ContrastiveDivergence # Logistic Function as activation function. from pydbm_mxnet.activation.logistic_function import LogisticFunction # ReLu function as activation function. from pydbm_mxnet.activation.relu_function import ReLuFunction # Tanh function as activation function. from pydbm_mxnet.activation.tanh_function import TanhFunction
Instantiate objects and call the method.
dbm = DeepBoltzmannMachine( DBMMultiLayerBuilder(), # Dimention in visible layer, hidden layer, and second hidden layer. [traning_x.shape, 10, traning_x.shape], [ReLuFunction(), LogisticFunction(), TanhFunction()], # Setting objects for activation function. ContrastiveDivergence(), # Setting the object for function approximation. 0.05, # Setting learning rate. 0.5 # Setting dropout rate. ) # Execute learning. dbm.learn(traning_arr, traning_count=1000)
And the feature points can be extracted by this method.
Usecase: Extracting all feature points for dimensions reduction(or pre-training)
Import Python modules.
# `StackedAutoEncoder` is-a `DeepBoltzmannMachine`. from pydbm_mxnet.dbm.deepboltzmannmachine.stacked_auto_encoder import StackedAutoEncoder # The `Concrete Builder` in Builder Pattern. from pydbm_mxnet.dbm.builders.dbm_multi_layer_builder import DBMMultiLayerBuilder # Contrastive Divergence for function approximation. from pydbm_mxnet.approximation.contrastive_divergence import ContrastiveDivergence # Logistic function as activation function. from pydbm_mxnet.activation.logistic_function import LogisticFunction
Instantiate objects and call the method.
dbm = StackedAutoEncoder( DBMMultiLayerBuilder(), [target_arr.shape, 10, target_arr.shape], [LogisticFunction(), LogisticFunction(), LogisticFunction()], ContrastiveDivergence(), 0.0005, 0.25 ) # Execute learning. dbm.learn(traning_arr, traning_count=1)
And the result of dimention reduction can be extracted by this property.
pre_trained_arr = dbm.feature_points_arr
If you want to get the pre-training weights, call
weight_arr_list = dbm.get_weight_arr_list()
weight_arr_list is the
list of weights of each links in DBM.
weight_arr_list is 2-d
np.ndarray of weights between visible layer and first hidden layer.
- Webクローラ型人工知能によるパラドックス探索暴露機能の社会進化論 (Japanese)
- 深層強化学習のベイズ主義的な情報探索に駆動された自然言語処理の意味論 (Japanese)
- ハッカー倫理に準拠した人工知能のアーキテクチャ設計 (Japanese)
- GNU General Public License v2.0
- Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive science, 9(1), 147-169.
- Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural computation, 14(8), 1771-1800.
- Le Roux, N., & Bengio, Y. (2008). Representational power of restricted Boltzmann machines and deep belief networks. Neural computation, 20(6), 1631-1649.
- Salakhutdinov, R., & Hinton, G. E. (2009). Deep boltzmann machines. InInternational conference on artificial intelligence and statistics (pp. 448-455).