/
RELEASE_NOTES
311 lines (256 loc) · 14.3 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
Release Notes - SINGA - Version singa-incubating-1.1.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-296] Add sign and to_host function for pysinga tensor module
* Model components
* [SINGA-254] Implement Adam for V1
* [SINGA-264] Extend the FeedForwardNet to accept multiple inputs
* [SINGA-267] Add spatial mode in batch normalization layer
* [SINGA-271] Add Concat and Slice layers
* [SINGA-275] Add Cross Entropy Loss for multiple labels
* [SINGA-278] Convert trained caffe parameters to singa
* [SINGA-287] Add memory size check for cudnn convolution
* Utility functions and CI
* [SINGA-242] Compile all source files into a single library.
* [SINGA-244] Separating swig interface and python binding files
* [SINGA-246] Imgtool for image augmentation
* [SINGA-247] Add windows support for singa
* [SINGA-251] Implement image loader for pysinga
* [SINGA-252] Use the snapshot methods to dump and load models for pysinga
* [SINGA-255] Compile mandatory depedent libaries together with SINGA code
* [SINGA-259] Add maven pom file for building java classes
* [SINGA-261] Add version ID into the checkpoint files
* [SINGA-266] Add Rafiki python toolkits
* [SINGA-273] Improve license and contributions
* [SINGA-284] Add python unittest into Jenkins and link static libs into whl file
* [SINGA-280] Jenkins CI support
* [SINGA-288] Publish wheel of PySINGA generated by Jenkins to public servers
* Documentation and usability
* [SINGA-263] Create Amazon Machine Image
* [SINGA-268] Add IPython notebooks to the documentation
* [SINGA-276] Create docker images
* [SINGA-289] Update SINGA website automatically using Jenkins
* [SINGA-295] Add an example of image classification using GoogleNet
* Bugs fixed
* [SINGA-245] float as the first operand can not multiply with a tensor object
* [SINGA-293] Bug from compiling PySINGA on Mac OS X with multiple version of Python
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.0.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* Core abstractions including Tensor and Device
* [SINGA-207] Update Tensor functions for matrices
* [SINGA-205] Enable slice and concatenate operations for Tensor objects
* [SINGA-197] Add CNMem as a submodule in lib/
* [SINGA-196] Rename class Blob to Block
* [SINGA-194] Add a Platform singleton
* [SINGA-175] Add memory management APIs and implement a subclass using CNMeM
* [SINGA-173] OpenCL Implementation
* [SINGA-171] Create CppDevice and CudaDevice
* [SINGA-168] Implement Cpp Math functions APIs
* [SINGA-162] Overview of features for V1.x
* [SINGA-165] Add cross-platform timer API to singa
* [SINGA-167] Add Tensor Math function APIs
* [SINGA-166] light built-in logging for making glog optional
* [SINGA-164] Add the base Tensor class
* IO components for file read/write, network and data pre-processing
* [SINGA-233] New communication interface
* [SINGA-215] Implement Image Transformation for Image Pre-processing
* [SINGA-214] Add LMDBReader and LMDBWriter for LMDB
* [SINGA-213] Implement Encoder and Decoder for CSV
* [SINGA-211] Add TextFileReader and TextFileWriter for CSV files
* [SINGA-210] Enable checkpoint and resume for v1.0
* [SINGA-208] Add DataIter base class and a simple implementation
* [SINGA-203] Add OpenCV detection for cmake compilation
* [SINGA-202] Add reader and writer for binary file
* [SINGA-200] Implement Encoder and Decoder for data pre-processing
* Module components including layer classes, training algorithms and Python binding
* [SINGA-235] Unify the engines for cudnn and singa layers
* [SINGA-230] OpenCL Convolution layer and Pooling layer
* [SINGA-222] Fixed bugs in IO
* [SINGA-218] Implementation for RNN CUDNN version
* [SINGA-204] Support the training of feed-forward neural nets
* [SINGA-199] Implement Python classes for SGD optimizers
* [SINGA-198] Change Layer::Setup API to include input Tensor shapes
* [SINGA-193] Add Python layers
* [SINGA-192] Implement optimization algorithms for SINGA v1 (nesterove, adagrad, rmsprop)
* [SINGA-191] Add "autotune" for CudnnConvolution Layer
* [SINGA-190] Add prelu layer and flatten layer
* [SINGA-189] Generate python outputs of proto files
* [SINGA-188] Add Dense layer
* [SINGA-187] Add popular parameter initialization methods
* [SINGA-186] Create Python Tensor class
* [SINGA-184] Add Cross Entropy loss computation
* [SINGA-183] Add the base classes for optimizer, constraint and regularizer
* [SINGA-180] Add Activation layer and Softmax layer
* [SINGA-178] Add Convolution layer and Pooling layer
* [SINGA-176] Add loss and metric base classes
* [SINGA-174] Add Batch Normalization layer and Local Response Nomalization layer.
* [SINGA-170] Add Dropout layer and CudnnDropout layer.
* [SINGA-169] Add base Layer class for V1.0
* Examples
* [SINGA-232] Alexnet on Imagenet
* [SINGA-231] Batchnormlized VGG model for cifar-10
* [SINGA-228] Add Cpp Version of Convolution and Pooling layer
* [SINGA-227] Add Split and Merge Layer and add ResNet Implementation
* Documentation
* [SINGA-239] Transfer documentation files of v0.3.0 to github
* [SINGA-238] RBM on mnist
* [SINGA-225] Documentation for installation and Cifar10 example
* [SINGA-223] Use Sphinx to create the website
* Tools for compilation and some utility code
* [SINGA-229] Complete install targets
* [SINGA-221] Support for Travis-CI
* [SINGA-217] build python package with setup.py
* [SINGA-216] add jenkins for CI support
* [SINGA-212] Disable the compilation of libcnmem if USE_CUDA is OFF
* [SINGA-195] Channel for sending training statistics
* [SINGA-185] Add CBLAS and GLOG detection for singav1
* [SINGA-181] Add NVCC supporting for .cu files
* [SINGA-177] Add fully cmake supporting for the compilation of singa_v1
* [SINGA-172] Add CMake supporting for Cuda and Cudnn libs
----------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-0.3.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* GPU Support
* [SINGA-131] Implement and optimize hybrid training using both CPU and GPU
* [SINGA-136] Support cuDNN v4
* [SINGA-134] Extend SINGA to run over a GPU cluster
* [SINGA-157] Change the priority of cudnn library and install libsingagpu.so
* Remove Dependencies
* [SINGA-156] Remove the dependency on ZMQ for single process training
* [SINGA-155] Remove zookeeper for single-process training
* Python Binding
* [SINGA-126] Python Binding for Interactive Training
* Other Improvements
* [SINGA-80] New Blob Level and Address Level Math Operation Interface
* [SINGA-130] Data Prefetching
* [SINGA-145] New SGD based optimization Updaters: AdaDelta, Adam, AdamMax
* Bugs Fixed
* [SINGA-148] Race condition between Worker threads and Driver
* [SINGA-150] Mesos Docker container failed
* [SIGNA-141] Undesired Hash collision when locating process id to worker…
* [SINGA-149] Docker build fail
* [SINGA-143] The compilation cannot detect libsingagpu.so file
-----------------------------------------
Release Notes - SINGA - Version singa-incubating-0.2.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is
designed with an intuitive programming model based on the layer abstraction. SINGA supports a wide variety of popular
deep learning models.
This release includes following features:
* Programming model
* [SINGA-80] New Blob Level and Address Level Math Operation Interface
* [SINGA-82] Refactor input layers using data store abstraction
* [SINGA-87] Replace exclude field to include field for layer configuration
* [SINGA-110] Add Layer member datavec_ and gradvec_
* [SINGA-120] Implemented GRU and BPTT (BPTTWorker)
* Neuralnet layers
* [SINGA-91] Add SoftmaxLayer and ArgSortLayer
* [SINGA-106] Add dummy layer for test purpose
* [SINGA-120] Implemented GRU and BPTT (GRULayer and OneHotLayer)
* GPU training support
* [SINGA-100] Implement layers using CUDNN for GPU training
* [SINGA-104] Add Context Class
* [SINGA-105] Update GUN make files for compiling cuda related code
* [SINGA-98] Add Support for AlexNet ImageNet Classification Model
* Model/Hybrid partition
* [SINGA-109] Refine bridge layers
* [SINGA-111] Add slice, concate and split layers
* [SINGA-113] Model/Hybrid Partition Support
* Python binding
* [SINGA-108] Add Python wrapper to singa
* Predict-only mode
* [SINGA-85] Add functions for extracting features and test new data
* Integrate with third-party tools
* [SINGA-11] Start SINGA on Apache Mesos
* [SINGA-78] Use Doxygen to generate documentation
* [SINGA-89] Add Docker support
* Unit test
* [SINGA-95] Add make test after building
* Other improvment
* [SINGA-84] Header Files Rearrange
* [SINGA-93] Remove the asterisk in the log tcp://169.254.12.152:*:49152
* [SINGA-94] Move call to google::InitGoogleLogging() from Driver::Init() to main()
* [SINGA-96] Add Momentum to Cifar10 Example
* [SINGA-101] Add ll (ls -l) command in .bashrc file when using docker
* [SINGA-114] Remove short logs in tmp directory
* [SINGA-115] Print layer debug information in the neural net graph file
* [SINGA-118] Make protobuf LayerType field id easy to assign
* [SIGNA-97] Add HDFS Store
* Bugs fixed
* [SINGA-85] Fix compilation errors in examples
* [SINGA-90] Miscellaneous trivial bug fixes
* [SINGA-107] Error from loading pre-trained params for training stacked RBMs
* [SINGA-116] Fix a bug in InnerProductLayer caused by weight matrix sharing
-----------------------------------------
Features included in singa-incubating-0.1.0:
* Job management
* [SINGA-3] Use Zookeeper to check stopping (finish) time of the system
* [SINGA-16] Runtime Process id Management
* [SINGA-25] Setup glog output path
* [SINGA-26] Run distributed training in a single command
* [SINGA-30] Enhance easy-to-use feature and support concurrent jobs
* [SINGA-33] Automatically launch a number of processes in the cluster
* [SINGA-34] Support external zookeeper service
* [SINGA-38] Support concurrent jobs
* [SINGA-39] Avoid ssh in scripts for single node environment
* [SINGA-43] Remove Job-related output from workspace
* [SINGA-56] No automatic launching of zookeeper service
* [SINGA-73] Refine the selection of available hosts from host list
* Installation with GNU Auto tool
* [SINGA-4] Refine thirdparty-dependency installation
* [SINGA-13] Separate intermediate files of compilation from source files
* [SINGA-17] Add root permission within thirdparty/install.
* [SINGA-27] Generate python modules for proto objects
* [SINGA-53] Add lmdb compiling options
* [SINGA-62] Remove building scrips and auxiliary files
* [SINGA-67] Add singatest into build targets
* Distributed training
* [SINGA-7] Implement shared memory Hogwild algorithm
* [SINGA-8] Implement distributed Hogwild
* [SINGA-19] Slice large Param objects for load-balance
* [SINGA-29] Update NeuralNet class to enable layer partition type customization
* [SINGA-24] Implement Downpour training framework
* [SINGA-32] Implement AllReduce training framework
* [SINGA-57] Improve Distributed Hogwild
* Training algorithms for different model categories
* [SINGA-9] Add Support for Restricted Boltzman Machine (RBM) model
* [SINGA-10] Add Support for Recurrent Neural Networks (RNN)
* Checkpoint and restore
* [SINGA-12] Support Checkpoint and Restore
* Unit test
* [SINGA-64] Add the test module for utils/common
* Programming model
* [SINGA-36] Refactor job configuration, driver program and scripts
* [SINGA-37] Enable users to set parameter sharing in model configuration
* [SINGA-54] Refactor job configuration to move fields in ModelProto out
* [SINGA-55] Refactor main.cc and singa.h
* [SINGA-61] Support user defined classes
* [SINGA-65] Add an example of writing user-defined layers
* Other features
* [SINGA-6] Implement thread-safe singleton
* [SINGA-18] Update API for displaying performance metric
* [SINGA-77] Integrate with Apache RAT
Some bugs are fixed during the development of this release
* [SINGA-2] Check failed: zsock_connect
* [SINGA-5] Server early terminate when zookeeper singa folder is not initially empty
* [SINGA-15] Fixg a bug from ConnectStub function which gets stuck for connecting layer_dealer_
* [SINGA-22] Cannot find openblas library when it is installed in default path
* [SINGA-23] Libtool version mismatch error.
* [SINGA-28] Fix a bug from topology sort of Graph
* [SINGA-42] Issue when loading checkpoints
* [SINGA-44] A bug when reseting metric values
* [SINGA-46] Fix a bug in updater.cc to scale the gradients
* [SINGA-47] Fix a bug in data layers that leads to out-of-memory when group size is too large
* [SINGA-48] Fix a bug in trainer.cc that assigns the same NeuralNet instance to workers from diff groups
* [SINGA-49] Fix a bug in HandlePutMsg func that sets param fields to invalid values
* [SINGA-66] Fix bugs in Worker::RunOneBatch function and ClusterProto
* [SINGA-79] Fix bug in singatool that can not parse -conf flag
Features planned for the next release
* [SINGA-11] Start SINGA using Mesos
* [SINGA-31] Extend Blob to support xpu (cpu or gpu)
* [SINGA-35] Add random number generators
* [SINGA-40] Support sparse Param update
* [SINGA-41] Support single node single GPU training