/
RELEASE_NOTES
410 lines (339 loc) · 18.9 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
Release Notes - SINGA - Version singa-incubating-2.0.0
SINGA is a general distributed deep learning platform for training big deep
learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-434] Support tensor broadcasting
* [SINGA-370] Improvement to tensor reshape and various misc. changes related to SINGA-341 and 351
* Model components
* [SINGA-333] Add support for Open Neural Network Exchange (ONNX) format
* [SINGA-385] Add new python module for optimizers
* [SINGA-394] Improve the CPP operations via Intel MKL DNN lib
* [SINGA-425] Add 3 operators , Abs(), Exp() and leakyrelu(), for Autograd
* [SINGA-410] Add two function, set_params() and get_params(), for Autograd Layer class
* [SINGA-383] Add Separable Convolution for autograd
* [SINGA-388] Develop some RNN layers by calling tiny operations like matmul, addbias.
* [SINGA-382] Implement concat operation for autograd
* [SINGA-378] Implement maxpooling operation and its related functions for autograd
* [SINGA-379] Implement batchnorm operation and its related functions for autograd
* Utility functions and CI
* [SINGA-432] Update depdent lib versions in conda-build config
* [SINGA-429] Update docker images for latest cuda and cudnn
* [SINGA-428] Move Docker images under Apache user name
* Documentation and usability
* [SINGA-395] Add documentation for autograd APIs
* [SINGA-344] Add a GAN example
* [SINGA-390] Update installation.md
* [SINGA-384] Implement ResNet using autograd API
* [SINGA-352] Complete SINGA documentation in Chinese version
* Bugs fixed
* [SINGA-431] Unit Test failed - Tensor Transpose
* [SINGA-422] ModuleNotFoundError: No module named "_singa_wrap"
* [SINGA-418] Unsupportive type 'long' in python3.
* [SINGA-409] Basic `singa-cpu` import throws error
* [SINGA-408] Unsupportive function definition in python3
* [SINGA-380] Fix bugs from Reshape
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.2.0
SINGA is a general distributed deep learning platform for training big deep
learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-290] Upgrade to Python 3
* [SINGA-341] Added stride functionality to tensors for CPP
* [SINGA-347] Create a function that supports einsum
* [SINGA-351] Added stride support and cudnn codes to cuda
* Model components
* [SINGA-300] Add residual networks for imagenet classification
* [SINGA-312] Rename layer parameters
* [SINGA-313] Add L2 norm layer
* [SINGA-315] Reduce memory footprint by Python generator for parameter
* [SINGA-316] Add SigmoidCrossEntropy
* [SINGA-324] Extend RNN layer to accept variant seq length across batches
* [SINGA-326] Add Inception V4 for ImageNet classification
* [SINGA-328] Add VGG models for ImageNet classification
* [SINGA-329] Support layer freezing during training (fine-tuning)
* [SINGA-346] Update cudnn from V5 to V7
* [SINGA-349] Create layer operations for autograd
* [SINGA-363] Add DenseNet for Imagenet classification
* Utility functions and CI
* [SINGA-274] Improve Debian packaging with CPack
* [SINGA-303] Create conda packages
* [SINGA-337] Add test cases for code
* [SINGA-348] Support autograd MLP Example
* [SINGA-345] Update Jenkins and fix bugs in compliation
* [SINGA-354] Update travis scripts to use conda-build for all platforms
* [SINGA-358] Consolidated RUN steps and cleaned caches in Docker containers
* [SINGA-359] Create alias for conda packages
* Documentation and usability
* [SINGA-223] Fix side navigation menu in the website
* [SINGA-294] Add instructions to run CUDA unit tests on Windows
* [SINGA-305] Add jupyter notebooks for SINGA V1 tutorial
* [SINGA-319] Fix link errors on the index page
* [SINGA-352] Complete SINGA documentation in Chinese version
* [SINGA-361] Add git instructions for contributors and committers
* Bugs fixed
* [SINGA-330] fix openblas building on i7 7700k
* [SINGA-331] Fix the bug of tensor division operation
* [SINGA-350] Error from python3 test
* [SINGA-356] Error using travis tool to build SINGA on mac os
* [SINGA-363] Fix some bugs in imagenet examples
* [SINGA-368] Fix the bug in Cifar10 examples
* [SINGA-369] the errors of examples in testing
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.1.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-296] Add sign and to_host function for pysinga tensor module
* Model components
* [SINGA-254] Implement Adam for V1
* [SINGA-264] Extend the FeedForwardNet to accept multiple inputs
* [SINGA-267] Add spatial mode in batch normalization layer
* [SINGA-271] Add Concat and Slice layers
* [SINGA-275] Add Cross Entropy Loss for multiple labels
* [SINGA-278] Convert trained caffe parameters to singa
* [SINGA-287] Add memory size check for cudnn convolution
* Utility functions and CI
* [SINGA-242] Compile all source files into a single library.
* [SINGA-244] Separating swig interface and python binding files
* [SINGA-246] Imgtool for image augmentation
* [SINGA-247] Add windows support for singa
* [SINGA-251] Implement image loader for pysinga
* [SINGA-252] Use the snapshot methods to dump and load models for pysinga
* [SINGA-255] Compile mandatory depedent libaries together with SINGA code
* [SINGA-259] Add maven pom file for building java classes
* [SINGA-261] Add version ID into the checkpoint files
* [SINGA-266] Add Rafiki python toolkits
* [SINGA-273] Improve license and contributions
* [SINGA-284] Add python unittest into Jenkins and link static libs into whl file
* [SINGA-280] Jenkins CI support
* [SINGA-288] Publish wheel of PySINGA generated by Jenkins to public servers
* Documentation and usability
* [SINGA-263] Create Amazon Machine Image
* [SINGA-268] Add IPython notebooks to the documentation
* [SINGA-276] Create docker images
* [SINGA-289] Update SINGA website automatically using Jenkins
* [SINGA-295] Add an example of image classification using GoogleNet
* Bugs fixed
* [SINGA-245] float as the first operand can not multiply with a tensor object
* [SINGA-293] Bug from compiling PySINGA on Mac OS X with multiple version of Python
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.0.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* Core abstractions including Tensor and Device
* [SINGA-207] Update Tensor functions for matrices
* [SINGA-205] Enable slice and concatenate operations for Tensor objects
* [SINGA-197] Add CNMem as a submodule in lib/
* [SINGA-196] Rename class Blob to Block
* [SINGA-194] Add a Platform singleton
* [SINGA-175] Add memory management APIs and implement a subclass using CNMeM
* [SINGA-173] OpenCL Implementation
* [SINGA-171] Create CppDevice and CudaDevice
* [SINGA-168] Implement Cpp Math functions APIs
* [SINGA-162] Overview of features for V1.x
* [SINGA-165] Add cross-platform timer API to singa
* [SINGA-167] Add Tensor Math function APIs
* [SINGA-166] light built-in logging for making glog optional
* [SINGA-164] Add the base Tensor class
* IO components for file read/write, network and data pre-processing
* [SINGA-233] New communication interface
* [SINGA-215] Implement Image Transformation for Image Pre-processing
* [SINGA-214] Add LMDBReader and LMDBWriter for LMDB
* [SINGA-213] Implement Encoder and Decoder for CSV
* [SINGA-211] Add TextFileReader and TextFileWriter for CSV files
* [SINGA-210] Enable checkpoint and resume for v1.0
* [SINGA-208] Add DataIter base class and a simple implementation
* [SINGA-203] Add OpenCV detection for cmake compilation
* [SINGA-202] Add reader and writer for binary file
* [SINGA-200] Implement Encoder and Decoder for data pre-processing
* Module components including layer classes, training algorithms and Python binding
* [SINGA-235] Unify the engines for cudnn and singa layers
* [SINGA-230] OpenCL Convolution layer and Pooling layer
* [SINGA-222] Fixed bugs in IO
* [SINGA-218] Implementation for RNN CUDNN version
* [SINGA-204] Support the training of feed-forward neural nets
* [SINGA-199] Implement Python classes for SGD optimizers
* [SINGA-198] Change Layer::Setup API to include input Tensor shapes
* [SINGA-193] Add Python layers
* [SINGA-192] Implement optimization algorithms for SINGA v1 (nesterove, adagrad, rmsprop)
* [SINGA-191] Add "autotune" for CudnnConvolution Layer
* [SINGA-190] Add prelu layer and flatten layer
* [SINGA-189] Generate python outputs of proto files
* [SINGA-188] Add Dense layer
* [SINGA-187] Add popular parameter initialization methods
* [SINGA-186] Create Python Tensor class
* [SINGA-184] Add Cross Entropy loss computation
* [SINGA-183] Add the base classes for optimizer, constraint and regularizer
* [SINGA-180] Add Activation layer and Softmax layer
* [SINGA-178] Add Convolution layer and Pooling layer
* [SINGA-176] Add loss and metric base classes
* [SINGA-174] Add Batch Normalization layer and Local Response Nomalization layer.
* [SINGA-170] Add Dropout layer and CudnnDropout layer.
* [SINGA-169] Add base Layer class for V1.0
* Examples
* [SINGA-232] Alexnet on Imagenet
* [SINGA-231] Batchnormlized VGG model for cifar-10
* [SINGA-228] Add Cpp Version of Convolution and Pooling layer
* [SINGA-227] Add Split and Merge Layer and add ResNet Implementation
* Documentation
* [SINGA-239] Transfer documentation files of v0.3.0 to github
* [SINGA-238] RBM on mnist
* [SINGA-225] Documentation for installation and Cifar10 example
* [SINGA-223] Use Sphinx to create the website
* Tools for compilation and some utility code
* [SINGA-229] Complete install targets
* [SINGA-221] Support for Travis-CI
* [SINGA-217] build python package with setup.py
* [SINGA-216] add jenkins for CI support
* [SINGA-212] Disable the compilation of libcnmem if USE_CUDA is OFF
* [SINGA-195] Channel for sending training statistics
* [SINGA-185] Add CBLAS and GLOG detection for singav1
* [SINGA-181] Add NVCC supporting for .cu files
* [SINGA-177] Add fully cmake supporting for the compilation of singa_v1
* [SINGA-172] Add CMake supporting for Cuda and Cudnn libs
----------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-0.3.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* GPU Support
* [SINGA-131] Implement and optimize hybrid training using both CPU and GPU
* [SINGA-136] Support cuDNN v4
* [SINGA-134] Extend SINGA to run over a GPU cluster
* [SINGA-157] Change the priority of cudnn library and install libsingagpu.so
* Remove Dependencies
* [SINGA-156] Remove the dependency on ZMQ for single process training
* [SINGA-155] Remove zookeeper for single-process training
* Python Binding
* [SINGA-126] Python Binding for Interactive Training
* Other Improvements
* [SINGA-80] New Blob Level and Address Level Math Operation Interface
* [SINGA-130] Data Prefetching
* [SINGA-145] New SGD based optimization Updaters: AdaDelta, Adam, AdamMax
* Bugs Fixed
* [SINGA-148] Race condition between Worker threads and Driver
* [SINGA-150] Mesos Docker container failed
* [SIGNA-141] Undesired Hash collision when locating process id to worker…
* [SINGA-149] Docker build fail
* [SINGA-143] The compilation cannot detect libsingagpu.so file
-----------------------------------------
Release Notes - SINGA - Version singa-incubating-0.2.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is
designed with an intuitive programming model based on the layer abstraction. SINGA supports a wide variety of popular
deep learning models.
This release includes following features:
* Programming model
* [SINGA-80] New Blob Level and Address Level Math Operation Interface
* [SINGA-82] Refactor input layers using data store abstraction
* [SINGA-87] Replace exclude field to include field for layer configuration
* [SINGA-110] Add Layer member datavec_ and gradvec_
* [SINGA-120] Implemented GRU and BPTT (BPTTWorker)
* Neuralnet layers
* [SINGA-91] Add SoftmaxLayer and ArgSortLayer
* [SINGA-106] Add dummy layer for test purpose
* [SINGA-120] Implemented GRU and BPTT (GRULayer and OneHotLayer)
* GPU training support
* [SINGA-100] Implement layers using CUDNN for GPU training
* [SINGA-104] Add Context Class
* [SINGA-105] Update GUN make files for compiling cuda related code
* [SINGA-98] Add Support for AlexNet ImageNet Classification Model
* Model/Hybrid partition
* [SINGA-109] Refine bridge layers
* [SINGA-111] Add slice, concate and split layers
* [SINGA-113] Model/Hybrid Partition Support
* Python binding
* [SINGA-108] Add Python wrapper to singa
* Predict-only mode
* [SINGA-85] Add functions for extracting features and test new data
* Integrate with third-party tools
* [SINGA-11] Start SINGA on Apache Mesos
* [SINGA-78] Use Doxygen to generate documentation
* [SINGA-89] Add Docker support
* Unit test
* [SINGA-95] Add make test after building
* Other improvment
* [SINGA-84] Header Files Rearrange
* [SINGA-93] Remove the asterisk in the log tcp://169.254.12.152:*:49152
* [SINGA-94] Move call to google::InitGoogleLogging() from Driver::Init() to main()
* [SINGA-96] Add Momentum to Cifar10 Example
* [SINGA-101] Add ll (ls -l) command in .bashrc file when using docker
* [SINGA-114] Remove short logs in tmp directory
* [SINGA-115] Print layer debug information in the neural net graph file
* [SINGA-118] Make protobuf LayerType field id easy to assign
* [SIGNA-97] Add HDFS Store
* Bugs fixed
* [SINGA-85] Fix compilation errors in examples
* [SINGA-90] Miscellaneous trivial bug fixes
* [SINGA-107] Error from loading pre-trained params for training stacked RBMs
* [SINGA-116] Fix a bug in InnerProductLayer caused by weight matrix sharing
-----------------------------------------
Features included in singa-incubating-0.1.0:
* Job management
* [SINGA-3] Use Zookeeper to check stopping (finish) time of the system
* [SINGA-16] Runtime Process id Management
* [SINGA-25] Setup glog output path
* [SINGA-26] Run distributed training in a single command
* [SINGA-30] Enhance easy-to-use feature and support concurrent jobs
* [SINGA-33] Automatically launch a number of processes in the cluster
* [SINGA-34] Support external zookeeper service
* [SINGA-38] Support concurrent jobs
* [SINGA-39] Avoid ssh in scripts for single node environment
* [SINGA-43] Remove Job-related output from workspace
* [SINGA-56] No automatic launching of zookeeper service
* [SINGA-73] Refine the selection of available hosts from host list
* Installation with GNU Auto tool
* [SINGA-4] Refine thirdparty-dependency installation
* [SINGA-13] Separate intermediate files of compilation from source files
* [SINGA-17] Add root permission within thirdparty/install.
* [SINGA-27] Generate python modules for proto objects
* [SINGA-53] Add lmdb compiling options
* [SINGA-62] Remove building scrips and auxiliary files
* [SINGA-67] Add singatest into build targets
* Distributed training
* [SINGA-7] Implement shared memory Hogwild algorithm
* [SINGA-8] Implement distributed Hogwild
* [SINGA-19] Slice large Param objects for load-balance
* [SINGA-29] Update NeuralNet class to enable layer partition type customization
* [SINGA-24] Implement Downpour training framework
* [SINGA-32] Implement AllReduce training framework
* [SINGA-57] Improve Distributed Hogwild
* Training algorithms for different model categories
* [SINGA-9] Add Support for Restricted Boltzman Machine (RBM) model
* [SINGA-10] Add Support for Recurrent Neural Networks (RNN)
* Checkpoint and restore
* [SINGA-12] Support Checkpoint and Restore
* Unit test
* [SINGA-64] Add the test module for utils/common
* Programming model
* [SINGA-36] Refactor job configuration, driver program and scripts
* [SINGA-37] Enable users to set parameter sharing in model configuration
* [SINGA-54] Refactor job configuration to move fields in ModelProto out
* [SINGA-55] Refactor main.cc and singa.h
* [SINGA-61] Support user defined classes
* [SINGA-65] Add an example of writing user-defined layers
* Other features
* [SINGA-6] Implement thread-safe singleton
* [SINGA-18] Update API for displaying performance metric
* [SINGA-77] Integrate with Apache RAT
Some bugs are fixed during the development of this release
* [SINGA-2] Check failed: zsock_connect
* [SINGA-5] Server early terminate when zookeeper singa folder is not initially empty
* [SINGA-15] Fixg a bug from ConnectStub function which gets stuck for connecting layer_dealer_
* [SINGA-22] Cannot find openblas library when it is installed in default path
* [SINGA-23] Libtool version mismatch error.
* [SINGA-28] Fix a bug from topology sort of Graph
* [SINGA-42] Issue when loading checkpoints
* [SINGA-44] A bug when reseting metric values
* [SINGA-46] Fix a bug in updater.cc to scale the gradients
* [SINGA-47] Fix a bug in data layers that leads to out-of-memory when group size is too large
* [SINGA-48] Fix a bug in trainer.cc that assigns the same NeuralNet instance to workers from diff groups
* [SINGA-49] Fix a bug in HandlePutMsg func that sets param fields to invalid values
* [SINGA-66] Fix bugs in Worker::RunOneBatch function and ClusterProto
* [SINGA-79] Fix bug in singatool that can not parse -conf flag
Features planned for the next release
* [SINGA-11] Start SINGA using Mesos
* [SINGA-31] Extend Blob to support xpu (cpu or gpu)
* [SINGA-35] Add random number generators
* [SINGA-40] Support sparse Param update
* [SINGA-41] Support single node single GPU training