Releases: chainer/chainer
v5.0.0b2
This is the release note of v5.0.0b2. See here for the complete list of solved issues and merged PRs.
Highlights
- New configuration value
chainer.config.dtype
has been introduced. This configuration can be used to switch your model to run withfloat16
/float32
/float64
without modifying your code. In this version of Chainer, this configuration is supported by initializers, built-in datasets and part of built-inLink
s. We're going to improve all built-inLink
s to support this feature towards the final v5 release (#4582). - New module
chainer.distributions
has been introduced (see API Reference). We're going to provide more probability distribution implementations towards the final v5 release (#4678). - Some functions (
Variable
operators andF.matmul
) now support NumPy-style broadcasting. We're going to improve more built-inFunction
s to support broadcast towards the final v5 release (#4679). L.ConvolutionND
andL.DeconvolutionND
now support grouped and dilated convolution.- Interoperability with Caffe model has been improved for both export (
Deconvolution
andLeakyReLU
functions) and import (Deconvolution
andReshape
layer). return_indices
option has been added toF.max_pooling_2d
andF.max_pooling_nd
, so that you can access indices (orindexes
) without usingMaxPooling2D
andMaxPoolingND
class directly.chainer.datasets.TextDataset
has been introduced to reduce host memory when loading large text files. See seq2seq example for an example usage.
Changes without compatibility
- Make updaters call
new_epoch
automatically (#4608)- This change should affect only a minority of users (who call
Optimizer.new_epoch
while using a trainer, or who implement their own updater class). See the Upgrade Guide for details.
- This change should affect only a minority of users (who call
- Fix
no_grads
option ofcheck_backward
(#4654)
New Features
- Add
L.DeformableConvolution2D
(#2468) - Support skipping padded regions in
F.average_pooling_nd
(#3486) - Support double-backprop in
F.contrastive
(#3830) - Add M-SVAG optimizer (#4473!)
- Add
chainer.config.dtype
and use it in initializers and dataset loaders. (#4510) - Add
InverseShift
extension (#4565, thanks @jinjiren!) - Support grouped and dilated convolutions for ConvolutionND and DeconvolutionND (#4591)
- Make updaters call
new_epoch
automatically (#4608) - Add group normalization (#4638, thanks @lyakaap!)
- Support remove/replace child link in ChainList (#4660, thanks @insanity!)
- Automatic broadcast in basic_math and matmul (#4679)
- Add normal distribution (#4773)
- Implement
TextDataset
to load line-oriented text files (#4782) - Add
CorrectedMomentumSGD
optimizer (#4835) - Support importing Caffe
Reshape
layer (#4875) - Add
F.diagonal
(#4901) - Add caffe import for
Deconvolution
layer; Fixnum_output
parameter in caffe export ofDeconvolution
layer (#4936, thanks @tsurumeso!) - Add caffe export for
LeakyReLU
layer (#4937, thanks @tsurumeso!)
Enhancements
- Issue a warning when
repeat
attribute of some iterators inEvaluator
isTrue
(#3436) - Use
input_num
to remove array creation overhead (#4162) - Avoid using retained inputs in
F.rsqrt
(#4614) - Assign user-friendly name for better type check exception message (#4680)
- Improve error message and exception type during
MultiprocessParallelUpdater
initialization (#4751) - Optimize
F.bilinear
(#4752) - Show progress report to stderr (#4766)
- Raise an error if neither
__call__
norforward
is defined in a link (#4770) - Output the direction of differentiation in
gradient_check
error (#4797) - Allow more than 2 axis in
F.normalize
(#4799) - Remove redundant device allocation in word2vec example (#4848, thanks @arisliang!)
- Deprecate
F.fixed_batch_renormalization
(#4869) - Improve argument processing performance (#4880)
- Add
return_indices
option toF.max_pooling_2d
(#4890) - Add
return_indices
option toF.max_pooling_nd
(#4891) - Remove experimental warning (#4896)
- Add
mask
andreturn_mask
option toF.dropout
(#4907) - Add
eps
andreturn_eps
option toF.gaussian
(#4909) - Simplify type checks in
F.broadcast
usingtype_check.expect_broadcast_shapes
(#4947)
Bug Fixes
- Fix stride in the
ResNeXt50
example (#4479, thanks @akitotakeki!) - Fix
no_grads
option ofcheck_backward
(#4654) - Raise an error when
devices
argument given toMultiprocessParallelUpdater
is neitherdict
norlist
(#4716) - Fix a stability issue in
LayerNormalization
(#4744, thanks @anaruse!) - Select devices appropriately when calling user code in
ParallelUpdater
(#4774) - Fix
ResNetLayers
to not pass-throughdownsample_fb
argument toL.Convolution2D
(#4829) - Fix
F.rollaxis
backward whenaxis=0
andstart=0
(#4836) - Remove wrong-length check from
GetItem.check_type_forward
(#4845) - Fix error message when checking types of grads (#4879, thanks @reo911gt3!)
- Fix Caffe exporter to export
eps
ofBatchNormalization
layer (#4884) - Fix
SigmoidCrossEntropy.backward
(#4915) - Fix
ValueError
in caffe exporter on Windows with Python 2.7 (#4928) - Fix operations in
ChainList
(#4945) - Raise an error on 0-dim input in
F.matmul
(#4949)
Documentation
- Improve documentation about wrapper functions (#4052)
- Add notes on differences between v1 and later versions of the
extract
method of vision models (ResNet, VGG16, GoogLeNet) (#4675) - Improve CNN example docs (#4714)
- Document the pros and cons of the multi-GPU updaters (#4778)
- Add type check trouble shooting to FAQ (#4785)
- Fix comment in
caffe.py
(#4808, thanks @poutyface!) - Document performance best practices (#4809)
- Fix a documenation bug in
TransformDataset
(#4814) - Correct docs of
batch_{re}normalization
about running averages (#4818, thanks @grafi-tt!) - Add testing utilities to documentation (#4828)
- Remove slack archive link from README.md (#4839, thanks @hidetomasuoka!)
- Document
call_for_each_param
attribute in optimizer hooks (#4849) - Improve function reference and documentation (#4855)
- Add distribution reference (#4886)
- Fix
gradient_check
docstring to enable doctest (#4889) - Fix normal distribution docs (#4892)
- Fix Adam alpha argument explanation (#4895)
- Improve the documentation of
F.moveaxis
and discourageF.rollaxis
(#4900) - Replace
0xa0
(nbsp) with0x20
(space) in documentation (#4921) - Update documentation to explicitly suggest using ideep4py v1 (#4934)
- Add
ZippedImageDataset
andMultiZippedImageDataset
to documentation (#4959)
Installation
- Remove deprecated
imp.load_source
insetup.py
(#4846, thanks @vilyaair!)
Examples
- Add pix2pix example (#4271!)
- Refactor sentiment example (#4694)
- Add serialization example (#4740)
- Simplify classification code in MNIST custom loop example (#4837)
Tests
- Fix occasional test failure of
F.average
by changing lower-bound of weight.sum (#4771) - Add backprop tests (#4779)
- Fix test warning filter for Theano 1.0.2 which triggers
DeprecationWarning
(#4810) - Test
F.layer_normalization
with large eps (#4816) - Fix input checks in
F.min
andF.max
tests (#4838) - Remove
flake8
from AppVeyor test (#4852) - Set output grad in
TestCUDAProfileHook
(#4863) - Stop using
freeze_running_statistics
inTestFixedBatchRenormalization
(#4868) - Improve batch renormalization tests (#4871)
- Mark
multi_gpu
tests asgpu
tests (#4872) - Deselect gpu tests instead of skipping them (#4873)
- Catch warnings in
test_count_params
(#4876) - Delete redundant use of
no_grads
argument ofcheck_backward
from test (#4914) - Check docs build in Travis CI (#4917)
- Tentatively restrict
pytest-timeout
version to <1.3.0 (#4918) - Fix keyword argument name in
TestResNetLayers
(#4965) - Remove unnecessary tests (#4966)
v4.2.0
This is the release note of v4.2.0. See here for the complete list of solved issues and merged PRs.
Highlights
return_indices
option has been added toF.max_pooling_2d
andF.max_pooling_nd
, so that you can access indices (orindexes
) without usingMaxPooling2D
andMaxPoolingND
classes directly.
New Features
- Add more functions with double-backprop support:
F.prod
(#4826) andF.contrastive
(#4861) - Support importing Caffe Reshape layer (#4905)
Enhancements
- Raise an error when CUDA context has been initialized before
MultiprocessParallelUpdater
is initialized (#4750) - Output the direction of differentiation in
gradient_check
error (#4817) - Show progress report to stderr (#4827)
- Optimize
F.bilinear
(#4834) - Remove redundant device allocation in word2vec example (#4856, thanks @arisliang!)
- Issue a warning when
repeat
attribute of some iterators inEvaluator
isTrue
(#4865) - Improve error message and exception type during
MultiprocessParallelUpdater
initialization (#4867) - Raise an error if neither
__call__
norforward
is defined in a link (#4888) - Deprecate
F.fixed_batch_renormalization
(#4940) - Add
return_indices
option toF.max_pooling_2d
(#4952) - Add
return_indices
option toF.max_pooling_nd
(#4953) - Add
mask
andreturn_mask
option toF.dropout
(#4954) - Add
eps
andreturn_eps
option toF.gaussian
(#4955)
Bug Fixes
- Select devices appropriately when calling user code in
ParallelUpdater
(#4842) - Fix
F.rollaxis
backward whenaxis=0
andstart=0
(#4843) - Raise an error when
devices
argument given toMultiprocessParallelUpdater
is neitherdict
norlist
(#4847) - Fix error message when checking types of grads (#4885, thanks @reo911gt3!)
- Fix Caffe exporter to export
eps
of theBatchNormalization
layer (#4913) - Fix
SigmoidCrossEntropy.backward
(#4924) - Fix
ValueError
in Caffe exporter on Windows with Python 2.7 (#4929) - Remove wrong-length check from
GetItem.check_type_forward
(#4944)
Documentation
- Add notes on differences between v1 and later versions to the document of the
extract
method of vision models (ResNet, VGG16, GoogLeNet) (#4941) - Fix some typos in tutorials (#4802, thanks @elvisyjlin!)
- Update URL in README (#4803)
- Fix comment in
caffe.py
(#4815, thanks @poutyface!) - Fix a documenation bug in
TransformDataset
(#4819) - Remove Slack archive link from README.md (#4840, thanks @hidetomasuoka!)
- Improve documentation about wrapper functions (#4850)
- Document
call_for_each_param
attribute in optimizer hooks (#4857) - Correct docs of
batch_{re}normalization
about running averages (#4870, thanks @grafi-tt!) - Add type check trouble shooting to FAQ (#4883)
- Fix
gradient_check
docstring to enable doctest (#4926) - Replace 0xa0 (nbsp) with 0x20 (space) in documentation (#4930)
- Update documentation to explicitly suggest using ideep4py v1 (#4939)
- Document performance best practices (#4942)
- Improve function reference and documentation (#4943)
Installation
- Fix required version of CuPy (#4830)
- Remove deprecated
imp.load_source
insetup.py
(#4859, thanks @vilyaair!)
Examples
- Fix custom loop examples to disable train mode during evaluation (#4804)
Tests
- Separate
MultiprocessParallelUpdater
tests (#4801) - Fix test warning filter for Theano 1.0.2 which triggers
DeprecationWarning
(#4820) - Fix input checks in
F.min
andF.max
tests (#4841) - Set output grad in
TestCUDAProfileHook
(#4866) - Mark
multi_gpu
tests as gpu tests (#4881) - Remove flake8 from AppVeyor test (#4882)
- Deselect gpu tests instead of skipping them (#4906)
- Stop using
freeze_running_statistics
inTestFixedBatchRenormalization
(#4908) - Add backprop tests (#4916)
- Tentatively restrict pytest-timeout version to <1.3.0 (#4919)
- Check docs build in Travis CI (#4931)
- Delete redundant use of
no_grads
argument ofcheck_backward
from test (#4938) - Fix occasional test failure of
F.average
by changing lower-bound of the sum of weights (#4960) - Remove unnecessary tests (#4967)
v5.0.0b1
This is the release notes of v5.0.0b1. See here for the complete list of solved issues and merged PRs.
New Features
- Add
order_sampler
option to Iterators (#3429) - New style
prod
function (#3764) - Moveaxis array operation proposal (#4112, thanks @fukatani!)
- Add
n_batch_axes
option toF.linear
(#4204) - Support Layer-wise Adaptive Rate Scaling (LARS) (#4237, thanks @tohmae!)
- Sparse matmul support (#4397, thanks @anaruse!)
- Optimize grouped convolution for intel64 backed environment (#4450, thanks @tkng!)
- Add
ignore_names
option toload_npz
(#4682) - Add the support of
PolynomialShift
in the extensions of training (#4693, thanks @tianshilei1992!)
Enhancements
- Use
rsqrt
inF.batch_normalization
(#4612) - Raise an error when CUDA has been initialized before
MultiprocessParallelUpdater
is initialized (#4717) - Fix performance regression in
F.bilinear
(#4738) - Support copying 1-dim array to iDeep (#4746)
- Optimize Huber loss by simpler calculation (#4775, thanks @grafi-tt!)
Bug Fixes
- Fix
Chain.repeat
raise error (#4649, thanks @mori97!) - Fix
ChainList.copy
not supportingmode
argument (#4652) - Warn only if the BatchNormalization input tensor is 2-dimensional (#4663)
- Fix import failure when matplotlib 1.x is installed (#4681)
- Select current device using grads of outputs in addition to inputs during backward (#4725)
- Fix dtype bug in
F.normalize
(#4763) - Fix
lazy_grad_sum
debug mode (#4768)
Documentation
- Fix heading level of Caffe docs (#4631)
- Fix typos in seq2seq tutorial document (#4643, thanks @kuni-kuni!)
- Update URL in README (#4657)
- Fix
F.batch_normalization
axis document (#4666) - Fix docs and tests of
L.BatchNormalization
(#4671) - Update
train_loop.rst
(#4700, thanks @arisliang!) - Update math formula typo (#4702, thanks @arisliang!)
- Update
word2vec.rst
(#4705, thanks @arisliang!) - Add example for
transpose_sequence
(#4719) - Update
seq2seq.rst
(#4721, thanks @arisliang!) - Add doc build steps to the contribution guide (#4722)
- Fix dead link to CuPy installation guide (#4724)
- Update top sentences in documentation (#4732)
- Add
abs
to document ofVariable
(#4757) - Fix some typos in tutorials (#4760, thanks @elvisyjlin!)
- Sparse matrix documentation enhancements (#4786)
Examples
- Mini-batch training for recursive neural networks example (#2135)
- Fix custom loop examples to disable train mode during evaluation (#4568)
Tests
- Unify
TestBatchNormalization
andTestBatchNormalizationAxis
(#4558) L.BatchNormalization
: miscellaneous fixes (#4671)- Support skipping tests decorated by condition or parametrize (#4685)
- Separate
MultiprocessParallelUpdater
tests (#4726) - Fix
hacking
version (#4731) - Fix incorrect attributes and incorrect indentation in pooling tests (#4743)
v4.1.0
This is the release note of v4.1.0. See here for the complete list of solved issues and merged PRs.
New Features
- Provide a cleaner way to collect threads and processes in
MultiprocessIterator
(#4637) - Support Layer-wise Adaptive Rate Scaling (LARS) (#4668, thanks @tohmae!)
- Optimize grouped convolution for intel64 backed environment (#4787, thanks @tkng!)
Enhancements
- Improve
F.rsqrt
performance in CPU (#4634) - Fix temporary file permission issue in
LogReport
(#4635) - Use
rsqrt
inF.batch_normalization
(#4665) - Fix performance regression in
F.bilinear
(#4762) - Optimize Huber loss by simpler calculation (#4788, thanks @grafi-tt!)
- Support copying 1-dim array to iDeep (#4794)
Bug Fixes
- Fix
Chain.repeat
raise error (#4653, thanks @mori97!) - Fix
ChainList.copy
not supportingmode
argument (#4664) - Fix import failure when matplotlib 1.x is installed (#4689)
- Fix
lazy_grad_sum
debug mode (#4776) - Select current device using grads of outputs in addition to inputs during backward (#4780)
- Fix dtype bug in
F.normalize
(#4781)
Documents
- Add tutorial for
Reporter
andreport
(#4632) - Rewrite installation guide to align with CuPy's installation guide (#4633)
- Add higher-order derivative support of Chainer to the comparison table (#4636)
- Fix typos in seq2seq tutorial document (#4651, thanks @kuni-kuni!)
- Fix heading level of Caffe docs (#4690)
- Update
word2vec.rst
(#4713, thanks @arisliang!) - Update math formula typo (#4715, thanks @arisliang!)
- Fix dead link to CuPy installation guide (#4727)
- Update
seq2seq.rst
(#4728, thanks @arisliang!) - Update top sentences in documentation (#4734)
- Add example for transpose_sequence (#4745)
- Update
train_loop.rst
(#4759, thanks @arisliang!) - Add
abs
to document ofVariable
(#4764) - Add doc build steps to the contribution guide (#4784)
Tests
v5.0.0a1
This is the release of v5.0.0a1. See here for the complete list of solved issues and merged PRs.
New Features
- Add
Sequential
class for easy model definition of a single-stream computational graph (#2918) - Add
count_params()
method toLink
which enables to count the number of trainable values in aLink
easily (#3101) - Implement Caffe export which can export a Chainer model into Caffe protobuf format (#3631)
- Add more support on double backward:
F.forget
(#3792) - Provide a cleaner way to collect threads and processes in
MultiprocessIterator
(#4155) - Add
axis
option to batch normalization (#4266, thanks @anaruse!) - Add
add_extra
option to SVHN (#4478, thanks @akitotakeki!) - Implement
Variable.xp
(#4497) - Add a new
Trainer
extension to kill the training whenNaN
orInf
is detected in the model parameters (FailOnNonNumber
) (#4545) - Add
chainer.print_runtime_info()
method to summarize the versions of libraries used in Chainer and CuPy (#4559)
Enhancements
- Remove submodule aliases (#4378)
- Avoid mutable default arguments (#4419)
- Avoid use of
from __future__ print_function
(#4470) - Optimize
F.depth2space
andF.space2depth
by reducing operations (#4482, thanks @ruimashita!) - Support deserializing into array of different dtypes (#4511)
- Fix temporary file permission issue in
LogReport
(#4528) - Support dilated convolution on iDeep backend (#4537, thanks @LuoYuanke!)
- Improve
F.rsqrt
performance in CPU (#4538) - Improve
chainer.utils.argument
exception message (#4551) - Removed redundant member variables (#4580)
Bug Fixes
- Save snapshot in the OS default permission (#4461, thanks @belltailjp!)
- Call
FunctionHooks
inchainer.grad
(#4499) - Remove
eps
from batch normalization statistics (#4505) - Change
group
argument ofF.convolution_2d
andF.deconvolution_2d
to keyword-only argument. (#4564) - Fix
to_intel64
to check if it is suitable for iDeep (#4577) - Fix
to_intel64
not updatingVariableNode.data
(#4592)
Documents
- Add higher-order derivative support of Chainer to the comparison table (#3477)
- Add tutorial: How to use
chainer.Reporter
(#3688) - Improve docstring of
F.connectionist_temporal_classification
(CTC) (#4309) - Add upgrade guide for
optimizer_hooks
namespace (#4468) - Add document on using iDeep (#4477)
- Fix debug mode documentation (#4492)
- Fix order of items in reference section (#4493)
- Add a document of arithmetic special functions (#4495)
- Improve docs for supported operators in
Variable
(#4516) - Fix docs of
convolution_2d
anddeconvolution_2d
(#4539) - Add document for the configuration
chainer.config.lazy_grad_sum
(#4543) - Improve description of extensions (#4552)
- Fix dead link to optimizer hooks (#4563)
- Fix typos (#4569, #4574, thanks @Hakuyume!)
- Fix document in
EarlyStoppingTrigger
(#4578, thanks @mori97!) - Add FAQ for
MultiprocessIterator
+ OpenCV problem (#4589) - Improve docstrings about axis of Caffe functions (#4599)
- Improve docs of
get_device_from_array
(#4604) - Fix docstrings of
Sequential
(#4605) - Rewrite installation guide to align with CuPy's installation guide (#4622)
Installation
Examples
- End-to-end memory networks example (#4222)
- Fix
seq2seq
example ignoring limitation for target length (#4611)
Tests
- Travis test against v4 branch (#4503)
- Fix doctest (#4506)
- Fix and simplify
TestForwardConsistency
inF.softmax_cross_entropy
(#4554) - Simplify
TestSoftmaxCrossEntropyInvalidReduce
inF.test_softmax_cross_entropy
(#4555)
Others
- Let the StaleBot ignore issues labelled with “roadmap” (#4496)
v4.0.0
This is a major release of Chainer v4.0.0. All the updates from the previous major version (v3.5.0) are found in the release notes below:
Summary of v4 update
- Major Performance Improvements
- Support iDeep backend for acceleration on Intel CPUs. We observed that the GoogLeNet inference with batch size 1 is made x8.9 faster (compared to the case without iDeep, both on Intel(R) Xeon(R) CPU E5-2623 v3 @ 3.00GHz with MKL).
- Support cuDNN convolution autotuning
- Better FP16 training support
- TensorCore
- Loss scaling
- Caffe export support (experimental)
- NCCL2 support
See the blog post for the details. Also see the Upgrade Guide for users migrating from Chainer v3 to v4.
Updates from the release candidate are as follows.
New Features
- Support double-backprop in
forget
function (#4522) - Implement
Variable.xp
(#4536) - Add
Sequential
class for easy model definition of a single-stream computational graph (#4601) - Add extension to kill training when NaN or Inf is detected (
FailOnNonNumber
) (#4602) - Implement interface to print runtime information (#4613)
- Implement Caffe export (#4621)
Enhancements
- Support deserializing into array of different dtype (#4529)
- Support dilated convolution on iDeep backend (#4540, thanks @LuoYuanke!)
- Optimize
depth2space
&space2depth
funcs by reducing operations (#4590, thanks @ruimashita!) - Avoid mutable default arguments (#4600)
- Improve
chainer.utils.argument
exception message (#4615) - Removed redundant member variables (#4627)
Bug Fixes
- Remove
eps
from batch normalization statistics (#4517) - Call
FunctionHooks
inchainer.grad
(#4541) - Fix group argument of
{de,}convolution_2d
to keyword-only argument. (#4573) - Fix
to_intel64
not updating VariableNode.data (#4597) - Fix
to_intel64
to check if it is suitable for iDeep (#4598) - Fix seq2seq example ignoring limitation for target length (#4617)
- Save snapshot in the OS default permission (#4618, thanks @belltailjp!)
- Fix docstrings of
Sequential
class (#4628)
Documents
- Document arithmetic special functions (#4515)
- Fix debug mode documentation (#4519)
- Add document on using iDeep (#4520)
- Fix order of items in Reference section (#4526)
- Improve docs for supported operators in Variable (#4527)
- Improve docstring of CTC (#4531)
- Improve description of Extensions (#4561)
- Fix a typo (#4570)
- Add upgrade guide for optimizer_hooks namespace (#4571)
- Fix docs in
{convolution,deconvolution}_2d
(#4575) - Fix typo (#4576, thanks @Hakuyume!)
- Fix document in
EarlyStoppingTrigger
(#4584, thanks @mori97!) - Add document for
chainer.config.lazy_grad_sum
(#4585) - Improve docstrings about axis of Caffe functions (#4603)
- Improve docs of get_device_from_array (#4624)
- Fix docstrings of Sequential (#4628)
- Add FAQ for MultiprocessIterator + OpenCV problem (#4629)
Installation
Examples
- fix seq2seq example ignoring limitation for target length (#4617)
Tests
v4.0.0rc1
This is the release candidate of v4. See here for the complete list of solved issues and merged PRs.
Announcements
- The master branch has been switched to v5 development. The development of v4 will continue in the v4 branch.
- The major release of v4 is planned on Apr. 17.
New Features
- New differentiable functions
repeat
(#3735)fft
,ifft
(#4241)local_convolution_2d
: 2D convolution with spatially unshared weights (#4073, thanks @mihirparadkar!)swish
: a new activation function proposed here (#4262, thanks @mizuno-gsinet!)convolution_{1,3}d
,deconvolution_{1,3}d
,average_pooling_{1,3}d
,max_pooling_{1,3}d
,unpooling_{1,3}d
(#4025)- These are thin wrappers of
***_nd
variants
- These are thin wrappers of
- New-style (double backpropable) function support
- Add
testing.patch
which callsmock.patch
withwraps
argument (#3883) - Added initial version of
AMSGrad
(#4032, thanks @kashif!) - Added
ZippedImageDataset
andMultiZippedImageDataset
(#4127, thanks @d0i!) - Accumulate the gradient to a tuple for lazy add operation (#4136, thanks @LuoYuanke!)
- Implement time trigger (#4294)
- Add CuPy support in
ParameterStatistics
; Add option to skip parameters with NaN values inParameterStatistics
(#4345) - Spatial pyramid pooling method specified by string instead of type (#4401)
- Add
n_cell
property to NStepRNN family (#4417, thanks @levelfour!)
Bug Fixes
- Fix to make it consistent with
numpy.split
andcupy.split
(#4153, thanks @ken-nakanishi!) - Fix
lstm
backward arguments (#4320) - Fix
BatchMatMulGrad.backward
(#4349) - Fix test of
MultiprocessParallelUpdater
(#4368) - Fix iDeep import error message not printed as expected (#4381)
- Fixed a bug that occur when using iDeep in
chainer/link.py
(#4382, thanks @ken-nakanishi!) - Fix
link.to_intel64
for persistent values (#4384) - Fix the index page of datasets in documentation (#4407)
- Fix
Variable
tests (#4408) - Remove
Swish
function class alias (#4409) - Reset
Link._device_id
toNone
into_intel64
(#4436) - Fix to disable using iDeep in optimizers if
to_intel64
is not called (#4457)
Installation
- Support using cupy wheels as a dependency (#4256)
- Prefer pip in installation guide; Add sphinx in docs requirements (#4366)
- Add
cupy-cuda91
wheel dependency (#4392) - Use CuPy wheels in Dockerfiles (#4406)
- Add Dockerfiles for Chainer with iDeep (#4476)
Enhancements
- Delegate cuDNN convolution operation to CuPy (#3782)
- Improve extensions part in Trainer tutorial (#3809)
- Create
chainer.optimizer_hooks
namespace and move hooks there. (#3977) - Add warning for batch normalization when batchsize=1 and train=True (#3996)
- Optimize
rsqrt
function with CuPy (#4108) - Fix progress bar exceeds 100% (#4152, thanks @ankokumoyashi!)
- Require "scalar" variables to be 0-dim (#4199)
- Update examples and tests for
chainer.backends.cuda
(#4259) - Child sum treelstm with less than 2 child (#4275)
- Add cuDNN support for
clipped_relu
(#4307, thanks @tkerola!) - Improve error messages of array type check. (#4312, thanks @mizuno-gsinet!)
- Use the same code for calculating forward and backward probabilities in CTC (#4313)
- Refactor
F.split_axis
(#4328) - Refactor RNNs (#4341)
- Detect backward of old-style functions in
chainer.grad
(#4352) - Extract a function that concatenates weight and bias matrixes for cuDNN (#4355)
- Fix numpy dtypes bug in
cuda.to_gpu
andcuda.copy
(#4380, thanks @tkerola!) - Improve ideep import error message fror missing shared objects (#4385)
- Change
group
argument name ofConvolution2D
andDeconvolution2D
(#4404) - Prefer
chainer.backends.cuda
in recent added codes (#4414) - Group imports in the order mentioned in the contribution guide (#4418)
- Extract stack function from CTC loss (#4430)
- Change permission of Python files (#4432)
- Typecheck
split_at
(#4434, thanks @corochann!) - Remove
future.types.newint
(#4435) - Accelerate multi-add for intel64 backend (#4447, thanks @LuoYuanke!)
- Fix to use
CUDNN_BN_MIN_EPSILON
(#4466) - Deprecate
sep
argument ofPrintHook
Documents
- Reorganize tutorial & add new contents (#3241)
- Update comparison.rst to add tensorboardX as an Web interface (#3610, thanks @lanpa!)
- Document missing environment variables (#3662)
- Improve extensions part in trainer tutorial (#3809)
- Update documentation for
chainer.backends.cuda
(#3981) - Fix sphinx build option assignment in
Makefile
(#4037) - Add autosummary check to documents (#4040)
- Add explanation to the document of
concat_examples
(#4164) - Add link to CuPy upgrade guide (#4188)
- Updated English for extension documentation (#4215, thanks @rcalland!)
- Update examples and tests for
chainer.backends.cuda
(#4259) - Prefer pip in installation guide; Add sphinx in docs requirements (#4366)
- Improve docs of get item (#4375, thanks @naoto0804)
- Add
F.shift
to docs (#4379) - Fix typo in
unpooling_nd
(#4398), in the document of configuration (#4474), in a comment ofUpdateRule
(#4494) - Fix the index page of datasets in documentation (#4407)
- Fix document in
NStepLSTM/GRU/RNN
(#4425) - Fix docs format in upgrade guide (#4428)
- Fix
Evaluator.device
docs (#4437) - Add ChainerUI link to comparison table (#4438)
- Fix underline in docs (#4446)
- Fix trainer docs (#4464)
- Fix sectioning of "Datasets" reference (#4486)
- Fix for optimizer hook namespace change (#4488)
- Deprecate
sep
argument ofPrintHook
(#4471)
Examples
- Fix some options of MNIST example not working (#3500)
- Add a tutorial of sequence-to-sequence models (#3984)
- Update examples and tests for
chainer.backends.cuda
(#4259)
Tests
- Add
testing.patch
which callsmock.patch
withwraps
argument (#3883) - Add test for inconsistent backend input outputs (#4044)
- Test of RNN in the case when some gradients are
None
(#4049) - Fix test of
MulprocessParallelUpdater
(#4368) - Revert "Skip
MultiprocessParallelUpdater
test for timeout" (#4376) - Add backward test for old-styled sclar variable (#4416)
- Reduce
UserWarning
(#4483) - Fix doctest failure in
chainer.dataset.concat_examples
(#4485) - Avoid running
TestSplitAxis
tests on NumPy 1.10 (#4500) - Add test cases for various shapes as function input (#3744)
- add Codecov.io configuration (#4402)
Others
v3.5.0
This is the release note of v3.5.0. See here for the complete list of solved issues and merged PRs.
New Features
- Add more Functions with double-backprop support:
simplified_dropconnect function
(#4403),zoneout
(#4423),average_pooling_nd
,max_pooling_nd
,unpooling_nd
(#4405)
Improvements
- Accept NumPy ints as a device ID in
cuda.to_gpu
andcuda.copy
(#4386) - Raise warning for batch normalization when
batchsize=1
andtrain=True
(#4433) - Add typechecking to
split_at
(#4441) - Update examples and tests for
chainer.backends.cuda
(#4463)
Bug Fixes
- Improve
ELU.backward
(#4351) - Fix crash bug when a child is reporting a value when using
MultiprocessParallelUpdater
(#4367) - Fix
BatchMatMulGrad.backward
(#4393)
Installation
Documentation
- Fix sphinx build option assignment in Makefile (#4387)
- Add autosummary check to documents (#4399)
- Fix typo in
F.unpooling_nd
documentation (#4411) - Improve
F.get_item
documentation (#4415, thanks @naoto0804) - Update the comparison with other frameworks on web interfaces (#4439, thanks @lanpa!)
- Fix markup in the upgrade guide (#4440)
- Fix
Evaluator.device
documentation (#4442) - Fix English wording for extension documentation (#4444)
- Add ChainerUI link to comparison table (#4445)
- Add link to CuPy upgrade guide (#4462)
- Document missing environment variables (#4465)
- Fix document in NStepLSTM/GRU/RNN (#4490)
Examples
- Fix some options of MNIST example not working (#4390)
Tests
- Test of RNN in the case when some gradients are None (#4420)
- Add test cases for various shapes as function input (#4389)
- Fix autopep8 config (#4391)
- Add Codecov.io configuration (#4410)
Others
- Add license to source distribution (#4452)
v4.0.0b4
This is the release of v4.0.0b4. See here for the complete list of solved issues and merged PRs.
Highlights
- This release provides experimental support for iDeep that accelerates DNN computations on Intel CPUs. Currently, to run your Chainer code with iDeep enabled, install iDeep using
pip install ideep4py
, set environment variable withexport CHAINER_USE_IDEEP="auto"
, addmodel.to_intel64()
to your code (wheremodel
is aChain
object; only needed if you are using supported Optimizers) and run the code in CPU mode. Currently the following functions and optimizers are supported:- Functions:
F.relu
,F.linear
,F.local_response_normalization
,F.batch_normalization
,F.split_axis
,F.average_pooling_2d
,F.lstm
,F.tree_lstm
,F.convolution_2d
,F.deconvolution_2d
,F.max_pooling_2d
,F.dropout
,F.concat
- Optimizers:
optimizers.SGD
,optimizers.MomentumSGD
- Functions:
- Starting from v4.0.0b4, CuPy starts wheel package support. Be careful that the usual source-based installation is used if you just upgrade Chainer; see CuPy v4.0.0b4 Release Notes for how to upgrade CuPy with a wheel package.
Changes without Compatibility
Please see the Upgrade Guide for details.
- Avoid keeping reference to function nodes in CupyMemoryProfileHook (#4300)
- Avoid keeping reference to function nodes in TimerHook (#4323)
New Features
- Variable statistics plot report (#3601)
- New functions
- Deprecated array util (#3980)
- New style
ConvolutionND/DeconvolutionND
(#4110) - Add support for post-update hook registration to optimizers (#4159, thanks @tkerola!)
- iDeep support (#4276, #4277, #4278, #4279, #4280, #4281, #4282, #4283, #4284, #4285, #4286, #4288, #4289)
Improvements
- Simplify CTC code (#3842)
- Refactor RNNs (#4109)
- Fix document about links in
chainer.functions
(#4192) - Optimize
stack
(#4203) - Raise friendly error for multi-gpu doctest failures (#4217)
- Skip unnecessary backward computation in
matmul
(#4243, thanks @anaruse!) - Improve CTC loss (#4252)
- Optimize CTC backward (#4258)
- Use
variable.array
instead ofvariable.data
inreporter.py
(#4260, thanks @crcrpar!) - Add cudnn mode selection in
softmax
(#4261, thanks @anaruse!) - Fix creation method of identity matrix (#4263)
- Enable
opt = optimizers.SGD().setup()
syntax (#4290) - Avoid keeping reference to function nodes in
CupyMemoryProfileHook
(#4300) - Avoid keeping reference to function nodes in
TimerHook
(#4323) - Fix some inconsistency in gradient variable names (#4331, thanks @mizuno-gsinet!)
- Make ideep disabled by default (#4337)
- Remove redundant stack (#4338)
- Remove input type consistency checks (#4339)
- Accept concat array in NStepRNNs (#4344)
- Prefer data type objects over character codes in the NumPy (#4350)
- Support
cupyx
namespace (#4363)
Bug Fixes
- Fix crash bug when a child is reporting a value when using
MultiprocessParallelUpdater
(#3402, thanks @tkerola!) - Simplify CTC code (#3842)
- Fix
backward_accumulate
to accept only tuples (#4186) - Fix backward of
NormalizeL2
(#4190) - Fix backward of
BatchRenormalizationFunction
and add tests (#4191) - Improve
ELU.backward
(#4347) - Cast to int type in
split_axis
(#4348) - Skip
multiprocess_parallel_updater
test for timeout (#4370)
Documentation
- Fix casing of header in README (#3816)
- Deepcopy to
Link.copy
(#4066) - Update example list (#4150)
- Fixing a minor notation error on the description of
n_step_bigru
(#4157, thanks @Yuichiroh!) - Update pydoc for Optimizer setup syntax sugar (#4167)
- Add Slack Chat link to docs and issue template (#4178)
- Fix document about links in
chainer.functions
(#4192) - Update documentation and examples (#4208)
- Fix description of built-in batch converters (#4236)
- Add missing extension doc ref (#4245)
- Fix typo:
backporp
->backprop
(#4291) - Fix typo: word2vec example (#4305, thanks @koki0702!)
- Fix to pass doctest around
optimizer.setup
(#4306) - Apply https in supported websites in the document (#4315)
- Fix invisible indentation with tabs in the document (#4316)
- Use version in external URL (#4317)
- Fix docstring (#4322)
- Add function hook changes to upgrade guide (#4324)
- Fix error messages for incompatible arrays (#4346)
- Prefer data type objects over character codes in the NumPy (#4350)
Examples
- Update documentation and examples (#4208)
- Fix unused argument (
--out
) of seq2seq example (#4238, thanks @okayu9!) - Fix typo in example code (#4239, thanks @rcalland!)
Tests
- Fix initialization of random vectors in the test for RNNs (#4048)
- Raise friendly error for multi-gpu doctest failures (#4217)
- Refactor
test_batch_normalization
(#4224) - Refactor
test_dropout
(#4225) - Refactor
test_concat
(#4226) - Refactor
test_split_axis
(#4227) - Refactor
test_lstm
(#4228) - Refactor
test_local_response_normalization
(#4229) - Refactor
test_max_pooling_2d
(#4233) - Refactor
test_average_pooling_2d
(#4234) - Avoid mutable default argument (#4359)
- Skip
multiprocess_parallel_updater
test for timeout (#4370)
v3.4.0
This is the release note of v3.4.0. See here for the complete list of solved issues and merged PRs.
Enhancement
- Raise appropriate error when
Adam.lr
is evaluated before updating starts (#4207) - Remove redundant
stack
(#4340)
Bug fixes
- Serialize
t
ofUpdateRule
(#4184, #4214) - Fix a corner-case bug in
gradient_check
(#4202) - Fix backward of
NormalizeL2
(#4268) - Fix the lack of type checkings in CTC loss (#4273)
- Fix backward of
BatchRenormalizationFunction
and add tests (#4293) - Fix
backward_accumulate
to accept only tuples (#4334)
Examples
- Fix typo in example code (#4244, thanks @rcalland!)
- Fix unused argument (
--out
) of seq2seq example (#4297, thanks @okayu9!) - Update example list (#4254)
Documentation
- Improve documentation of
gru
(#3928) - Fix the docstring of
Variable.backward()
(#4196) - Prefix class name to attributes in pydoc (#4209)
- Add Slack Chat link to docs and issue template (#4255)
- Fix document about links in
chainer.functions
(#4269) - Deep copy to
Link.copy
(#4295) - Fix notation error (#4321, thanks @Yuichiroh!)
- Use version in external URL (#4327)
- Fix docstring (#4330)
- Update code in tutorials to use
init_scope()
in the mode definition (#4231)