v5.0.0b2

This is the release note of v5.0.0b2. See here for the complete list of solved issues and merged PRs.

Highlights

• New configuration value chainer.config.dtype has been introduced. This configuration can be used to switch your model to run with float16 / float32 / float64 without modifying your code. In this version of Chainer, this configuration is supported by initializers, built-in datasets and part of built-in Links. We're going to improve all built-in Links to support this feature towards the final v5 release (#4582).
• New module chainer.distributions has been introduced (see API Reference). We're going to provide more probability distribution implementations towards the final v5 release (#4678).
• Some functions (Variable operators and F.matmul) now support NumPy-style broadcasting. We're going to improve more built-in Functions to support broadcast towards the final v5 release (#4679).
• L.ConvolutionND and L.DeconvolutionND now support grouped and dilated convolution.
• Interoperability with Caffe model has been improved for both export (Deconvolution and LeakyReLU functions) and import (Deconvolution and Reshape layer).
• return_indices option has been added to F.max_pooling_2d and F.max_pooling_nd, so that you can access indices (or indexes) without using MaxPooling2D and MaxPoolingND class directly.
• chainer.datasets.TextDataset has been introduced to reduce host memory when loading large text files. See seq2seq example for an example usage.

Changes without compatibility

• Make updaters call new_epoch automatically (#4608)
  • This change should affect only a minority of users (who call Optimizer.new_epoch while using a trainer, or who implement their own updater class). See the Upgrade Guide for details.
• Fix no_grads option of check_backward (#4654)

New Features

• Add L.DeformableConvolution2D (#2468)
• Support skipping padded regions in F.average_pooling_nd (#3486)
• Support double-backprop in F.contrastive (#3830)
• Add M-SVAG optimizer (#4473!)
• Add chainer.config.dtype and use it in initializers and dataset loaders. (#4510)
• Add InverseShift extension (#4565, thanks @jinjiren!)
• Support grouped and dilated convolutions for ConvolutionND and DeconvolutionND (#4591)
• Make updaters call new_epoch automatically (#4608)
• Add group normalization (#4638, thanks @lyakaap!)
• Support remove/replace child link in ChainList (#4660, thanks @insanity!)
• Automatic broadcast in basic_math and matmul (#4679)
• Add normal distribution (#4773)
• Implement TextDataset to load line-oriented text files (#4782)
• Add CorrectedMomentumSGD optimizer (#4835)
• Support importing Caffe Reshape layer (#4875)
• Add F.diagonal (#4901)
• Add caffe import for Deconvolution layer; Fix num_output parameter in caffe export of Deconvolution layer (#4936, thanks @tsurumeso!)
• Add caffe export for LeakyReLU layer (#4937, thanks @tsurumeso!)

Enhancements

• Issue a warning when repeat attribute of some iterators in Evaluator is True (#3436)
• Use input_num to remove array creation overhead (#4162)
• Avoid using retained inputs in F.rsqrt (#4614)
• Assign user-friendly name for better type check exception message (#4680)
• Improve error message and exception type during MultiprocessParallelUpdater initialization (#4751)
• Optimize F.bilinear (#4752)
• Show progress report to stderr (#4766)
• Raise an error if neither __call__ nor forward is defined in a link (#4770)
• Output the direction of differentiation in gradient_check error (#4797)
• Allow more than 2 axis in F.normalize (#4799)
• Remove redundant device allocation in word2vec example (#4848, thanks @arisliang!)
• Deprecate F.fixed_batch_renormalization (#4869)
• Improve argument processing performance (#4880)
• Add return_indices option to F.max_pooling_2d (#4890)
• Add return_indices option to F.max_pooling_nd (#4891)
• Remove experimental warning (#4896)
• Add mask and return_mask option to F.dropout (#4907)
• Add eps and return_eps option to F.gaussian (#4909)
• Simplify type checks in F.broadcast using type_check.expect_broadcast_shapes (#4947)

Bug Fixes

• Fix stride in the ResNeXt50 example (#4479, thanks @akitotakeki!)
• Fix no_grads option of check_backward (#4654)
• Raise an error when devices argument given to MultiprocessParallelUpdater is neither dict nor list (#4716)
• Fix a stability issue in LayerNormalization (#4744, thanks @anaruse!)
• Select devices appropriately when calling user code in ParallelUpdater (#4774)
• Fix ResNetLayers to not pass-through downsample_fb argument to L.Convolution2D (#4829)
• Fix F.rollaxis backward when axis=0 and start=0 (#4836)
• Remove wrong-length check from GetItem.check_type_forward (#4845)
• Fix error message when checking types of grads (#4879, thanks @reo911gt3!)
• Fix Caffe exporter to export eps of BatchNormalization layer (#4884)
• Fix SigmoidCrossEntropy.backward (#4915)
• Fix ValueError in caffe exporter on Windows with Python 2.7 (#4928)
• Fix operations in ChainList (#4945)
• Raise an error on 0-dim input in F.matmul (#4949)

Documentation

• Improve documentation about wrapper functions (#4052)
• Add notes on differences between v1 and later versions of the extract method of vision models (ResNet, VGG16, GoogLeNet) (#4675)
• Improve CNN example docs (#4714)
• Document the pros and cons of the multi-GPU updaters (#4778)
• Add type check trouble shooting to FAQ (#4785)
• Fix comment in caffe.py (#4808, thanks @poutyface!)
• Document performance best practices (#4809)
• Fix a documenation bug in TransformDataset (#4814)
• Correct docs of batch_{re}normalization about running averages (#4818, thanks @grafi-tt!)
• Add testing utilities to documentation (#4828)
• Remove slack archive link from README.md (#4839, thanks @hidetomasuoka!)
• Document call_for_each_param attribute in optimizer hooks (#4849)
• Improve function reference and documentation (#4855)
• Add distribution reference (#4886)
• Fix gradient_check docstring to enable doctest (#4889)
• Fix normal distribution docs (#4892)
• Fix Adam alpha argument explanation (#4895)
• Improve the documentation of F.moveaxis and discourage F.rollaxis (#4900)
• Replace 0xa0 (nbsp) with 0x20 (space) in documentation (#4921)
• Update documentation to explicitly suggest using ideep4py v1 (#4934)
• Add ZippedImageDataset and MultiZippedImageDataset to documentation (#4959)

Installation

• Remove deprecated imp.load_source in setup.py (#4846, thanks @vilyaair!)

Examples

• Add pix2pix example (#4271!)
• Refactor sentiment example (#4694)
• Add serialization example (#4740)
• Simplify classification code in MNIST custom loop example (#4837)

Tests

• Fix occasional test failure of F.average by changing lower-bound of weight.sum (#4771)
• Add backprop tests (#4779)
• Fix test warning filter for Theano 1.0.2 which triggers DeprecationWarning (#4810)
• Test F.layer_normalization with large eps (#4816)
• Fix input checks in F.min and F.max tests (#4838)
• Remove flake8 from AppVeyor test (#4852)
• Set output grad in TestCUDAProfileHook (#4863)
• Stop using freeze_running_statistics in TestFixedBatchRenormalization (#4868)
• Improve batch renormalization tests (#4871)
• Mark multi_gpu tests as gpu tests (#4872)
• Deselect gpu tests instead of skipping them (#4873)
• Catch warnings in test_count_params (#4876)
• Delete redundant use of no_grads argument of check_backward from test (#4914)
• Check docs build in Travis CI (#4917)
• Tentatively restrict pytest-timeout version to <1.3.0 (#4918)
• Fix keyword argument name in TestResNetLayers (#4965)
• Remove unnecessary tests (#4966)

v4.2.0

This is the release note of v4.2.0. See here for the complete list of solved issues and merged PRs.

Highlights

• return_indices option has been added to F.max_pooling_2d and F.max_pooling_nd, so that you can access indices (or indexes) without using MaxPooling2D and MaxPoolingND classes directly.

New Features

• Add more functions with double-backprop support: F.prod (#4826) and F.contrastive (#4861)
• Support importing Caffe Reshape layer (#4905)

Enhancements

• Raise an error when CUDA context has been initialized before MultiprocessParallelUpdater is initialized (#4750)
• Output the direction of differentiation in gradient_check error (#4817)
• Show progress report to stderr (#4827)
• Optimize F.bilinear (#4834)
• Remove redundant device allocation in word2vec example (#4856, thanks @arisliang!)
• Issue a warning when repeat attribute of some iterators in Evaluator is True (#4865)
• Improve error message and exception type during MultiprocessParallelUpdater initialization (#4867)
• Raise an error if neither __call__ nor forward is defined in a link (#4888)
• Deprecate F.fixed_batch_renormalization (#4940)
• Add return_indices option to F.max_pooling_2d (#4952)
• Add return_indices option to F.max_pooling_nd (#4953)
• Add mask and return_mask option to F.dropout (#4954)
• Add eps and return_eps option to F.gaussian (#4955)

Bug Fixes

• Select devices appropriately when calling user code in ParallelUpdater (#4842)
• Fix F.rollaxis backward when axis=0 and start=0 (#4843)
• Raise an error when devices argument given to MultiprocessParallelUpdater is neither dict nor list (#4847)
• Fix error message when checking types of grads (#4885, thanks @reo911gt3!)
• Fix Caffe exporter to export eps of the BatchNormalization layer (#4913)
• Fix SigmoidCrossEntropy.backward (#4924)
• Fix ValueError in Caffe exporter on Windows with Python 2.7 (#4929)
• Remove wrong-length check from GetItem.check_type_forward (#4944)

Documentation

• Add notes on differences between v1 and later versions to the document of the extract method of vision models (ResNet, VGG16, GoogLeNet) (#4941)
• Fix some typos in tutorials (#4802, thanks @elvisyjlin!)
• Update URL in README (#4803)
• Fix comment in caffe.py (#4815, thanks @poutyface!)
• Fix a documenation bug in TransformDataset (#4819)
• Remove Slack archive link from README.md (#4840, thanks @hidetomasuoka!)
• Improve documentation about wrapper functions (#4850)
• Document call_for_each_param attribute in optimizer hooks (#4857)
• Correct docs of batch_{re}normalization about running averages (#4870, thanks @grafi-tt!)
• Add type check trouble shooting to FAQ (#4883)
• Fix gradient_check docstring to enable doctest (#4926)
• Replace 0xa0 (nbsp) with 0x20 (space) in documentation (#4930)
• Update documentation to explicitly suggest using ideep4py v1 (#4939)
• Document performance best practices (#4942)
• Improve function reference and documentation (#4943)

Installation

• Fix required version of CuPy (#4830)
• Remove deprecated imp.load_source in setup.py (#4859, thanks @vilyaair!)

Examples

• Fix custom loop examples to disable train mode during evaluation (#4804)

Tests

• Separate MultiprocessParallelUpdater tests (#4801)
• Fix test warning filter for Theano 1.0.2 which triggers DeprecationWarning (#4820)
• Fix input checks in F.min and F.max tests (#4841)
• Set output grad in TestCUDAProfileHook (#4866)
• Mark multi_gpu tests as gpu tests (#4881)
• Remove flake8 from AppVeyor test (#4882)
• Deselect gpu tests instead of skipping them (#4906)
• Stop using freeze_running_statistics in TestFixedBatchRenormalization (#4908)
• Add backprop tests (#4916)
• Tentatively restrict pytest-timeout version to <1.3.0 (#4919)
• Check docs build in Travis CI (#4931)
• Delete redundant use of no_grads argument of check_backward from test (#4938)
• Fix occasional test failure of F.average by changing lower-bound of the sum of weights (#4960)
• Remove unnecessary tests (#4967)

v5.0.0b1

This is the release notes of v5.0.0b1. See here for the complete list of solved issues and merged PRs.

New Features

• Add order_sampler option to Iterators (#3429)
• New style prod function (#3764)
• Moveaxis array operation proposal (#4112, thanks @fukatani!)
• Add n_batch_axes option to F.linear (#4204)
• Support Layer-wise Adaptive Rate Scaling (LARS) (#4237, thanks @tohmae!)
• Sparse matmul support (#4397, thanks @anaruse!)
• Optimize grouped convolution for intel64 backed environment (#4450, thanks @tkng!)
• Add ignore_names option to load_npz (#4682)
• Add the support of PolynomialShift in the extensions of training (#4693, thanks @tianshilei1992!)

Enhancements

• Use rsqrt in F.batch_normalization (#4612)
• Raise an error when CUDA has been initialized before MultiprocessParallelUpdater is initialized (#4717)
• Fix performance regression in F.bilinear (#4738)
• Support copying 1-dim array to iDeep (#4746)
• Optimize Huber loss by simpler calculation (#4775, thanks @grafi-tt!)

Bug Fixes

• Fix Chain.repeat raise error (#4649, thanks @mori97!)
• Fix ChainList.copy not supporting mode argument (#4652)
• Warn only if the BatchNormalization input tensor is 2-dimensional (#4663)
• Fix import failure when matplotlib 1.x is installed (#4681)
• Select current device using grads of outputs in addition to inputs during backward (#4725)
• Fix dtype bug in F.normalize (#4763)
• Fix lazy_grad_sum debug mode (#4768)

Documentation

• Fix heading level of Caffe docs (#4631)
• Fix typos in seq2seq tutorial document (#4643, thanks @kuni-kuni!)
• Update URL in README (#4657)
• Fix F.batch_normalization axis document (#4666)
• Fix docs and tests of L.BatchNormalization (#4671)
• Update train_loop.rst (#4700, thanks @arisliang!)
• Update math formula typo (#4702, thanks @arisliang!)
• Update word2vec.rst (#4705, thanks @arisliang!)
• Add example for transpose_sequence (#4719)
• Update seq2seq.rst (#4721, thanks @arisliang!)
• Add doc build steps to the contribution guide (#4722)
• Fix dead link to CuPy installation guide (#4724)
• Update top sentences in documentation (#4732)
• Add abs to document of Variable (#4757)
• Fix some typos in tutorials (#4760, thanks @elvisyjlin!)
• Sparse matrix documentation enhancements (#4786)

Examples

• Mini-batch training for recursive neural networks example (#2135)
• Fix custom loop examples to disable train mode during evaluation (#4568)

Tests

• Unify TestBatchNormalization and TestBatchNormalizationAxis (#4558)
• L.BatchNormalization: miscellaneous fixes (#4671)
• Support skipping tests decorated by condition or parametrize (#4685)
• Separate MultiprocessParallelUpdater tests (#4726)
• Fix hacking version (#4731)
• Fix incorrect attributes and incorrect indentation in pooling tests (#4743)

v4.1.0

This is the release note of v4.1.0. See here for the complete list of solved issues and merged PRs.

New Features

• Provide a cleaner way to collect threads and processes in MultiprocessIterator (#4637)
• Support Layer-wise Adaptive Rate Scaling (LARS) (#4668, thanks @tohmae!)
• Optimize grouped convolution for intel64 backed environment (#4787, thanks @tkng!)

Enhancements

• Improve F.rsqrt performance in CPU (#4634)
• Fix temporary file permission issue in LogReport (#4635)
• Use rsqrt in F.batch_normalization (#4665)
• Fix performance regression in F.bilinear (#4762)
• Optimize Huber loss by simpler calculation (#4788, thanks @grafi-tt!)
• Support copying 1-dim array to iDeep (#4794)

Bug Fixes

• Fix Chain.repeat raise error (#4653, thanks @mori97!)
• Fix ChainList.copy not supporting mode argument (#4664)
• Fix import failure when matplotlib 1.x is installed (#4689)
• Fix lazy_grad_sum debug mode (#4776)
• Select current device using grads of outputs in addition to inputs during backward (#4780)
• Fix dtype bug in F.normalize (#4781)

Documents

• Add tutorial for Reporter and report (#4632)
• Rewrite installation guide to align with CuPy's installation guide (#4633)
• Add higher-order derivative support of Chainer to the comparison table (#4636)
• Fix typos in seq2seq tutorial document (#4651, thanks @kuni-kuni!)
• Fix heading level of Caffe docs (#4690)
• Update word2vec.rst (#4713, thanks @arisliang!)
• Update math formula typo (#4715, thanks @arisliang!)
• Fix dead link to CuPy installation guide (#4727)
• Update seq2seq.rst (#4728, thanks @arisliang!)
• Update top sentences in documentation (#4734)
• Add example for transpose_sequence (#4745)
• Update train_loop.rst (#4759, thanks @arisliang!)
• Add abs to document of Variable (#4764)
• Add doc build steps to the contribution guide (#4784)

Tests

• Support skipping tests decorated by condition or parameterize (#4688)
• Fix hacking version (#4733)
• Fix incorrect attributes and incorrect indentation in pooling tests (#4747)

v5.0.0a1

This is the release of v5.0.0a1. See here for the complete list of solved issues and merged PRs.

New Features

• Add Sequential class for easy model definition of a single-stream computational graph (#2918)
• Add count_params() method to Link which enables to count the number of trainable values in a Link easily (#3101)
• Implement Caffe export which can export a Chainer model into Caffe protobuf format (#3631)
• Add more support on double backward: F.forget (#3792)
• Provide a cleaner way to collect threads and processes in MultiprocessIterator (#4155)
• Add axis option to batch normalization (#4266, thanks @anaruse!)
• Add add_extra option to SVHN (#4478, thanks @akitotakeki!)
• Implement Variable.xp (#4497)
• Add a new Trainer extension to kill the training when NaN or Inf is detected in the model parameters (FailOnNonNumber) (#4545)
• Add chainer.print_runtime_info() method to summarize the versions of libraries used in Chainer and CuPy (#4559)

Enhancements

• Remove submodule aliases (#4378)
• Avoid mutable default arguments (#4419)
• Avoid use of from __future__ print_function (#4470)
• Optimize F.depth2space and F.space2depth by reducing operations (#4482, thanks @ruimashita!)
• Support deserializing into array of different dtypes (#4511)
• Fix temporary file permission issue in LogReport (#4528)
• Support dilated convolution on iDeep backend (#4537, thanks @LuoYuanke!)
• Improve F.rsqrt performance in CPU (#4538)
• Improve chainer.utils.argument exception message (#4551)
• Removed redundant member variables (#4580)

Bug Fixes

• Save snapshot in the OS default permission (#4461, thanks @belltailjp!)
• Call FunctionHooks in chainer.grad (#4499)
• Remove eps from batch normalization statistics (#4505)
• Change group argument of F.convolution_2d and F.deconvolution_2d to keyword-only argument. (#4564)
• Fix to_intel64 to check if it is suitable for iDeep (#4577)
• Fix to_intel64 not updating VariableNode.data (#4592)

Documents

• Add higher-order derivative support of Chainer to the comparison table (#3477)
• Add tutorial: How to use chainer.Reporter (#3688)
• Improve docstring of F.connectionist_temporal_classification (CTC) (#4309)
• Add upgrade guide for optimizer_hooks namespace (#4468)
• Add document on using iDeep (#4477)
• Fix debug mode documentation (#4492)
• Fix order of items in reference section (#4493)
• Add a document of arithmetic special functions (#4495)
• Improve docs for supported operators in Variable (#4516)
• Fix docs of convolution_2d and deconvolution_2d (#4539)
• Add document for the configuration chainer.config.lazy_grad_sum (#4543)
• Improve description of extensions (#4552)
• Fix dead link to optimizer hooks (#4563)
• Fix typos (#4569, #4574, thanks @Hakuyume!)
• Fix document in EarlyStoppingTrigger (#4578, thanks @mori97!)
• Add FAQ for MultiprocessIterator + OpenCV problem (#4589)
• Improve docstrings about axis of Caffe functions (#4599)
• Improve docs of get_device_from_array (#4604)
• Fix docstrings of Sequential (#4605)
• Rewrite installation guide to align with CuPy's installation guide (#4622)

Installation

• Use --no-cache-dir in Dockerfile (#4532)
• Fix installation order of hacking (#4606)

Examples

• End-to-end memory networks example (#4222)
• Fix seq2seq example ignoring limitation for target length (#4611)

Tests

• Travis test against v4 branch (#4503)
• Fix doctest (#4506)
• Fix and simplify TestForwardConsistency in F.softmax_cross_entropy (#4554)
• Simplify TestSoftmaxCrossEntropyInvalidReduce in F.test_softmax_cross_entropy (#4555)

Others

• Let the StaleBot ignore issues labelled with “roadmap” (#4496)

v4.0.0

This is a major release of Chainer v4.0.0. All the updates from the previous major version (v3.5.0) are found in the release notes below:

• alpha1
• beta1
• beta2
• beta3
• beta4
• RC1

Summary of v4 update

• Major Performance Improvements
  • Support iDeep backend for acceleration on Intel CPUs. We observed that the GoogLeNet inference with batch size 1 is made x8.9 faster (compared to the case without iDeep, both on Intel(R) Xeon(R) CPU E5-2623 v3 @ 3.00GHz with MKL).
  • Support cuDNN convolution autotuning
• Better FP16 training support
  • TensorCore
  • Loss scaling
• Caffe export support (experimental)
• NCCL2 support

See the blog post for the details. Also see the Upgrade Guide for users migrating from Chainer v3 to v4.

Updates from the release candidate are as follows.

New Features

• Support double-backprop in forget function (#4522)
• Implement Variable.xp (#4536)
• Add Sequential class for easy model definition of a single-stream computational graph (#4601)
• Add extension to kill training when NaN or Inf is detected (FailOnNonNumber) (#4602)
• Implement interface to print runtime information (#4613)
• Implement Caffe export (#4621)

Enhancements

• Support deserializing into array of different dtype (#4529)
• Support dilated convolution on iDeep backend (#4540, thanks @LuoYuanke!)
• Optimize depth2space & space2depth funcs by reducing operations (#4590, thanks @ruimashita!)
• Avoid mutable default arguments (#4600)
• Improve chainer.utils.argument exception message (#4615)
• Removed redundant member variables (#4627)

Bug Fixes

• Remove eps from batch normalization statistics (#4517)
• Call FunctionHooks in chainer.grad (#4541)
• Fix group argument of {de,}convolution_2d to keyword-only argument. (#4573)
• Fix to_intel64 not updating VariableNode.data (#4597)
• Fix to_intel64 to check if it is suitable for iDeep (#4598)
• Fix seq2seq example ignoring limitation for target length (#4617)
• Save snapshot in the OS default permission (#4618, thanks @belltailjp!)
• Fix docstrings of Sequential class (#4628)

Documents

• Document arithmetic special functions (#4515)
• Fix debug mode documentation (#4519)
• Add document on using iDeep (#4520)
• Fix order of items in Reference section (#4526)
• Improve docs for supported operators in Variable (#4527)
• Improve docstring of CTC (#4531)
• Improve description of Extensions (#4561)
• Fix a typo (#4570)
• Add upgrade guide for optimizer_hooks namespace (#4571)
• Fix docs in {convolution,deconvolution}_2d (#4575)
• Fix typo (#4576, thanks @Hakuyume!)
• Fix document in EarlyStoppingTrigger (#4584, thanks @mori97!)
• Add document for chainer.config.lazy_grad_sum (#4585)
• Improve docstrings about axis of Caffe functions (#4603)
• Improve docs of get_device_from_array (#4624)
• Fix docstrings of Sequential (#4628)
• Add FAQ for MultiprocessIterator + OpenCV problem (#4629)

Installation

• use --no-cache-dir in Dockerfile (#4535)
• Fix installation order of hacking (#4609)

Examples

• fix seq2seq example ignoring limitation for target length (#4617)

Tests

• Travis test against v4 branch (#4518)
• Fix doctest (#4521)
• Simplify TestSoftmaxCrossEntropyInvalidReduce in test_softmax_cross_entropy (#4579)
• Fix installation order of hacking (#4609)
• Fix and simplify TestForwardConsistency in softmax_cross_entropy (#4626)

v4.0.0rc1

This is the release candidate of v4. See here for the complete list of solved issues and merged PRs.

Announcements

• The master branch has been switched to v5 development. The development of v4 will continue in the v4 branch.
• The major release of v4 is planned on Apr. 17.

New Features

• New differentiable functions
  • repeat (#3735)
  • fft, ifft (#4241)
  • local_convolution_2d: 2D convolution with spatially unshared weights (#4073, thanks @mihirparadkar!)
  • swish: a new activation function proposed here (#4262, thanks @mizuno-gsinet!)
  • convolution_{1,3}d, deconvolution_{1,3}d, average_pooling_{1,3}d, max_pooling_{1,3}d, unpooling_{1,3}d (#4025)
    • These are thin wrappers of ***_nd variants
• New-style (double backpropable) function support
  • simplified_dropconnect (#3807)
  • zoneout function (#3949)
  • average_pooling_nd, max_pooling_nd, unpooling_nd (#4132)
• Add testing.patch which calls mock.patch with wraps argument (#3883)
• Added initial version of AMSGrad (#4032, thanks @kashif!)
• Added ZippedImageDataset and MultiZippedImageDataset (#4127, thanks @d0i!)
• Accumulate the gradient to a tuple for lazy add operation (#4136, thanks @LuoYuanke!)
• Implement time trigger (#4294)
• Add CuPy support in ParameterStatistics; Add option to skip parameters with NaN values in ParameterStatistics (#4345)
• Spatial pyramid pooling method specified by string instead of type (#4401)
• Add n_cell property to NStepRNN family (#4417, thanks @levelfour!)

Bug Fixes

• Fix to make it consistent with numpy.split and cupy.split (#4153, thanks @ken-nakanishi!)
• Fix lstm backward arguments (#4320)
• Fix BatchMatMulGrad.backward (#4349)
• Fix test of MultiprocessParallelUpdater (#4368)
• Fix iDeep import error message not printed as expected (#4381)
• Fixed a bug that occur when using iDeep in chainer/link.py (#4382, thanks @ken-nakanishi!)
• Fix link.to_intel64 for persistent values (#4384)
• Fix the index page of datasets in documentation (#4407)
• Fix Variable tests (#4408)
• Remove Swish function class alias (#4409)
• Reset Link._device_id to None in to_intel64 (#4436)
• Fix to disable using iDeep in optimizers if to_intel64 is not called (#4457)

Installation

• Support using cupy wheels as a dependency (#4256)
• Prefer pip in installation guide; Add sphinx in docs requirements (#4366)
• Add cupy-cuda91 wheel dependency (#4392)
• Use CuPy wheels in Dockerfiles (#4406)
• Add Dockerfiles for Chainer with iDeep (#4476)

Enhancements

• Delegate cuDNN convolution operation to CuPy (#3782)
• Improve extensions part in Trainer tutorial (#3809)
• Create chainer.optimizer_hooks namespace and move hooks there. (#3977)
• Add warning for batch normalization when batchsize=1 and train=True (#3996)
• Optimize rsqrt function with CuPy (#4108)
• Fix progress bar exceeds 100% (#4152, thanks @ankokumoyashi!)
• Require "scalar" variables to be 0-dim (#4199)
• Update examples and tests for chainer.backends.cuda (#4259)
• Child sum treelstm with less than 2 child (#4275)
• Add cuDNN support for clipped_relu (#4307, thanks @tkerola!)
• Improve error messages of array type check. (#4312, thanks @mizuno-gsinet!)
• Use the same code for calculating forward and backward probabilities in CTC (#4313)
• Refactor F.split_axis (#4328)
• Refactor RNNs (#4341)
• Detect backward of old-style functions in chainer.grad (#4352)
• Extract a function that concatenates weight and bias matrixes for cuDNN (#4355)
• Fix numpy dtypes bug in cuda.to_gpu and cuda.copy (#4380, thanks @tkerola!)
• Improve ideep import error message fror missing shared objects (#4385)
• Change group argument name of Convolution2D and Deconvolution2D (#4404)
• Prefer chainer.backends.cuda in recent added codes (#4414)
• Group imports in the order mentioned in the contribution guide (#4418)
• Extract stack function from CTC loss (#4430)
• Change permission of Python files (#4432)
• Typecheck split_at (#4434, thanks @corochann!)
• Remove future.types.newint (#4435)
• Accelerate multi-add for intel64 backend (#4447, thanks @LuoYuanke!)
• Fix to use CUDNN_BN_MIN_EPSILON (#4466)
• Deprecate sep argument of PrintHook

Documents

• Reorganize tutorial & add new contents (#3241)
• Update comparison.rst to add tensorboardX as an Web interface (#3610, thanks @lanpa!)
• Document missing environment variables (#3662)
• Improve extensions part in trainer tutorial (#3809)
• Update documentation for chainer.backends.cuda (#3981)
• Fix sphinx build option assignment in Makefile (#4037)
• Add autosummary check to documents (#4040)
• Add explanation to the document of concat_examples (#4164)
• Add link to CuPy upgrade guide (#4188)
• Updated English for extension documentation (#4215, thanks @rcalland!)
• Update examples and tests for chainer.backends.cuda (#4259)
• Prefer pip in installation guide; Add sphinx in docs requirements (#4366)
• Improve docs of get item (#4375, thanks @naoto0804)
• Add F.shift to docs (#4379)
• Fix typo in unpooling_nd (#4398), in the document of configuration (#4474), in a comment of UpdateRule (#4494)
• Fix the index page of datasets in documentation (#4407)
• Fix document in NStepLSTM/GRU/RNN (#4425)
• Fix docs format in upgrade guide (#4428)
• Fix Evaluator.device docs (#4437)
• Add ChainerUI link to comparison table (#4438)
• Fix underline in docs (#4446)
• Fix trainer docs (#4464)
• Fix sectioning of "Datasets" reference (#4486)
• Fix for optimizer hook namespace change (#4488)
• Deprecate sep argument of PrintHook (#4471)

Examples

• Fix some options of MNIST example not working (#3500)
• Add a tutorial of sequence-to-sequence models (#3984)
• Update examples and tests for chainer.backends.cuda (#4259)

Tests

• Add testing.patch which calls mock.patch with wraps argument (#3883)
• Add test for inconsistent backend input outputs (#4044)
• Test of RNN in the case when some gradients are None (#4049)
• Fix test of MulprocessParallelUpdater (#4368)
• Revert "Skip MultiprocessParallelUpdater test for timeout" (#4376)
• Add backward test for old-styled sclar variable (#4416)
• Reduce UserWarning (#4483)
• Fix doctest failure in chainer.dataset.concat_examples (#4485)
• Avoid running TestSplitAxis tests on NumPy 1.10 (#4500)
• Add test cases for various shapes as function input (#3744)
• add Codecov.io configuration (#4402)

Others

• Fix autopep8 config (#4388)
• Add license to source distribution (#4451)

v3.5.0

This is the release note of v3.5.0. See here for the complete list of solved issues and merged PRs.

New Features

• Add more Functions with double-backprop support: simplified_dropconnect function (#4403), zoneout (#4423), average_pooling_nd, max_pooling_nd, unpooling_nd (#4405)

Improvements

• Accept NumPy ints as a device ID in cuda.to_gpu and cuda.copy (#4386)
• Raise warning for batch normalization when batchsize=1 and train=True (#4433)
• Add typechecking to split_at (#4441)
• Update examples and tests for chainer.backends.cuda (#4463)

Bug Fixes

• Improve ELU.backward (#4351)
• Fix crash bug when a child is reporting a value when using MultiprocessParallelUpdater (#4367)
• Fix BatchMatMulGrad.backward (#4393)

Installation

• Use pip in installation guide (#4396)
• Add sphinx in docs requirements (#4396)

Documentation

• Fix sphinx build option assignment in Makefile (#4387)
• Add autosummary check to documents (#4399)
• Fix typo in F.unpooling_nd documentation (#4411)
• Improve F.get_item documentation (#4415, thanks @naoto0804)
• Update the comparison with other frameworks on web interfaces (#4439, thanks @lanpa!)
• Fix markup in the upgrade guide (#4440)
• Fix Evaluator.device documentation (#4442)
• Fix English wording for extension documentation (#4444)
• Add ChainerUI link to comparison table (#4445)
• Add link to CuPy upgrade guide (#4462)
• Document missing environment variables (#4465)
• Fix document in NStepLSTM/GRU/RNN (#4490)

Examples

• Fix some options of MNIST example not working (#4390)

Tests

• Test of RNN in the case when some gradients are None (#4420)
• Add test cases for various shapes as function input (#4389)
• Fix autopep8 config (#4391)
• Add Codecov.io configuration (#4410)

Others

• Add license to source distribution (#4452)

v4.0.0b4

This is the release of v4.0.0b4. See here for the complete list of solved issues and merged PRs.

Highlights

• This release provides experimental support for iDeep that accelerates DNN computations on Intel CPUs. Currently, to run your Chainer code with iDeep enabled, install iDeep using pip install ideep4py, set environment variable with export CHAINER_USE_IDEEP="auto", add model.to_intel64() to your code (where model is a Chain object; only needed if you are using supported Optimizers) and run the code in CPU mode. Currently the following functions and optimizers are supported:
  • Functions: F.relu, F.linear, F.local_response_normalization, F.batch_normalization, F.split_axis, F.average_pooling_2d, F.lstm, F.tree_lstm, F.convolution_2d, F.deconvolution_2d, F.max_pooling_2d, F.dropout, F.concat
  • Optimizers: optimizers.SGD, optimizers.MomentumSGD
• Starting from v4.0.0b4, CuPy starts wheel package support. Be careful that the usual source-based installation is used if you just upgrade Chainer; see CuPy v4.0.0b4 Release Notes for how to upgrade CuPy with a wheel package.

Changes without Compatibility

Please see the Upgrade Guide for details.

• Avoid keeping reference to function nodes in CupyMemoryProfileHook (#4300)
• Avoid keeping reference to function nodes in TimerHook (#4323)

New Features

• Variable statistics plot report (#3601)
• New functions
  • erf (#3846)
  • floordiv and rfloordiv (#3967)
  • tensordot (#4253, thanks @anaruse!)
• Deprecated array util (#3980)
• New style ConvolutionND/DeconvolutionND (#4110)
• Add support for post-update hook registration to optimizers (#4159, thanks @tkerola!)
• iDeep support (#4276, #4277, #4278, #4279, #4280, #4281, #4282, #4283, #4284, #4285, #4286, #4288, #4289)

Improvements

• Simplify CTC code (#3842)
• Refactor RNNs (#4109)
• Fix document about links in chainer.functions (#4192)
• Optimize stack (#4203)
• Raise friendly error for multi-gpu doctest failures (#4217)
• Skip unnecessary backward computation in matmul (#4243, thanks @anaruse!)
• Improve CTC loss (#4252)
• Optimize CTC backward (#4258)
• Use variable.array instead of variable.data in reporter.py (#4260, thanks @crcrpar!)
• Add cudnn mode selection in softmax (#4261, thanks @anaruse!)
• Fix creation method of identity matrix (#4263)
• Enable opt = optimizers.SGD().setup() syntax (#4290)
• Avoid keeping reference to function nodes in CupyMemoryProfileHook (#4300)
• Avoid keeping reference to function nodes in TimerHook (#4323)
• Fix some inconsistency in gradient variable names (#4331, thanks @mizuno-gsinet!)
• Make ideep disabled by default (#4337)
• Remove redundant stack (#4338)
• Remove input type consistency checks (#4339)
• Accept concat array in NStepRNNs (#4344)
• Prefer data type objects over character codes in the NumPy (#4350)
• Support cupyx namespace (#4363)

Bug Fixes

• Fix crash bug when a child is reporting a value when using MultiprocessParallelUpdater (#3402, thanks @tkerola!)
• Simplify CTC code (#3842)
• Fix backward_accumulate to accept only tuples (#4186)
• Fix backward of NormalizeL2 (#4190)
• Fix backward of BatchRenormalizationFunction and add tests (#4191)
• Improve ELU.backward (#4347)
• Cast to int type in split_axis (#4348)
• Skip multiprocess_parallel_updater test for timeout (#4370)

Documentation

• Fix casing of header in README (#3816)
• Deepcopy to Link.copy (#4066)
• Update example list (#4150)
• Fixing a minor notation error on the description of n_step_bigru (#4157, thanks @Yuichiroh!)
• Update pydoc for Optimizer setup syntax sugar (#4167)
• Add Slack Chat link to docs and issue template (#4178)
• Fix document about links in chainer.functions (#4192)
• Update documentation and examples (#4208)
• Fix description of built-in batch converters (#4236)
• Add missing extension doc ref (#4245)
• Fix typo: backporp -> backprop (#4291)
• Fix typo: word2vec example (#4305, thanks @koki0702!)
• Fix to pass doctest around optimizer.setup (#4306)
• Apply https in supported websites in the document (#4315)
• Fix invisible indentation with tabs in the document (#4316)
• Use version in external URL (#4317)
• Fix docstring (#4322)
• Add function hook changes to upgrade guide (#4324)
• Fix error messages for incompatible arrays (#4346)
• Prefer data type objects over character codes in the NumPy (#4350)

Examples

• Update documentation and examples (#4208)
• Fix unused argument (--out) of seq2seq example (#4238, thanks @okayu9!)
• Fix typo in example code (#4239, thanks @rcalland!)

Tests

• Fix initialization of random vectors in the test for RNNs (#4048)
• Raise friendly error for multi-gpu doctest failures (#4217)
• Refactor test_batch_normalization (#4224)
• Refactor test_dropout (#4225)
• Refactor test_concat (#4226)
• Refactor test_split_axis (#4227)
• Refactor test_lstm (#4228)
• Refactor test_local_response_normalization (#4229)
• Refactor test_max_pooling_2d (#4233)
• Refactor test_average_pooling_2d (#4234)
• Avoid mutable default argument (#4359)
• Skip multiprocess_parallel_updater test for timeout (#4370)

v3.4.0

This is the release note of v3.4.0. See here for the complete list of solved issues and merged PRs.

Enhancement

• Raise appropriate error when Adam.lr is evaluated before updating starts (#4207)
• Remove redundant stack (#4340)

Bug fixes

• Serialize t of UpdateRule (#4184, #4214)
• Fix a corner-case bug in gradient_check (#4202)
• Fix backward of NormalizeL2 (#4268)
• Fix the lack of type checkings in CTC loss (#4273)
• Fix backward of BatchRenormalizationFunction and add tests (#4293)
• Fix backward_accumulate to accept only tuples (#4334)

Examples

• Fix typo in example code (#4244, thanks @rcalland!)
• Fix unused argument (--out) of seq2seq example (#4297, thanks @okayu9!)
• Update example list (#4254)

Documentation

• Improve documentation of gru (#3928)
• Fix the docstring of Variable.backward() (#4196)
• Prefix class name to attributes in pydoc (#4209)
• Add Slack Chat link to docs and issue template (#4255)
• Fix document about links in chainer.functions (#4269)
• Deep copy to Link.copy (#4295)
• Fix notation error (#4321, thanks @Yuichiroh!)
• Use version in external URL (#4327)
• Fix docstring (#4330)
• Update code in tutorials to use init_scope() in the mode definition (#4231)

Tests

• Fix trigger test imports (#4213)
• Raise friendly error for multi-GPU doctest failures (#4232)
• Fix initialization of random vectors in the test for RNNs (#4335)
• Avoid mutable default argument (#4361)