Read the announcement at https://blog.konduit.ai/2020/05/14/deeplearning4j-1-0-0-beta7-released/ for the highlights of this release.
- Added Keras model import support for tf.keras models Link, Link
- Full inference and training support is available for ops/layers in the tf.keras namespace; inference only for general Tensorflow operations outside of the tf.keras namespace
- Note also improvements to Keras import for reshape, permute, etc operations due to NHWC and NWC support in DL4J
- DL4J now supports NHWC (channels last) data format for all CNN 2D layers, in addition to NCHW Link
- DL4J now supports NWC (channels last - [minibatch, sequence_length, size]) for all RNN and CNN 1D layers, in addition to NCW Link
- Added Deconvolution3D layer Link
- Keras import: added ReLU, ELU and Softmax advanced activation layers Link and Swish activation function Link
- Added DL4J SameDiffLoss class (for easily-defined DL4J ILossFunction's via SameDiff) Link
- Useful exceptions are now thrown when attempting to perform unsupported operations on FastText Link
- Added MultiLayerNetwork.evaluate(MultiDataSetIterator) and .evaluateRegression(MultiDataSetIterator) methods Link, Link
- Updaters (Adam, AdaGrad, etc) optimized via C++ operations (significant training performance boost) for DL4J and SameDiff Link, Link
- Some packages relocated to avoid split packages (that can be a problem for OSGi and Java 9 modules) Link
- Note: this is a breaking change for some class packages/imports. See this link for details on exact package changes
- Deeplearning4j UI: Webjars versions locked down using dependency management to avoid check on each build Link
- Added MKLDNN (DNNL/OneDNN) support for depthwise_conv2d operation for DL4J and SameDiff Link
- Refactored/merged modules dl4j-perf and dl4j-util into deeplearning4j-core Link
- Fixed an issue with BertWordPieceTokenizer - potential StackOverflowError with certain inputs Link
- Fixed an issue with GlobalPooling layer with masks of different datatype to the activations datatype Link
- Fixed an issue with DL4JModelValidator for ComputationGraph Link
- Fixed an issue where SameDiff layers in DL4J could throw an exception when used with transfer learning Link
- Weight initialization for EmbeddingLayer and EmbeddingSequenceLayer now no longer depend on the vocabulary size (only the vector size) Link
- Fixed an issue with Keras import with bidirectional layers + preprocessors Link
- DL4J UI: added redirect from /train to /train/overview Link
- Fixed an issue where RecordReaderDataSetIterator builder collectMetaData configuration was not being applied Link
- Fixed an issue where MultiLayerNetwork evaluation was not passing metadata to the IEvaluation instances during evaluation Link, Link
- Fixed an issue with Spark training SharedTrainingMaster when training with a ComputationGraph and MultiDataSets Link
- Assorted fixes for edge cases for DL4J Keras import Link
- deelpearning4j-nlp-korean will no longer be released for Scala 2.12 due to required dependency only having Scala 2.11 version avairable Link
- Fix for ConvolutionalIterationListener for ComputationGraph Link
- Fixed an issue where dataset and model zoo downloads could get stuck if the server fails to send any data (now: timeout + retry) Link
- DL4J ModelSerializer no longer writes temporary files when restoring models from InputStream Link
- Fixes issues with UIServer multi session mode, and potential shutdown race condition Link
- Fixed an issue where TfidfVectorizer.vectorize() could throw a NPE when fit from LabelAwareIterator Link
- SameDiff multi-threaded inference enhanced (and fixed) - a single SameDiff instance can now be used for inference safely and efficiently from multiple threads Link Link
- cuDNN support added to SameDiff (automatically enabled for nd4j-cuda-10.x backend) Link
- Added ND4J namespaces: Nd4j.cnn, Nd4j.rnn, Nd4j.image Link
- Added new Image operations namespace operations:
- Added new Random operations namespace operations:
- gamma, poisson, shuffle Link
- Added new Math namespace operations:
- Added new NN namespace operations:
- cReLU Link
- Added new CNN namespace operations:
- upsampling3d Link
- Added new linalg operations namespace
- Added new RNN operation namespace operations:
- Added new Loss operations namespace - Nd4j.loss Link
- Mapped operations for Tensorflow import:
- HSVToRGB, RGBToHSV, Igamma, Igammac, RandomGamma, RandomPoisson, RandomPoissonV2, RandomShuffle Link
- Added SameDiff ProfilingListener - writes op performance profiles in Chrome profiler format (load in
chrome://tracing/) Link Link - Added SameDiff ProfileAnalyzer tool to compare profiles output from ProfilingListener (or Tensorflow) Link Link
- SameDiff listener API: added frame and iteration information for listener methods Link Link
- Added (non-backend-specific) method of accessing Nd4j environment:
Nd4j.getEnvironment()method (environment info and low-level configuration options) Link Link - Improved memory limits/configuration support for libnd4j (c++) Link
- Added pairwise (broadcastable) power backprop operation Link
- Updated JavaCPP presets MKL version to 2020.0 from 2019.5 Link
- Added DynamicCustomOp dargs - datatype arguments Link Link
- Added tensormmul_bp op Link
- OpenBLAS version upgraded to 0.3.8 Link
- libnd4j (c++ codebase underlying DL4J, ND4J and SameDiff) refactored to be more easily embeddable in other C++ projects Link
- ImagePreProcessingScaler now supports preprocessing of labels (for segmentation) Link
- Additional datatypes now supported for nd4j-tensorflow TensorflowConversion Link
- SameDiff operation namespaces (sd.math, sd.image, etc) are now code generated to ensure SameDiff and ND4J namespaces are identical (all operations included, same API) Link
- Added ND4J
ArchiveUtils.unzipFileTo(String, String, boolean logFiles)overload to enable/disable extracted file path logging Link - Added weight format configuration for following operations: conv1D, conv2D, conv3D, deconv2d, deconv3d, depthwiseConv2d, pointwiseConv2d, sconv2d Link
- Added backprop operation implementations for mergemax, mergeadd, mergeavg operations Link
- MKL version upgraded to 2020.0 2020.1; OpenCV upgraded from 4.2.0 to 4.3.0 Link
- SameDiff: DifferentialFunctionFactory class removed in favor of namespace methods (sd.math, sd.linalg, etc) Link
- Added lstmLayer_bp operation Link
- Added gru_bp operation Link
- linspace operation can now use both targs and arrays for start/end/size arguments Link
- Assorted dependency updates - OpenBLAS (0.3.9), OpenCV (4.3.0), Leptonica (1.79.0) Link
- Upgraded assorted dependency versions: javax.activation:activation (1.1 -> 1.1.1), stream analytics (2.7.0->2.9.8), Apache Spark (2.4.3->2.4.5), Jackson databind (2.10.1 -> 2.10.3), Vertx (3.8.3 -> 3.9.0) Link
- Added nd4j-common-tests ResourceUtils.listClassPathfiles method Link
- Updaters (Adam, AdaGrad, etc) optimized via C++ operations (significant training performance boost) for DL4J and SameDiff Link, Link
- SameDiff - added CuDNN support Link
- Some packages relocated to avoid split packages (that can be a problem for OSGi and Java 9 modules) Link
- Note: this is a breaking change for some class packages/imports. See this link for details on exact package changes
- Fixed some issues with Tensorflow import of FusedBatchNorm operation Link
- Fixed an issue where the Roll operation did not match Tensorflow operation Link Link
- Fixed an issue where ArchiveUtils could fail to create the top level destination directory when it does not exist Link
- Fixed an issue where resize_bicubic operation did not match Tensorflow for some configuration values Link Link
- Pad operation now supports long/int64 values for padding array Link Link
- Fixed an issue where hashcode operation shape function wasn't always returning int64/long dtype Link
- Fixed an issue with reshape operation on empty arrays with -1s Link Link
- Improved performance on CUDA for concat operation Link and CPU/GPU Link
- Improved performance for bias_add operation
- Added MKLDNN (DNNL/OneDNN) support for depthwise_conv2d operation for DL4J and SameDiff Link
- Fixed a small SameDiff execution issue for switch operation where the predicate is a constant Link
- Fixed an issue with batchnorm operation when input arrays have unusual strides Link
- Merged nd4j-buffer, nd4j-content modules into nd4j-api Link
- Deleted deprecated nd4j-jackson module (remaining functionality available in nd4j-api) Link
- Deleted unused/unmaintained nd4j-camel and nd4j-gson modules Link
- Optimization for legacy random ops Link
- Optimization for broadcast operations Link, Link, Link, Link, Link
- Performance optimization for multiple operations: softmax, squeeze, expand_dims, tanh Link
- Optimization for transpose/permute operations Link
- Performance enhancement: MKLDNN matmul used for some mmul operation cases Link
- Optimization for gather operation on CPU Link
- Optimization for stack/unstack operations on CPU Link
- Optimization for split operation (CPU and CUDA) Link Link
- ND4J initialization no longer logs number of OpenMP BLAS threads for CUDA Link
- Optimization: Fixed issues with auto-vectorization on multple CPU operations Link
- Optimization for reshape operation Link, Link
- Fixed an issue where INDArray.hashCode() could cause an exception on some datatypes Link
- Optimization for CPU: MKLDNN is now used for softmax, tanh, softmax_bp and tanh_bp operations Link, Link, Link, Link
- Fixed random_exponential operation Link
- Improved performance on C++ SameDiff graph execution via reduced array zeroing where safe to do so Link
- Improved C++ indexing implementation impacting CPU performance on some operations Link
- Fixed an issue where Split operation could have incorrect output shapes for empty arrays Link
- Fixed some issues with SameDiff.equals method Link
- Fixed an issue with reshape operation output shape on empty arrays Link, Link
- Nd4j.gemm now uses Mmul operation internally to avoid potential threading issues with direct BLAS calls on CUDA Link
- Fixed an edge case issue with percentile operation link
- Fixed an edge case issue for cusolved (CUDA) in libnd4j Link
- Fixed an issue with error formatting for segment operations for incorrect lengths Link
- Fixed an issue where ND4J workspaces were not guaranteed to be unique Link
- Fixed some operation implementations when operating on views (Batch/Space to Space/Batch/Depth; batchnorm_bp) Link
- Fixed an issue where exponential distribution random number generation operation could produce infinities extremely rarely (~1 in 10^9 values) Link
- Fixed an issue with long file paths for memory mapped workspaces on Windows Link
- Memory for memory mapped workspaces are now deallocated immediately when workspace is destroyed, instead of waiting for GC to free memory Link
- Fall-back to other BLAS implementation for cases where MKLDNN GEMM implementation is slow Link
- Set nd4j-native source/target to Java 7 Link, Link
- datavec-python: added zero-copy support for bytes/byte buffers Link
- datavec-python: Python exceptions are now thrown as Java exceptions Link
- datavec-python: Added support for additional NumPy datatypes Link
- datavec-python: Python version upgraded from 3.7.6 to 3.7.7 Link
- Deleted not properly maintained modules: datavec-camel, datavec-perf Link
- Fixed missing BOOL datatype support for arrow conversion functionality Link
- Assorted fixes for datavec-python Link Link, Link
- Fixed an issue with LineRecordReader where initialization was performed unnecessarily (adding performance overhead) Link