From ce60921f76782cddc00646905cad69f1e23b4721 Mon Sep 17 00:00:00 2001 From: Justin Long Date: Wed, 4 Apr 2018 03:29:15 -0700 Subject: [PATCH 01/30] Create ISSUE_TEMPLATE.md --- .github/ISSUE_TEMPLATE.md | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE.md diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md new file mode 100644 index 000000000..74b741374 --- /dev/null +++ b/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,9 @@ +## Due Date +*To be completed by:* YYYY-MM-DD + + +## Description +*Write a short description of what needs to be done.* + +## Assignees +*Please ensure you have assigned at least one person to this issue. Include any authors and reviewers required.* From e8e1be6d1a4b90a8b087145ede809b6a822b0727 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 10:05:41 -0700 Subject: [PATCH 02/30] Update java-ai.md --- java-ai.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/java-ai.md b/java-ai.md index 5292ebb08..516c8e259 100644 --- a/java-ai.md +++ b/java-ai.md @@ -5,6 +5,14 @@ layout: default # Artificial Intelligence (AI) for Java +## Why Java? + +And more broadly, why JVM languagues like Java, [Scala](./scala-ai.html), Clojure or Kotlin? + +Java is the most widely used programming language in the world. Large organizations in the public and private sector have enormous Java code bases, and rely heavily on the JVM as a compute environment. In particular, much of the open-source big data stack is written for the JVM. + +This includes Apache Hadoop for data storage and management, Apache Spark as a distributed run-time for fast ETL, Apache Kafka as a message queue, ElasticSearch and Cassandra to name just a few. Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why AI and machine-learning tooling is crucial. It solves a lot of integration problems, and will enable the digital transformation of many of the world's businesses. We have listed the most important machine-learning tools written in Java below. + ## Deep Learning & Neural Networks Deep learning usually refers to deep artificial neural networks. [Neural networks](https://deeplearning4j.org/neuralnet-overview) are a type of machine learning algorithm loosely modeled on the neurons in the human brain. Deep neural nets involve stacking several neural nets on top of each other to enable a feature hierarchy for more accurate classification and prediction. Deep learning is the state of the art in most tasks or machine perception, involved classification, clustering and prediction applied to raw sensory data. @@ -101,12 +109,12 @@ All machine learning libraries depend on some form of scientific computing. For people just getting started with deep learning, the following tutorials and videos provide an easy entrance to the fundamental ideas of feedforward networks: * [Introduction to Deep Neural Networks](./neuralnet-overview.html) -* [Convolutional Networks for Image Recognition](./convolutionalnets.html) +* [Convolutional Networks for Image Recognition](./convolutionalnetwork.html) * [Recurrent Networks and LSTMs](./lstm.html) * [Generative Adversarial Networks (GANs)](/generative-adversarial-network.html) * [Deep Reinforcement Learning](./deepreinforcementlearning.html) * [Symbolic Reasoning and Deep Learning](./symbolicreasoning.html) -* [Graph Data and Deep Learning](./graphdata.html) +* [Graph Data and Deep Learning](./graphanalytics.html) * [Word2vec and Natural-Language Processing](./word2vec.html) * [MNIST for Beginners](./mnist-for-beginners.html) * [Restricted Boltzmann Machines](./restrictedboltzmannmachine.html) @@ -114,5 +122,5 @@ For people just getting started with deep learning, the following tutorials and * [Glossary of Deep-Learning and Neural-Net Terms](./glossary.html) * [Deeplearning4j Examples via Quickstart](./quickstart.html) * [Artificial Intelligence (AI) for Scala](./scala-ai.html) -* [Inference: Machine Learning Model Server](./modelserver.html) +* [Inference: Machine Learning Model Server](./machine-learning-server.html) * [Multilayer Perceptron (MLPs) for Classification](./multilayerperceptron.html) From 1d31d073a23bf16bb9af1e31a8c68918214faf47 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 10:19:09 -0700 Subject: [PATCH 03/30] Create automated-machine-learning.md --- automated-machine-learning.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 automated-machine-learning.md diff --git a/automated-machine-learning.md b/automated-machine-learning.md new file mode 100644 index 000000000..a715b0574 --- /dev/null +++ b/automated-machine-learning.md @@ -0,0 +1,23 @@ +--- +title: Automated Machine Learning & AI +layout: default +--- + +# Automated Machine Learning and AI + +One of the way AI vendors try to convince companies to buy their machine learning platforms and tools is by claiming that it's automated. That's a key selling point, because most companies are acutely aware that they haven't hired enough data scientists (even they have managed to hire any data scientists at all). + +Data scientists are people who explore data, clean it, test algorithms that they think might make accurate predictions about that data, and then tune those algorithms until they work well, like an auto mechanic might tune a car. Here's a more complete list of [tasks in the machine learning workflow](./machine-learning-workflow.html). + +If the data scientists are lucky, they are given tools to perform those tasks efficiently, and they may even be enabled to deploy those trained machine-learning models to a production environment, to make predictions about data outside the lab. + +

+GET STARTED WITH MACHINE LEARNING +

+ +Many machine learning vendors, ranging from Google to startups such as Datarobot and H2O.ai, claim that they can automate machine learning. That sounds great! Then you, the hiring manager, won't need to go chasing after data science talent whose skills you can't judge in a bidding war you can't win. You'll just automate all those skills away. + +The problem is, the skills that data scientists possess are hard to automate, and people who seek to buy automated AI should be aware of what exactly can be automated, and what can't, with present technology. + + + From 01744ae71b91f3adeb321ac9035510ba64bfb380 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 10:36:54 -0700 Subject: [PATCH 04/30] Update automated-machine-learning.md --- automated-machine-learning.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index a715b0574..dd677bd99 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -17,7 +17,12 @@ If the data scientists are lucky, they are given tools to perform those tasks ef Many machine learning vendors, ranging from Google to startups such as Datarobot and H2O.ai, claim that they can automate machine learning. That sounds great! Then you, the hiring manager, won't need to go chasing after data science talent whose skills you can't judge in a bidding war you can't win. You'll just automate all those skills away. -The problem is, the skills that data scientists possess are hard to automate, and people who seek to buy automated AI should be aware of what exactly can be automated, and what can't, with present technology. +The problem is, the skills that data scientists possess are hard to automate, and people who seek to buy automated AI should be aware of what exactly can be automated, and what can't, with present technology. Data scientists perform many tasks. While automating some of those tasks may lighten their workload, unless you can automate all of their tasks, they are still necessary, and that scarce talent will remain a chokepoint that hinders the implementation of machine learning in many organizations. +## What Can We Automate in Machine Learning? +I mentioned that data scientists *tune* algorithms. When you tune a complex machine (and these algorithms are just mathematical and symbolic machines), you usually have several knobs to turn. It's kind of like cooking something with several ingredients. To produce the right taste, to tune your dish as it were, those ingredients should be added in proper proportion to one another, just like you might add twice as much [buttermilk as you do butter to a biscuit recipe](https://www.marthastewart.com/349650/biscuits). The idea is, the right proportions matter. +A data scientist is frequently operating without a "recipe", and must tune knobs in combination with each other other to explore which combination works. In this case, "working" means tuning an algorithm until it is able to learn efficiently from the data it is given to train upon. + +In data science, the knobs on an algorithm are called hyperparameters, and so the data scientists are performing "hyperparameter search". From 0d9a560588059b4c1ddb4b510ed9f4351a43af8a Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 10:41:13 -0700 Subject: [PATCH 05/30] Update java-ai.md --- java-ai.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/java-ai.md b/java-ai.md index 516c8e259..678c4d42f 100644 --- a/java-ai.md +++ b/java-ai.md @@ -9,9 +9,11 @@ layout: default And more broadly, why JVM languagues like Java, [Scala](./scala-ai.html), Clojure or Kotlin? -Java is the most widely used programming language in the world. Large organizations in the public and private sector have enormous Java code bases, and rely heavily on the JVM as a compute environment. In particular, much of the open-source big data stack is written for the JVM. +Java is the [most widely used programming language in the world](https://www.tiobe.com/tiobe-index/). Large organizations in the public and private sector have enormous Java code bases, and rely heavily on the JVM as a compute environment. In particular, much of the open-source big data stack is written for the JVM. -This includes Apache Hadoop for data storage and management, Apache Spark as a distributed run-time for fast ETL, Apache Kafka as a message queue, ElasticSearch and Cassandra to name just a few. Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why AI and machine-learning tooling is crucial. It solves a lot of integration problems, and will enable the digital transformation of many of the world's businesses. We have listed the most important machine-learning tools written in Java below. +This includes Apache Hadoop for data storage and management, Apache Spark as a distributed run-time for fast ETL, Apache Kafka as a message queue, ElasticSearch and Cassandra to name just a few. Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. + +"Accelerating digital transformation" sounds like a list of dumb buzzwords, so to paraphrase that, choosing the right tools allows you to produce more accurate predictions about data using your existing technology stack, and more accurate predictions will allow you to make better decisions. We have listed the most important machine-learning tools written in Java below. ## Deep Learning & Neural Networks From e2af6c80fe2a363b40391a7b5ab220a67a810f08 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 10:42:21 -0700 Subject: [PATCH 06/30] Update java-ai.md --- java-ai.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java-ai.md b/java-ai.md index 678c4d42f..e6bd96cbf 100644 --- a/java-ai.md +++ b/java-ai.md @@ -13,7 +13,7 @@ Java is the [most widely used programming language in the world](https://www.tio This includes Apache Hadoop for data storage and management, Apache Spark as a distributed run-time for fast ETL, Apache Kafka as a message queue, ElasticSearch and Cassandra to name just a few. Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. -"Accelerating digital transformation" sounds like a list of dumb buzzwords, so to paraphrase that, choosing the right tools allows you to produce more accurate predictions about data using your existing technology stack, and more accurate predictions will allow you to make better decisions. We have listed the most important machine-learning tools written in Java below. +"Accelerating digital transformation" sounds like a list of dumb buzzwords, so to paraphrase that, choosing the right tools allows you to produce more accurate predictions about data using your existing technology stack, and more accurate predictions will allow you to make better decisions. Those predictions might be the basis of a cool new product (self-piloting drones) or lead to big cost savings. We have listed the most important machine-learning tools written in Java below. ## Deep Learning & Neural Networks From c05e6b6add5b3172be473b586cc1e5e3b99dd674 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 10:46:36 -0700 Subject: [PATCH 07/30] Update java-ai.md --- java-ai.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java-ai.md b/java-ai.md index e6bd96cbf..395e0e7db 100644 --- a/java-ai.md +++ b/java-ai.md @@ -21,7 +21,7 @@ Deep learning usually refers to deep artificial neural networks. [Neural network ### Deeplearning4j -[Deeplearning4j](deeplearning4j.org) is the most widely used open source deep learning library for Java and the JVM. It also has a Scala API and uses Keras as its Python API for neural network configuration. The official website provides many tutorials and simple theoretical explanations for deep learning and neural networks. +[Deeplearning4j](deeplearning4j.org) is the most widely used open source deep learning library for Java and the JVM. It includes [multilayer perceptrons](./multilayerperceptron), [convolutional neural networks (CNNs) for image and text classification](./convolutionalnetwork), [recurrent neural networks such as LSTMs for text and time series data](./lstm), and [various autoencoders like VAEs and GANs](./generative-adversarial-network). Its auto-differentiation library, SameDiff, allows developers to create any neural network. It has a Scala API and uses Keras as its optional Python API. The official website provides many tutorials and simple theoretical explanations for deep learning and neural networks. Deeplearning4j includes machine-learning algorithms such as logistic regression and k-neurest neighbors.

GET STARTED WITH DEEP LEARNING From 80df3b662d66172f630c87bc8febad719cf99b37 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:15:04 -0700 Subject: [PATCH 08/30] Update java-ai.md --- java-ai.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/java-ai.md b/java-ai.md index 395e0e7db..6dd62efff 100644 --- a/java-ai.md +++ b/java-ai.md @@ -7,11 +7,11 @@ layout: default ## Why Java? -And more broadly, why JVM languagues like Java, [Scala](./scala-ai.html), Clojure or Kotlin? +And more broadly, why should you use JVM languagues like Java, [Scala](./scala-ai.html), Clojure or Kotlin to build AI and machine-learning solutions? -Java is the [most widely used programming language in the world](https://www.tiobe.com/tiobe-index/). Large organizations in the public and private sector have enormous Java code bases, and rely heavily on the JVM as a compute environment. In particular, much of the open-source big data stack is written for the JVM. +Java is the [most widely used programming language in the world](https://www.tiobe.com/tiobe-index/). Large organizations in the public and private sector have enormous Java code bases, and rely heavily on the JVM as a compute environment. In particular, much of the open-source big data stack is written for the JVM. This includes [Apache Hadoop](http://hadoop.apache.org/) for distributed data management; [Apache Spark](./spark) as a distributed run-time for fast ETL; [Apache Kafka](https://kafka.apache.org/) as a message queue; [ElasticSearch](https://www.elastic.co/), [Apache Lucene](https://lucene.apache.org/) and [Apache Solr](http://lucene.apache.org/solr/) for search; and [Apache Cassandra](http://cassandra.apache.org/) for data storage to name a few. -This includes Apache Hadoop for data storage and management, Apache Spark as a distributed run-time for fast ETL, Apache Kafka as a message queue, ElasticSearch and Cassandra to name just a few. Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. +Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. "Accelerating digital transformation" sounds like a list of dumb buzzwords, so to paraphrase that, choosing the right tools allows you to produce more accurate predictions about data using your existing technology stack, and more accurate predictions will allow you to make better decisions. Those predictions might be the basis of a cool new product (self-piloting drones) or lead to big cost savings. We have listed the most important machine-learning tools written in Java below. @@ -19,9 +19,9 @@ This includes Apache Hadoop for data storage and management, Apache Spark as a d Deep learning usually refers to deep artificial neural networks. [Neural networks](https://deeplearning4j.org/neuralnet-overview) are a type of machine learning algorithm loosely modeled on the neurons in the human brain. Deep neural nets involve stacking several neural nets on top of each other to enable a feature hierarchy for more accurate classification and prediction. Deep learning is the state of the art in most tasks or machine perception, involved classification, clustering and prediction applied to raw sensory data. -### Deeplearning4j +### Eclipse Deeplearning4j -[Deeplearning4j](deeplearning4j.org) is the most widely used open source deep learning library for Java and the JVM. It includes [multilayer perceptrons](./multilayerperceptron), [convolutional neural networks (CNNs) for image and text classification](./convolutionalnetwork), [recurrent neural networks such as LSTMs for text and time series data](./lstm), and [various autoencoders like VAEs and GANs](./generative-adversarial-network). Its auto-differentiation library, SameDiff, allows developers to create any neural network. It has a Scala API and uses Keras as its optional Python API. The official website provides many tutorials and simple theoretical explanations for deep learning and neural networks. Deeplearning4j includes machine-learning algorithms such as logistic regression and k-neurest neighbors. +[Eclipse Deeplearning4j](https://deeplearning4j.org/) is the most widely used open source deep learning library for Java and the JVM. It includes [multilayer perceptrons](./multilayerperceptron), [convolutional neural networks (CNNs) for image and text classification](./convolutionalnetwork), [recurrent neural networks such as LSTMs for text and time series data](./lstm), and [various autoencoders like VAEs and GANs](./generative-adversarial-network). Its auto-differentiation library, SameDiff, allows developers to create any neural network. It has a Scala API and uses Keras as its optional Python API. The official website provides many tutorials and simple theoretical explanations for deep learning and neural networks. Deeplearning4j includes machine-learning algorithms such as logistic regression and k-neurest neighbors.

GET STARTED WITH DEEP LEARNING From b1de36ab3d4eb467773aba616b8a5c79dc36b32d Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:16:32 -0700 Subject: [PATCH 09/30] Update java-ai.md --- java-ai.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java-ai.md b/java-ai.md index 6dd62efff..3f3106cbf 100644 --- a/java-ai.md +++ b/java-ai.md @@ -13,7 +13,7 @@ Java is the [most widely used programming language in the world](https://www.tio Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. -"Accelerating digital transformation" sounds like a list of dumb buzzwords, so to paraphrase that, choosing the right tools allows you to produce more accurate predictions about data using your existing technology stack, and more accurate predictions will allow you to make better decisions. Those predictions might be the basis of a cool new product (self-piloting drones) or lead to big cost savings. We have listed the most important machine-learning tools written in Java below. +"Accelerating digital transformation" sounds like a bunch of empty buzzwords, so let's paraphrase it. Choosing the right machine learning tools allows you to produce more accurate predictions about your data while using your existing technology stack, and those predictions will allow you to make better decisions for your business. Those predictions might be the basis of a cool new product (self-piloting drones) or lead to big cost savings. We have listed the most important machine-learning tools written in Java below. ## Deep Learning & Neural Networks From 40069d2908a0ca6734386805f775bc6e92e3b29e Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:17:38 -0700 Subject: [PATCH 10/30] Update java-ai.md --- java-ai.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/java-ai.md b/java-ai.md index 3f3106cbf..06cfa3bd3 100644 --- a/java-ai.md +++ b/java-ai.md @@ -5,13 +5,13 @@ layout: default # Artificial Intelligence (AI) for Java -## Why Java? +## Why Use Java for AI? And more broadly, why should you use JVM languagues like Java, [Scala](./scala-ai.html), Clojure or Kotlin to build AI and machine-learning solutions? Java is the [most widely used programming language in the world](https://www.tiobe.com/tiobe-index/). Large organizations in the public and private sector have enormous Java code bases, and rely heavily on the JVM as a compute environment. In particular, much of the open-source big data stack is written for the JVM. This includes [Apache Hadoop](http://hadoop.apache.org/) for distributed data management; [Apache Spark](./spark) as a distributed run-time for fast ETL; [Apache Kafka](https://kafka.apache.org/) as a message queue; [ElasticSearch](https://www.elastic.co/), [Apache Lucene](https://lucene.apache.org/) and [Apache Solr](http://lucene.apache.org/solr/) for search; and [Apache Cassandra](http://cassandra.apache.org/) for data storage to name a few. -Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. +Since access to data is a prequisite to building AI and machine-learning solutions, AI tools need to integrate well with those technologies. AI starts with the data you gather. That's why the AI and machine-learning tooling you choose is crucial. The right tools solve a lot of integration problems (many data science projects fail when prototypes can't integrate with the production stack), and they will accelerate the digital transformation of many of the world's businesses. "Accelerating digital transformation" sounds like a bunch of empty buzzwords, so let's paraphrase it. Choosing the right machine learning tools allows you to produce more accurate predictions about your data while using your existing technology stack, and those predictions will allow you to make better decisions for your business. Those predictions might be the basis of a cool new product (self-piloting drones) or lead to big cost savings. We have listed the most important machine-learning tools written in Java below. From f2fe45e9a688cc29a0602ccd023581f9b11577ae Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:20:45 -0700 Subject: [PATCH 11/30] Update automated-machine-learning.md --- automated-machine-learning.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index dd677bd99..f6c4fab9d 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -25,4 +25,20 @@ I mentioned that data scientists *tune* algorithms. When you tune a complex mach A data scientist is frequently operating without a "recipe", and must tune knobs in combination with each other other to explore which combination works. In this case, "working" means tuning an algorithm until it is able to learn efficiently from the data it is given to train upon. -In data science, the knobs on an algorithm are called hyperparameters, and so the data scientists are performing "hyperparameter search". +### Hyperparameter Optimization + +In data science, the knobs on an algorithm are called hyperparameters, and so the data scientists are performing "hyperparameter search" as they test different combinations of those hyperparameters, different ratios between their ingredients. + +Hyperparameter search can be automated. [Eclipse Arbiter](https://github.com/deeplearning4j/arbiter) is a hyperparameter optimization library designed to automate hyperparameter tuning for deep neural net training. It is the equivalent of Google Tensorflow's Vizier, or the open-source Python library Spearmint. Arbiter is part of the Deeplearning4j framework. Some startups, like [SigOpt](https://sigopt.com/), are focused solely on hyperparameter optimization. + +You can search for the best combination of hyperparameters with different kinds of search algorithm, like grid search, random search and Bayesian methods. + +### Algorithm Selection + + + +### Transfer Learning and Pre-Trained Models + +### Limited Use Cases + +### Professional Services From 1801adc931ff60722218d31936908099f4a0f9b5 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:28:02 -0700 Subject: [PATCH 12/30] Update java-ai.md --- java-ai.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/java-ai.md b/java-ai.md index 06cfa3bd3..2e75aafd0 100644 --- a/java-ai.md +++ b/java-ai.md @@ -63,7 +63,7 @@ Machine learning encompasses a wide range of algorithms that are able to adapt t ### SMILE -[SMILE](https://github.com/haifengl/smile) stands for Statistical and Machine Intelligence Learning Engine. SMILE was create by Haifeng Lee, and provides fast, scalable machine learning for Java. +[SMILE](https://github.com/haifengl/smile) stands for Statistical and Machine Intelligence Learning Engine. SMILE was create by Haifeng Lee, and provides fast, scalable machine learning for Java. SMILE uses ND4J to perform scientific computing for large-scale tensor manipulations. It includes algorithms such as support vector machines (SVMs), [decision trees](./decision-tree), [random forests](./random-forest) and gradient boosting, among others. ### SINGA @@ -79,7 +79,7 @@ Machine learning encompasses a wide range of algorithms that are able to adapt t ### Weka  -[Weka](http://www.cs.waikato.ac.nz/ml/weka/) is a collection of machine learning algorithms that can be applied directly to a dataset, through the Weka GUI or API. The WEKA community is large, providing various tutorials for Weka and machine learning itself. +[Weka](http://www.cs.waikato.ac.nz/ml/weka/) is a collection of machine learning algorithms that can be applied directly to a dataset, through the Weka GUI or API. The WEKA community is large, providing various tutorials for Weka and machine learning itself. WEKA uses Deeplearning4j for its neural network implementation. ### MOA (Massive On-line Analysis) [MOA (Massive On-line Analysis)](https://moa.cms.waikato.ac.nz/) is for mining data streams. @@ -111,12 +111,12 @@ All machine learning libraries depend on some form of scientific computing. For people just getting started with deep learning, the following tutorials and videos provide an easy entrance to the fundamental ideas of feedforward networks: * [Introduction to Deep Neural Networks](./neuralnet-overview.html) -* [Convolutional Networks for Image Recognition](./convolutionalnetwork.html) +* [Convolutional Neural Networks (CNNs) for Image Recognition](./convolutionalnetwork.html) * [Recurrent Networks and LSTMs](./lstm.html) * [Generative Adversarial Networks (GANs)](/generative-adversarial-network.html) * [Deep Reinforcement Learning](./deepreinforcementlearning.html) * [Symbolic Reasoning and Deep Learning](./symbolicreasoning.html) -* [Graph Data and Deep Learning](./graphanalytics.html) +* [Graph Data Analytics and Deep Learning](./graphanalytics.html) * [Word2vec and Natural-Language Processing](./word2vec.html) * [MNIST for Beginners](./mnist-for-beginners.html) * [Restricted Boltzmann Machines](./restrictedboltzmannmachine.html) From 29d2bf75b37c452221287861458da1f2790b1ccc Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:45:27 -0700 Subject: [PATCH 13/30] Update java-ai.md --- java-ai.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/java-ai.md b/java-ai.md index 2e75aafd0..04fab56d2 100644 --- a/java-ai.md +++ b/java-ai.md @@ -1,9 +1,9 @@ --- -title: Artificial Intelligence (AI) for Java +title: Artificial Intelligence (AI) and Machine Learning for Java layout: default --- -# Artificial Intelligence (AI) for Java +# Artificial Intelligence (AI) and Machine Learning for Java ## Why Use Java for AI? From a3827f95424da926a5f938850da17880bcdee3c4 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 11:54:09 -0700 Subject: [PATCH 14/30] Update automated-machine-learning.md --- automated-machine-learning.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index f6c4fab9d..ec16c4945 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -35,10 +35,24 @@ You can search for the best combination of hyperparameters with different kinds ### Algorithm Selection +One thing that AI vendors will do is run the same data through several algorithms whose hyperparameters are set by default, to determine which algorithm can learn best on your data. At the end of the contest, they select the winner. Visualizing these algorithmic beauty contests is a dramatic way to show the work being done. However, it has its limits, notably in the range of algorithms that are chosen to run in any given race, and how well they are tuned. +### Limited Use Cases on the Happy Path + +AI vendors can be smart about the algorithms they select only if they have some knowledge of the problem that is being solved, and the data that is being used to train the algorithm. In many real-world situations, lengthy data exploration and some domain-specific knowledge are necessary to select the right algorithms. + +In the world of automated machine learning, we pretend that data exploration and domain knowledge don't matter. We can only do that for a few limited use cases. In software, this is called the [happy path](https://en.wikipedia.org/wiki/Happy_path), or the use case where everything goes as we expect it to. Automated machine learning has a narrow happy path; that is, it's easy to step off the path and get into trouble. + +For example, it's easy to automate machine learning for a simple use case like scoring your leads to Salesforce to predict the likelihood that you will close a sale. That's because the schema of the data -- the things you know about your customers -- is constrained by Salesforce software and fairly standardized across sales teams. An automated machine learning solution focused on lead scoring can make strong assumptions about the type of data you will feed it. + +But companies need machine learning for more than lead scoring. Their use cases differ, and so does their data. In those cases, it can be hard to offer a pre-baked solution. + +Those companies have two choices: they can hire their own data scientists or rely on processional services from consulting firms. ### Transfer Learning and Pre-Trained Models -### Limited Use Cases + + + ### Professional Services From 9f99913899c180632822ccdd0710de6235a4276d Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:14:49 -0700 Subject: [PATCH 15/30] Update automated-machine-learning.md --- automated-machine-learning.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index ec16c4945..148838d85 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -47,12 +47,17 @@ For example, it's easy to automate machine learning for a simple use case like s But companies need machine learning for more than lead scoring. Their use cases differ, and so does their data. In those cases, it can be hard to offer a pre-baked solution. -Those companies have two choices: they can hire their own data scientists or rely on processional services from consulting firms. +Those companies have two choices: they can hire their own data scientists or rely on processional services from consulting firms. Every major public cloud vendor has introduced machine-learning solutions teams in an attempt to close the talent gap and make machine learning more available to potential users of their clouds. The major consultancies, from Accenture to Bain, have hired teams of data scientists to build solutions for their clients. Even automated machine-learning startups like Data Robot offer "Customer-facing Data Scientists". -### Transfer Learning and Pre-Trained Models +So a lot of time, AI vendors that sell automated machine learning are really "automating" those tasks with humans. This is because the tasks and decisions involved in building AI solutions are many, varied and complex, and the technology does not yet exist to automate all of them. +### Transfer Learning and Pre-Trained Models +Machine learning models start out dumb and get smart by being exposed to data that they "train" on. Training involves making guesses about the data, measuring the error in their guesses, and correcting themselves until they make more accurate guesses. Machine learning algorithms train on data to produce an accurate "model" of the data. A trained, accurate model of the data is one that is capable of producing good predictions when it is fed new data that resembles what it trained on. For the purposes of this discussion, imagine a model as a black box that performs a mathematical operation on data to make a prediction about it. The data goes into the model, the prediction comes out; e.g. feed an image of one of your friends into the model, and it will predict the name of the friend in the image. +Sometimes, you can train a machine-learning model on one set of data, and then use it for another, slightly different set of data later. This only works when the two datasets resemble each other. For example, most photographs have certain characteristics in common. If you train a machine-learning model on, say, celebrity faces, it will learn what humans look like, and with just a little extra learning, you could teach it to transfer what it knows to photographs of your family and friends, whom it has never seen before. +Pre-trained machine-learning models that gain some knowledge of the world are useful in computer vision, and widely available. Some well-known pre-trained computer vision models include AlexNet, LeNet, VGG16, YOLO and Inception. [Those pre-trained computer vision models are available here](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model). Google's [Cloud AutoMl](https://cloud.google.com/automl/) relies on transfer learning, among other methods, to support its claim that it has "automated machine learning." ### Professional Services + From 6c1cfb9efd57ab4e84d0060fc7440180e11d6847 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:16:45 -0700 Subject: [PATCH 16/30] Update automated-machine-learning.md --- automated-machine-learning.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index 148838d85..e8a532255 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -47,6 +47,8 @@ For example, it's easy to automate machine learning for a simple use case like s But companies need machine learning for more than lead scoring. Their use cases differ, and so does their data. In those cases, it can be hard to offer a pre-baked solution. +### Professional Services + Those companies have two choices: they can hire their own data scientists or rely on processional services from consulting firms. Every major public cloud vendor has introduced machine-learning solutions teams in an attempt to close the talent gap and make machine learning more available to potential users of their clouds. The major consultancies, from Accenture to Bain, have hired teams of data scientists to build solutions for their clients. Even automated machine-learning startups like Data Robot offer "Customer-facing Data Scientists". So a lot of time, AI vendors that sell automated machine learning are really "automating" those tasks with humans. This is because the tasks and decisions involved in building AI solutions are many, varied and complex, and the technology does not yet exist to automate all of them. @@ -58,6 +60,3 @@ Machine learning models start out dumb and get smart by being exposed to data th Sometimes, you can train a machine-learning model on one set of data, and then use it for another, slightly different set of data later. This only works when the two datasets resemble each other. For example, most photographs have certain characteristics in common. If you train a machine-learning model on, say, celebrity faces, it will learn what humans look like, and with just a little extra learning, you could teach it to transfer what it knows to photographs of your family and friends, whom it has never seen before. Pre-trained machine-learning models that gain some knowledge of the world are useful in computer vision, and widely available. Some well-known pre-trained computer vision models include AlexNet, LeNet, VGG16, YOLO and Inception. [Those pre-trained computer vision models are available here](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model). Google's [Cloud AutoMl](https://cloud.google.com/automl/) relies on transfer learning, among other methods, to support its claim that it has "automated machine learning." - -### Professional Services - From 73eff9ada99e47e2e627814af03a34b207013caa Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:17:47 -0700 Subject: [PATCH 17/30] Update automated-machine-learning.md --- automated-machine-learning.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index e8a532255..00498d3e9 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -57,6 +57,6 @@ So a lot of time, AI vendors that sell automated machine learning are really "au Machine learning models start out dumb and get smart by being exposed to data that they "train" on. Training involves making guesses about the data, measuring the error in their guesses, and correcting themselves until they make more accurate guesses. Machine learning algorithms train on data to produce an accurate "model" of the data. A trained, accurate model of the data is one that is capable of producing good predictions when it is fed new data that resembles what it trained on. For the purposes of this discussion, imagine a model as a black box that performs a mathematical operation on data to make a prediction about it. The data goes into the model, the prediction comes out; e.g. feed an image of one of your friends into the model, and it will predict the name of the friend in the image. -Sometimes, you can train a machine-learning model on one set of data, and then use it for another, slightly different set of data later. This only works when the two datasets resemble each other. For example, most photographs have certain characteristics in common. If you train a machine-learning model on, say, celebrity faces, it will learn what humans look like, and with just a little extra learning, you could teach it to transfer what it knows to photographs of your family and friends, whom it has never seen before. +Sometimes, you can train a machine-learning model on one set of data, and then use it for another, slightly different set of data later. This only works when the two datasets resemble each other. For example, most photographs have certain characteristics in common. If you train a machine-learning model on, say, celebrity faces, it will learn what humans look like, and with just a little extra learning, you could teach it to transfer what it knows to photographs of your family and friends, whom it has never seen before. Using a pre-trained model could save you the cost of training your own over thousands of hours on distributed GPUs, an expensive proposition. -Pre-trained machine-learning models that gain some knowledge of the world are useful in computer vision, and widely available. Some well-known pre-trained computer vision models include AlexNet, LeNet, VGG16, YOLO and Inception. [Those pre-trained computer vision models are available here](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model). Google's [Cloud AutoMl](https://cloud.google.com/automl/) relies on transfer learning, among other methods, to support its claim that it has "automated machine learning." +Pre-trained machine-learning models that gain some knowledge of the world are useful in computer vision, and widely available. Some well-known pre-trained computer vision models include AlexNet, LeNet, VGG16, YOLO and Inception. [Those pre-trained computer vision models are available here](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model). Google's [Cloud AutoML](https://cloud.google.com/automl/) relies on transfer learning, among other methods, to support its claim that it has "automated machine learning." From 5b20e053b7138729828367101b196bee0911e0f4 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:24:35 -0700 Subject: [PATCH 18/30] Update automated-machine-learning.md --- automated-machine-learning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index 00498d3e9..fff4feed4 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -51,7 +51,7 @@ But companies need machine learning for more than lead scoring. Their use cases Those companies have two choices: they can hire their own data scientists or rely on processional services from consulting firms. Every major public cloud vendor has introduced machine-learning solutions teams in an attempt to close the talent gap and make machine learning more available to potential users of their clouds. The major consultancies, from Accenture to Bain, have hired teams of data scientists to build solutions for their clients. Even automated machine-learning startups like Data Robot offer "Customer-facing Data Scientists". -So a lot of time, AI vendors that sell automated machine learning are really "automating" those tasks with humans. This is because the tasks and decisions involved in building AI solutions are many, varied and complex, and the technology does not yet exist to automate all of them. +So a lot of time, AI vendors that sell automated machine learning are really "automating" those tasks with humans; i.e. that is, they're allowing their clients to outsource the talent that their clients can't otherwise get access to. This is because the tasks and decisions involved in building AI solutions are many, varied and complex, and the technology does not yet exist to automate all of them. That's automation, it's services. We should call it what it is and recognize that buiding machine-learing often requires the refined judgment of experts, combined with automation for a few narrow tasks in a larger AI workflow. ### Transfer Learning and Pre-Trained Models From 7f9cad0d5a72a39de242b9e05d05e5e9ef53716c Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:26:38 -0700 Subject: [PATCH 19/30] Update automated-machine-learning.md --- automated-machine-learning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/automated-machine-learning.md b/automated-machine-learning.md index fff4feed4..1979a9c43 100644 --- a/automated-machine-learning.md +++ b/automated-machine-learning.md @@ -45,7 +45,7 @@ In the world of automated machine learning, we pretend that data exploration and For example, it's easy to automate machine learning for a simple use case like scoring your leads to Salesforce to predict the likelihood that you will close a sale. That's because the schema of the data -- the things you know about your customers -- is constrained by Salesforce software and fairly standardized across sales teams. An automated machine learning solution focused on lead scoring can make strong assumptions about the type of data you will feed it. -But companies need machine learning for more than lead scoring. Their use cases differ, and so does their data. In those cases, it can be hard to offer a pre-baked solution. +But companies need machine learning for more than lead scoring. Their use cases differ, and so does their data. In those cases, it can be hard to offer a pre-baked solution. Data pipelines, also known as ETL, are often the stage of the AI workflow that require the most human attention. The real world is messy and data, which represents that world, is usually messy, too. Most datasets need to be explored, cleaned and otherwise pre-processed before that data can be fruitfully used to train a machine-learning algorithm. That cleaning and exploration often requires expert humans. ### Professional Services From 16798426fa5fefef3f0604944d44c86b545f4f92 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:27:17 -0700 Subject: [PATCH 20/30] Rename automated-machine-learning.md to automated-machine-learning-ai.md --- automated-machine-learning.md => automated-machine-learning-ai.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename automated-machine-learning.md => automated-machine-learning-ai.md (100%) diff --git a/automated-machine-learning.md b/automated-machine-learning-ai.md similarity index 100% rename from automated-machine-learning.md rename to automated-machine-learning-ai.md From 670b061a918ca6a0e23263ac7ad47020e0534e41 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:31:08 -0700 Subject: [PATCH 21/30] Update index.html --- _layouts/index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_layouts/index.html b/_layouts/index.html index 123050bd1..021116899 100644 --- a/_layouts/index.html +++ b/_layouts/index.html @@ -67,7 +67,7 @@

What is Eclipse Deeplearning4j

Welcome to Eclipse Deeplearning4j

-

Deeplearning4j aims to be cutting-edge plug and play, more convention than configuration, which allows for fast prototyping for non-researchers. DL4J is customizable at scale. Released under the Apache 2.0 license, all derivatives of DL4J belong to their authors. DL4J can import neural net models from most major frameworks via Keras, including TensorFlow, Caffe and Theano, bridging the gap between the Python ecosystem and the JVM with a cross-team toolkit for data scientists, data engineers and DevOps. Keras is employed as Deeplearning4j's Python API. Skymind is the second-largest contributor to Keras after Google, and offers commercial support for Keras. Machine learning models are served in production with Skymind's machine learning server.

+

Deeplearning4j aims to be cutting-edge plug and play, more convention than configuration, which allows for fast prototyping for data scientists, machine-learning practitioners and software engineers. DL4J is customizable at scale. Released under the Apache 2.0 license, all derivatives of DL4J belong to their authors. DL4J can import neural net models from most major frameworks via Keras, including TensorFlow, Caffe and Theano, bridging the gap between the Python ecosystem and the JVM with a cross-team toolkit for data scientists, data engineers and DevOps. Keras is Deeplearning4j's Python API. Skymind is the second-largest contributor to Keras after Google, and offers commercial support for Keras. Machine learning models are served in production with Skymind's machine learning server.

From 49e68fc473fb4a56347184b7095e399166495a5c Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 12:35:05 -0700 Subject: [PATCH 22/30] Update sidebar.html --- _includes/sidebar.html | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/_includes/sidebar.html b/_includes/sidebar.html index 3b822b013..25bb89992 100644 --- a/_includes/sidebar.html +++ b/_includes/sidebar.html @@ -31,6 +31,7 @@
  • Build a Web Application for Image Classification
  • Deploy Deeplearning4j to Android
  • What is Artificial Intelligence (AI)?
    • +
  • What is Strong AI?
    • @@ -52,11 +53,12 @@
  • Neural Networks
      -
    • Long Short-Term Memory Units
      • +
    • Long Short-Term Memory Units (LSTMs)
    • Convolutional Nets for Image Processing
      • -
    • Recurrent Nets and LSTMs
      • -
    • Word2Vec: Neural Word Embeddings
      • +
    • Recurrent Neural Networks (RNNs)
      • +
    • Word2Vec, Doc2vec, GloVe: Neural Word Embeddings
    • Restricted Boltzmann Machines
      • +
    • Generative Adversarial Network (GAN)
    • Multilayer Perceptron
    • Deep AutoEncoder
    • Denoising Autoencoders
      • @@ -158,6 +160,8 @@
    • Decision Trees
    • Random Forests
    • Scala, Spark and Deep Learning
      • +
    • Java AI and Machine Learning Tools
      • +
    • Automated Machine Learning and AI
    • DL4J, TensorFlow, Pytorch, Caffe
    • Glossary of Terms for Deep Learning and Neural Nets
    • Free Online Courses, Tutorials and Papers
      • From f6873b1e141282ebbc428ab2dda7f2066a380371 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 13:33:41 -0700 Subject: [PATCH 23/30] Update robotic-process-automation-rpa.md --- robotic-process-automation-rpa.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/robotic-process-automation-rpa.md b/robotic-process-automation-rpa.md index 10c61e670..44cd700b4 100644 --- a/robotic-process-automation-rpa.md +++ b/robotic-process-automation-rpa.md @@ -1,9 +1,9 @@ --- -title: Robotic Process Automation (RPA), AI and Deep Learning +title: Robotic Process Automation (RPA), AI and Machine Learning layout: default --- -# Robotic Process Automation (RPA), AI and Deep Learning +# Robotic Process Automation (RPA), AI and Machine Learning RPA automates business workflows, or clerical processes, by emulating human interaction within a graphical user interface (GUI). From 2899bf0948eb552b0584ebc4ec5050aa62fea006 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Wed, 4 Apr 2018 15:44:46 -0700 Subject: [PATCH 24/30] Update opendata.md --- opendata.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/opendata.md b/opendata.md index 8da5b73fa..082ed20fb 100644 --- a/opendata.md +++ b/opendata.md @@ -1,9 +1,9 @@ --- -title: Open Datasets for Deep Learning +title: Open Datasets for Deep Learning & Machine Learning layout: default --- -# Open Data for Deep Learning +# Open Data for Deep Learning & Machine Learning Here you'll find an organized list of interesting, high-quality datasets for machine learning research. We welcome your contributions for [curating this list](https://github.com/deeplearning4j/deeplearning4j/blob/gh-pages/opendata.md)! You can find other lists of such datasets [on Wikipedia](https://en.wikipedia.org/wiki/List_of_datasets_for_machine_learning_research), for example. @@ -19,10 +19,13 @@ Here you'll find an organized list of interesting, high-quality datasets for mac * [Open Data Monitor](https://opendatamonitor.eu/) * [Quandl Data Portal](https://www.quandl.com/) * [Mut1ny Face/Head segmentation dataset] (http://www.mut1ny.com/face-headsegmentation-dataset) + +

      GET STARTED WITH DEEP LEARNING

      + ## Natural-Image Datasets * [MNIST: handwritten digits](http://yann.lecun.com/exdb/mnist/): The most commonly used sanity check. Dataset of 25x25, centered, B&W handwritten digits. It is an easy task — just because something works on MNIST, doesn’t mean it works. From b8846d4921d877c0ebc65ef474c2aa198354982d Mon Sep 17 00:00:00 2001 From: Marat Gubaidullin Date: Wed, 4 Apr 2018 22:20:11 -0400 Subject: [PATCH 25/30] Fixes #17 (#18) --- _layouts/index.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_layouts/index.html b/_layouts/index.html index 021116899..25d146186 100644 --- a/_layouts/index.html +++ b/_layouts/index.html @@ -60,7 +60,7 @@

      Open-Source, Distributed, Deep Learning Library for the JVM

      What is Eclipse Deeplearning4j?

      -

      Eclipse Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Spark, DL4J brings AIAI to business environments for use on distributed GPUs and CPUs.

      +

      Eclipse Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Spark, DL4J brings AI to business environments for use on distributed GPUs and CPUs.

      Skymind is its commercial support arm, bundling Deeplearning4j and other libraries such as Tensorflow and Keras in the Skymind Intelligence Layer (Community Edition), a deep learning environment that gives developers an easy, fast way to train and deploy AI models. SKIL CE is free and downloadable here. SKIL acts as a bridge between Python data science environments and the JVM.

      @@ -90,7 +90,7 @@

      Open-Source

      JVM/Python/C++

      -

      Deeplearning4j is written in Java and is compatible with any JVM language, such as Scala, Clojure or Kotlin. The underlying computations are written in C, C++ and Cuda. Keras will serve as the Python API.

      +

      Deeplearning4j is written in Java and is compatible with any JVM language, such as Scala, Clojure or Kotlin. The underlying computations are written in C, C++ and Cuda. Keras will serve as the Python API.

    • @@ -189,7 +189,7 @@
    DL4J's Neural Networks
    Deep Convolutional Networks (CNNs) Multilayer Perceptron (MLP) for classification Recurrent Nets (RNNs) - Word2vec: Extracting Relations From Raw Text + Word2vec: Extracting Relations From Raw Text Generative Adversarial Networks (GANs) Restricted Boltzmann Machines Deep-Belief Networks (DBN) From 1c0bb38e754c0839c79997497deb9e78ab71e33d Mon Sep 17 00:00:00 2001 From: ChrisN Date: Thu, 5 Apr 2018 04:05:13 -0700 Subject: [PATCH 26/30] Update spark.md --- spark.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spark.md b/spark.md index 95879471f..5535514a8 100644 --- a/spark.md +++ b/spark.md @@ -1,9 +1,9 @@ --- -title: Deeplearning4j on Spark +title: Deep Learning on Apache Spark layout: default --- -# Deeplearning4j on Spark +# Deep Learning on Apache Spark Deep learning is computationally intensive, so on very large datasets, speed matters. You can tackle the problem with faster hardware (usually GPUs), optimized code and some form of parallelism. From d46f19f7e74bb273d9def740ddceb7ef98704ac5 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Thu, 5 Apr 2018 04:06:38 -0700 Subject: [PATCH 27/30] Update lstm.md --- lstm.md | 1 + 1 file changed, 1 insertion(+) diff --git a/lstm.md b/lstm.md index 15039aa6d..cd28060c1 100644 --- a/lstm.md +++ b/lstm.md @@ -231,6 +231,7 @@ Here are a few ideas to keep in mind when manually optimizing hyperparameters fo * [Introduction to Decision Trees](./decision-tree.html) * [Introduction to Random Forests](./random-forest.html) * [Open Datasets for Machine Learning](./opendata.html) +* [Deep Learning on Apache Spark](./spark.html) * [AI vs. Machine Learning vs. Deep Learning](./ai-machinelearning-deeplearning.html) * [Inference in Production: Machine Learning Model Server](./machine-learning-server.html) From b24032e0b66bafa92024e26a3b500998d0097168 Mon Sep 17 00:00:00 2001 From: ChrisN Date: Thu, 5 Apr 2018 04:45:14 -0700 Subject: [PATCH 28/30] Update convolutionalnetwork.md --- convolutionalnetwork.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/convolutionalnetwork.md b/convolutionalnetwork.md index df9ebb47d..443cab30a 100755 --- a/convolutionalnetwork.md +++ b/convolutionalnetwork.md @@ -1,24 +1,24 @@ --- -title: A Beginner's Guide to Deep Convolutional Networks (CNNs) +title: A Beginner's Guide to Deep Convolutional Neural Networks (CNNs) layout: default redirect_from: convolutionalnets --- -# A Beginner's Guide to Deep Convolutional Networks (CNNs) +# A Beginner's Guide to Deep Convolutional Neural Networks (CNNs) Contents -* Deep Convolutional Network Introduction +* Deep Convolutional Neural Network Introduction * Images Are 4-D Tensors? -* Convolutional Net Definition -* How Deep Convolutional Networks Work +* Convolutional Neural Network Definition +* How Deep Convolutional Neural Networks Work * Maxpooling/Downsampling * Just Show Me the Code * More CNN Resources -## Introduction to Deep Convolutional Networks +## Introduction to Deep Convolutional Neural Networks -Convolutional networks are deep artificial neural networks that are used primarily to classify images (e.g. name what they see), cluster them by similarity (photo search), and perform object recognition within scenes. They are algorithms that can identify faces, individuals, street signs, tumors, platypuses and many other aspects of visual data. +Convolutional neural networks are deep artificial neural networks that are used primarily to classify images (e.g. name what they see), cluster them by similarity (photo search), and perform object recognition within scenes. They are algorithms that can identify faces, individuals, street signs, tumors, platypuses and many other aspects of visual data. Convolutional networks perform optical character recognition (OCR) to digitize text and make natural-language processing possible on analog and hand-written documents, where the images are symbols to be transcribed. CNNs can also be applied to sound when it is represented visually as a spectrogram. More recently, convolutional networks have been applied directly to [text analytics](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/) as well as graph data with [graph convolutional networks](./graphanalytics). @@ -30,7 +30,7 @@ The efficacy of convolutional nets (ConvNets or CNNs) in image recognition is on ## Images Are 4-D Tensors? -Convolutional nets ingest and process images as tensors, and tensors are matrices of numbers with additional dimensions. +Convolutional neural networks ingest and process images as tensors, and tensors are matrices of numbers with additional dimensions. They can be hard to visualize, so let’s approach them by analogy. A scalar is just a number, such as 7; a vector is a list of numbers (e.g., `[7,8,9]`); and a matrix is a rectangular grid of numbers occupying several rows and columns like a spreadsheet. Geometrically, if a scalar is a zero-dimensional point, then a vector is a one-dimensional line, a matrix is a two-dimensional plane, a stack of matrices is a three-dimensional cube, and when each element of those matrices has a stack of *feature maps* atttached to it, you enter the fourth dimension. For reference, here’s a 2 x 2 matrix: @@ -181,7 +181,7 @@ All Deeplearning4j [examples of convolutional networks are available here](https ## Other Resources -To see DL4J convolutional networks in action, please run our [examples](https://github.com/deeplearning4j/dl4j-examples/tree/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/) after following the instructions on the [Quickstart page](./quickstart). +To see DL4J convolutional neural networks in action, please run our [examples](https://github.com/deeplearning4j/dl4j-examples/tree/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/) after following the instructions on the [Quickstart page](./quickstart). Skymind wraps NVIDIA's cuDNN and integrates with OpenCV. Our convolutional nets run on distributed GPUs using Spark, making them among the fastest in the world. You can learn how to build a [image recognition web app with VGG16 here](./build_vgg_webapp) and how to [deploy CNNs to Android here](./android). From 88befdad09c7b6c08bd0220054febb9f975c21a9 Mon Sep 17 00:00:00 2001 From: maxpumperla Date: Thu, 5 Apr 2018 15:58:39 +0200 Subject: [PATCH 29/30] scalnet and nd4s --- releasenotes.md | 58 ++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 27 deletions(-) diff --git a/releasenotes.md b/releasenotes.md index 1e7bc219a..c47dbe80a 100644 --- a/releasenotes.md +++ b/releasenotes.md @@ -37,7 +37,7 @@ layout: default - Layers (new and enhanced) - - Added Yolo2OutputLayer CNN layer for object detection ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/layers/objdetect/Yolo2OutputLayer.java)). See also DataVec's [ObjectDetectionRecordReader](https://github.com/deeplearning4j/DataVec/blob/master/datavec-data/datavec-data-image/src/main/java/org/datavec/image/recordreader/objdetect/ObjectDetectionRecordReader.java) + - Added Yolo2OutputLayer CNN layer for object detection ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/layers/objdetect/Yolo2OutputLayer.java)). See also DataVec's [ObjectDetectionRecordReader](https://github.com/deeplearning4j/DataVec/blob/master/datavec-data/datavec-data-image/src/main/java/org/datavec/image/recordreader/objdetect/ObjectDetectionRecordReader.java) - Adds support for 'no bias' layers via ```hasBias(boolean)``` config (DenseLayer, EmbeddingLayer, OutputLayer, RnnOutputLayer, CenterLossOutputLayer, ConvolutionLayer, Convolution1DLayer). EmbeddingLayer now defaults to no bias ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/3882)) - Adds support for dilated convolutions (aka 'atrous' convolutions) - ConvolutionLayer, SubsamplingLayer, and 1D versions there-of. ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/3922)) - Added Upsampling2D layer, Upsampling1D layer ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/layers/Upsampling2D.java), [Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/layers/Upsampling1D.java)) @@ -59,7 +59,7 @@ layout: default - Added ISchedule interface; added Exponential, Inverse, Map, Poly, Sigmoid and Step schedule implementations ([Link](https://github.com/deeplearning4j/nd4j/tree/master/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/schedule)) - Added support for both iteration-based and epoch-based schedules via ISchedule. Also added support for custom (user defined) schedules - Learning rate schedules are configured on the updaters, via the ```.updater(IUpdater)``` method -- Added dropout API (IDropout - previously dropout was available but not a class); added Dropout, AlphaDropout (for use with self-normalizing NNs), GaussianDropout (multiplicative), GaussianNoise (additive). Added support for custom dropout types ([Link](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/dropout)) +- Added dropout API (IDropout - previously dropout was available but not a class); added Dropout, AlphaDropout (for use with self-normalizing NNs), GaussianDropout (multiplicative), GaussianNoise (additive). Added support for custom dropout types ([Link](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/dropout)) - Added support for dropout schedules via ISchedule interface ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/dropout/Dropout.java#L64)) - Added weight/parameter noise API (IWeightNoise interface); added DropConnect and WeightNoise (additive/multiplicative Gaussian noise) implementations ([Link](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/weightnoise)); dropconnect and dropout can now be used simultaneously - Adds layer configuration alias ```.units(int)``` equivalent to ```.nOut(int)``` ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/3900)) @@ -69,12 +69,12 @@ layout: default - MultiLayerNetwork, ComputationGraph and layerwise trainable layers now track the number of epochs ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/3957)) - Added deeplearning4j-ui-standalone module: uber-jar for easy launching of UI server (usage: ```java -jar deeplearning4j-ui-standalone-1.0.0-alpha.jar -p 9124 -r true -f c:/UIStorage.bin```) - Weight initializations: - - Added ```.weightInit(Distribution)``` convenience/overload (previously: required ```.weightInit(WeightInit.DISTRIBUTION).dist(Distribution)```) ([Link](https://github.com/deeplearning4j/deeplearning4j/commit/45cbb6efc2ad015397b4fdf5eac9d1e9dc70ac9c)) + - Added ```.weightInit(Distribution)``` convenience/overload (previously: required ```.weightInit(WeightInit.DISTRIBUTION).dist(Distribution)```) ([Link](https://github.com/deeplearning4j/deeplearning4j/commit/45cbb6efc2ad015397b4fdf5eac9d1e9dc70ac9c)) - WeightInit.NORMAL (for self-normalizing neural networks) ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/weights/WeightInit.java)) - Ones, Identity weight initialization ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/weights/WeightInit.java)) - Added new distributions (LogNormalDistribution, TruncatedNormalDistribution, OrthogonalDistribution, ConstantDistribution) which can be used for weight initialization ([Link](https://github.com/deeplearning4j/deeplearning4j/tree/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/distribution)) - RNNs: Added ability to specify weight initialization for recurrent weights separately to "input" weights ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4579)) -- Added layer alias: Convolution2D (ConvolutionLayer), Pooling1D (Subsampling1DLayer), Pooling2D (SubsamplingLayer) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4026)) +- Added layer alias: Convolution2D (ConvolutionLayer), Pooling1D (Subsampling1DLayer), Pooling2D (SubsamplingLayer) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4026)) - Added Spark IteratorUtils - wraps a RecordReaderMultiDataSetIterator for use in Spark network training ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-scaleout/spark/dl4j-spark/src/main/java/org/deeplearning4j/spark/datavec/iterator/IteratorUtils.java)) - CuDNN-supporting layers (ConvolutionLayer, etc) now warn the user if using CUDA without CuDNN ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4039)) - Binary cross entropy (LossBinaryXENT) now implements clipping (1e-5 to (1 - 1e-5) by default) to avoid numerical underflow/NaNs ([Link](https://github.com/deeplearning4j/nd4j/pull/2121)) @@ -99,7 +99,7 @@ layout: default - Added additional score functions for early stopping (ROC metrics, full set of Evaluation/Regression metrics, etc) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4630)) - Added additional ROC and ROCMultiClass evaluation overloads for MultiLayerNetwork and ComputationGraph ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4642)) - Clarified Evaluation.stats() output to refer to "Predictions" instead of "Examples" (former is more correct for RNNs) ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4674)) -- EarlyStoppingConfiguration now supports ```Supplier``` for use with non-serializable score calculators ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4694)) +- EarlyStoppingConfiguration now supports ```Supplier``` for use with non-serializable score calculators ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4694)) - Improved ModelSerializer exceptions when trying to load a model via wrong method (i.e., try to load ComputationGraph via restoreMultiLayerNetwork) ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4487)) - Added SparkDataValidation utility methods to validate saved DataSet and MultiDataSet on HDFS or local ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-scaleout/spark/dl4j-spark/src/main/java/org/deeplearning4j/spark/util/data/SparkDataValidation.java)) - ModelSerializer: added restoreMultiLayerNetworkAndNormalizer and restoreComputationGraphAndNormalizer methods ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4827)) @@ -130,12 +130,12 @@ layout: default - ModelSerializer temporary files would not be deleted if JVM crashes; now are deleted immediately when no longer required ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/3855)) - RecordReaderMultiDataSetIterator may not add mask arrays under some circumstances, when set to ALIGN_END mode ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4238)) - ConvolutionIterationListener previously produced an IndexOutOfBoundsException when all convolution layers are frozen ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4313)) -- PrecisionRecallCurve.getPointAtRecall could return a point with a correct but sub-optimal precision when multiple points had identical recall ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4327)) +- PrecisionRecallCurve.getPointAtRecall could return a point with a correct but sub-optimal precision when multiple points had identical recall ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4327)) - Setting dropout(0) on transfer learning FineTuneConfiguration did not remove dropout if present on existing layer ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4368)) - Under some rare circumstances, Spark evaluation could lead to a NullPointerException ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/3970)) - ComputationGraph: disconnected vertices were not always detected in configuration validation ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/2714)) - Activation layers would not always inherit the global activation function configuration ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4094)) -- RNN evaluation memory optimization: when TBPTT is configured for training, also use TBPTT-style splitting for evaluation (identical result, less memory) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4405), [Link](https://github.com/deeplearning4j/deeplearning4j/issues/3482)) +- RNN evaluation memory optimization: when TBPTT is configured for training, also use TBPTT-style splitting for evaluation (identical result, less memory) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4405), [Link](https://github.com/deeplearning4j/deeplearning4j/issues/3482)) - PerformanceListener is now serializable ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4423)) - ScoreIterationListener and PerformanceListener now report model iteration, not "iterations since listener creation" ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4444)) - Precision/recall curves cached values in ROC class may not be updated after merging ROC instances ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4442)) @@ -148,21 +148,21 @@ layout: default - MnistDataSetIterator / EmnistDataSetIterator: updated download location after hosting URL change ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4632), [Link](https://github.com/deeplearning4j/deeplearning4j/pull/4637)) - Fixes to propagation of thread interruptions ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4644)) - MultiLayerNetwork/ComputationGraph will no longer throw an ND4JIllegalStateException during initialization if a network contains no parameters ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4635), [Link](https://github.com/deeplearning4j/deeplearning4j/pull/4664)) -- Fixes for TSNE posting of data to UI for visualization ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4667)) +- Fixes for TSNE posting of data to UI for visualization ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4667)) - PerformanceListener now throws a useful exception (in constructor) on invalid frequency argument, instead of runtime ArithmeticException ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4679)) - RecordReader(Multi)DataSetIterator now throws more useful exceptions when Writable values are non-numerical ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4484)) - UI: Fixed possible character encoding issues for non-English languages when internationalization data .txt files are read from uber JARs ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4512)) - UI: Fixed UI incorrectly trying to parse non-DL4J UI resources when loading I18N data ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4497)) - Various threading fixes ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4794)) - Evaluation: no-arg methods (f1(), precion(), etc) now return single class value for binary case instead of macro-averaged value; clarify values in stats() method and javadoc ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4802)) -- Early stopping training: TrainingListener opEpochStart/End (etc) methods were not being called correctly ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4798)) +- Early stopping training: TrainingListener opEpochStart/End (etc) methods were not being called correctly ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4798)) - Fixes issue where dropout was not always applied to input of RNN layers ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4823)) - ModelSerializer: improved validation/exceptions when reading from invalid/empty/closed streams ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4827)) - ParallelInference fixes: - fixes for variable size inputs (variable length time series, variable size CNN inputs) when using batch mode ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4836)) - fixes undelying model exceptions during output method are now properly propagated back to the user ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4836)) - - fixes support for 'pre-batched' inputs (i.e., inputs where minibatch size is > 1) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4836)) -- Memory optimization for network weight initialization via in-place random ops ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4837)) + - fixes support for 'pre-batched' inputs (i.e., inputs where minibatch size is > 1) ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4836)) +- Memory optimization for network weight initialization via in-place random ops ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4837)) - Fixes for CuDNN with SAME mode padding ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4864), [Link](https://github.com/deeplearning4j/deeplearning4j/pull/4871)) - Fix for VariationalAutoencoder builder decoder layer size validation ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4874)) @@ -187,7 +187,7 @@ layout: default - RBM (Restricted Boltzmann Machine) layers have been removed entirely. Consider using VariationalAutoencoder layers as a replacement ([Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/layers/variational/VariationalAutoencoder.java)) - GravesBidirectionalLSTM has been deprecated; use ```new Bidirectional(Bidirectional.Mode.ADD, new GravesLSTM.Builder()....build()))``` instead - Previously deprecated WordVectorSerializer methods have now been removed ([Link](https://github.com/deeplearning4j/deeplearning4j/issues/4359)) -- Removed deeplearning4j-ui-remote-iterationlisteners module and obsolete RemoteConvolutionalIterationListener ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4772)) +- Removed deeplearning4j-ui-remote-iterationlisteners module and obsolete RemoteConvolutionalIterationListener ([Link](https://github.com/deeplearning4j/deeplearning4j/pull/4772)) ## Deeplearing4J: Keras Import @@ -216,9 +216,10 @@ layout: default - SpaceToBatch and BatchToSpace layers in DL4J for better YOLO support, plus end-to-end YOLO Keras import test. - Cropping2D support in DL4J and Keras model import + ### Deeplearning4J: Keras Import - API Changes (Transition Guide): 0.9.1 to 1.0.0-alpha -[Not sure if required - but if so, cover API changes etc - and what to use now - go here] +- In 0.9.1 deprecated `Model` and `ModelConfiguration` have been permanently removed. Use [KerasModelImport](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-modelimport/src/main/java/org/deeplearning4j/nn/modelimport/keras/KerasModelImport.java) instead, which is now the only entry point for Keras model import. ### Deeplearning4J: Keras Import - Known Issues @@ -238,7 +239,7 @@ layout: default ### ND4J: Known Issues -### ND4J: API Changes (Transition Guide): 0.9.1 to 1.0.0-alpha +### ND4J: API Changes (Transition Guide): 0.9.1 to 1.0.0-alpha ## ND4J - SameDiff @@ -340,17 +341,20 @@ layout: default - As per DL4J updater API changes: old updater configuration (learningRate, momentum, etc) methods have been removed. Use ```.updater(IUpdater)``` or ```.updater(ParameterSpace)``` methods instead - - ## RL4J +## ScalNet -## ScalNet +- First release of [ScalNet Scala API](https://github.com/deeplearning4j/scalnet), which closely resembles Keras' API. +- Can be built with sbt and maven. +- Supports both Keras inspired [Sequential](https://github.com/deeplearning4j/ScalNet/blob/master/src/main/scala/org/deeplearning4j/scalnet/models/Sequential.scala) models, corresponding to DL4J's `MultiLayerNetwork`, and [Model](https://github.com/deeplearning4j/ScalNet/blob/master/src/main/scala/org/deeplearning4j/scalnet/models/Model.scala), corresponding to `ComputationGraph`. +- Project structure is closely aligned to both DL4J model-import module and Keras. +- Supports the following layers: Convolution2D, Dense, EmbeddingLayer, AvgPooling2D, MaxPooling2D, GravesLSTM, LSTM, Bidirectional layer wrapper, Flatten, Reshape. Additionally, DL4J OutputLayers are supported. ## ND4S - +- Scala 2.12 support # Release Notes for Version 0.9.1 @@ -466,7 +470,7 @@ layout: default - Added CnnSentenceDataSetIterator (for use with ‘CNN for Sentence Classification’ architecture) [Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nlp-parent/deeplearning4j-nlp/src/main/java/org/deeplearning4j/iterator/CnnSentenceDataSetIterator.java) [Link2](https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/sentenceClassification/CnnSentenceClassificationExample.java) - Spark + Kryo: now test serialization + throw exception if misconfigured (instead of logging an error that can be missed) - MultiLayerNetwork now adds default layer names if no name is specified -- DataVec: +- DataVec: - JSON/YAML support for DataAnalysis, custom Transforms etc - ImageRecordReader refactored to reduce garbage collection load (hence improve performance with large training sets) - Faster quality analysis. @@ -522,8 +526,8 @@ layout: default # Release Notes for Version 0.7.1 -* RBM and AutoEncoder key fixes: - - Ensured visual bias updated and applied during pretraining. +* RBM and AutoEncoder key fixes: + - Ensured visual bias updated and applied during pretraining. - RBM HiddenUnit is the activation function for this layer; thus, established derivative calculations for backprop according to respective HiddenUnit. * RNG performance issues fixed for CUDA backend * OpenBLAS issues fixed for macOS, powerpc, linux. @@ -569,14 +573,14 @@ Notable changes for upgrading codebases based on 0.6.0 to 0.7.0: - See ConvolutionMode javadoc for more details: [Link](https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/ConvolutionMode.java) * Xavier weight initialization change for CNNs and LSTMs: Xavier now aligns better with original Glorot paper and other libraries. Xavier weight init. equivalent to 0.6.0 is available as XAVIER_LEGACY * DataVec: Custom RecordReader and SequenceRecordReader classes require additional methods, for the new metadata functionality. Refer to existing record reader implementations for how to implement these methods. -* Word2Vec/ParagraphVectors: +* Word2Vec/ParagraphVectors: - Few new builder methods: - allowParallelTokenization(boolean) - useHierarchicSoftmax(boolean) - Behaviour change: batchSize: now batch size is ALSO used as threshold to execute number of computational batches for sg/cbow -# Release Notes for Version 0.6.0 +# Release Notes for Version 0.6.0 * Custom layer support * Support for custom loss functions @@ -595,7 +599,7 @@ Notable changes for upgrading codebases based on 0.6.0 to 0.7.0: * Performance improvements to Spark training implementation * Improved network configuration validation using InputType functionality -# Release Notes for Version 0.5.0 +# Release Notes for Version 0.5.0 * FP16 support for CUDA * [Better performance for multi-gpu}(http://deeplearning4j.org/gpu) @@ -606,11 +610,11 @@ Notable changes for upgrading codebases based on 0.6.0 to 0.7.0: * Numerous bug fixes * Spark improvements -## Release Notes for version 0.4.0 +## Release Notes for version 0.4.0 -* Initial multi-GPU support viable for standalone and Spark. +* Initial multi-GPU support viable for standalone and Spark. * Refactored the Spark API significantly -* Added CuDNN wrapper +* Added CuDNN wrapper * Performance improvements for ND4J * Introducing [DataVec](https://github.com/deeplearning4j/datavec): Lots of new functionality for transforming, preprocessing, cleaning data. (This replaces Canova) * New DataSetIterators for feeding neural nets with existing data: ExistingDataSetIterator, Floats(Double)DataSetIterator, IteratorDataSetIterator From bbae3fb098f367605863ee417e3ced0f2489b7a1 Mon Sep 17 00:00:00 2001 From: maxpumperla Date: Thu, 5 Apr 2018 16:25:47 +0200 Subject: [PATCH 30/30] SameDiff --- releasenotes.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/releasenotes.md b/releasenotes.md index c47dbe80a..4c40d7880 100644 --- a/releasenotes.md +++ b/releasenotes.md @@ -244,10 +244,19 @@ layout: default ## ND4J - SameDiff +Alpha release of [SameDiff](https://github.com/deeplearning4j/nd4j/tree/master/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff) auto-differentiation engine for ND4J. + ### Features +- Building and running computation graphs build from SameDiff operations. +- Graphs can run forward pass on input data and compute gradients for the backward pass. +- Already supports many high-level layers, like dense layers, convolutions (1D-3D) deconvolutions, separable convolutions, pooling and upsampling, batch normalization, local response normalization, LSTMs and GRUs. +- In total there are about 350 SameDiff operations available, including many basic operations used in building complex graphs. +- Supports rudimentary import of [TensorFlow](https://github.com/deeplearning4j/nd4j/tree/d4a15e394ef81592237677aee932eb734d64f5a7/nd4j-backends/nd4j-tests/src/test/java/org/nd4j/imports) and ONNX graphs for inference. +- [TFOpTests](https://github.com/deeplearning4j/TFOpTests) is a dedicated project for creating test resources for TensorFlow import. ### Known Issues and Limitations - +- While many of the widely used base operations and high-level layers used in practice are supported, op coverage is still limited. Goal is to achieve feature parity with TensorFlow and fully support import for TF graphs. +- Some of the existing ops do not have a backward pass implemented (called `doDiff` in SameDiff).