Skip to content
GregLinthicum edited this page Nov 6, 2020 · 98 revisions

Introduction

The technology emerges from the state where the machine did not even get the words you were uttering and responded in a choppy robotic way to the open space of the creativity unimaginable ever before.

The time has arrived to venture into improving the attention span of Amazon's Alexa, Google's Assistant, and Apple's Siri. Alexa services exploded into over seventy thousand of skills. Companies and governments use speech recognition and speech synthesis in telephony, in the browser or in in the field applications.

Now we're talking.

The text you are reading on this Wiki page right now is just a graphical representation of the soundtrack. Let's cut out this impediment. Typing skills just like a calligraphy a century ago would become obliviated. Not so quick. Writing provides a spectrum of services perhaps underestimated before. First, the written text allows us to synchronize on pronunciation - or communicating despite different pronunciations ( ask Chinese). Second, the written text provides an access to the treasury of historical records. At this time the written text is still a better form of data representation for conducting searches and for more advanced forms of information processing than soundtracks allow to achieve.

Taxonomy

Supervised Learning

  • Artificial Neural networks ( ANN )
  • Multilayer Perceptrons (MLPs)
  • Convolutional Neural Networks (CNNs)
  • Recurrent Neural Networks (RNNs)
  • Long short-term memory (LSTM)

Unsupervised Learning

  • Self-Organizing Maps (SOM)
  • Restricted Boltzman Machines (RBM)
  • Deep Boltzmann Machines (DBM)
  • Deep Believe Networks (DBN)
  • Auto-Encoders (AE)
  • Sparse Auto-Encoder (SAE)
  • Stacked Auto-Encoder (SAE)
  • Stacked Sparse Auto-Encoder (SSAE)
  • Variational Autoencoder (VAE)
  • Generative Pre-trained Transformer (GPT)

Application

Supervised Learning ( Discriminative )

Typical discriminative models include logistic regression (LR), support vector machines (SVM), conditional random fields (CRFs) (specified over an undirected graph), decision trees, neural networks.

Artificial Neural networks ( ANN )

Used mainly for Regression and Classification "Many studies have shown that ANNs can better model heteroskedasticity i.e. data with high volatility and non-constant variance, given its ability to learn hidden relationships in the data without imposing any fixed relationships in the data. This is something very useful in financial time series forecasting (e.g. stock prices) where data volatility is very high." [TDS]

Multilayer Perceptrons (MLPs)

Use Hiddenlayer to render a graph of your neural network in Jupyter Notebook, or to a pdf or png file. See Jupyter notebook examples for TensorFlow, PyTorch, and Keras.

Convolutional Neural Networks (CNNs)

Used Mainly for Machine Vision

Recurrent Neural Networks (RNNs)

Used mainly for Time Series Analysis

Long short-term memory (LSTM)

Unsupervised Learning ( Generative )

Typical generative model approaches include naive Bayes classifiers, Gaussian mixture models, variational autoencoders.

Self-Organizing Maps (SOM)

Used mainly for Feature Detection.

Restricted Boltzman Machines (RBM)

Used mainly for dimensionality reduction, classification, regression, collaborative filtering, feature learning, and topic modelling.

Deep Boltzmann Machines (DBM) Used Mainly for Recommendation Systems.

Deep Believe Networks (DBN)

Difference between Deep Belief and Deep Boltzman is that Deep Boltzman is bidirectional on every level.

Auto-Encoders (AE)

Used Mainly for Recommendation Systems. sparse-stacked-and-variational-autoencoder

Transformers

NLP The Transformer is a deep learning model introduced in 2017, used primarily in the field of natural language processing (NLP). Like recurrent neural networks (RNNs), Transformers are designed to handle sequential data, such as natural language, for tasks such as translation and text summarization. However, unlike RNNs, Transformers do not require that the sequential data be processed in the order.

  • Attention Is All You Need

  • YouTube Transformer Neural Networks - EXPLAINED

  • TDS

  • Slavic Languages Please note that high volume of attention is given to analysis dependency of semantics and sequence. This perspective is amplified by the fact that most analysis is done on English. In English the semantics is embedded in the sequence of words. It is not so in Slavic languages. In many instances of short sentences in Polish you can say words in any order and the meaning remains unchanged. This is so because words themselves are morphing in order to indicate their relation to other words in the sentence.

  • LSTM is dead, long live Transformers! Boudajenek


Benefits of Noise

Clone this wiki locally