Deep Learning is a subdomain of machine learning, concerned with the algorithm which imitates the function and structure of the brain called the artificial neural network.
Linear algebra, probability and calculus are the 'languages' in which machine learning is formulated. Learning these topics will contribute a deeper understanding of the underlying algorithmic mechanics and allow development of new algorithms.
When confined to smaller levels, everything is math behind deep learning. So it is important to understand basic linear algebra before getting started with deep learning and programming it.
The core data structures behind Deep-Learning are Scalars, Vectors, Matrices and Tensors. Programmatically, let's solve all the basic linear algebra problems using these.
Scalars are single numbers and are an example of a 0th-order tensor. The notation x ∈ ℝ states that x is a scalar belonging to a set of Real-values numbers, ℝ. There are different sets of numbers of interest in deep learning. ℕ represents the set of positive integers (1,2,3,…). ℤ designates the integers, which combine positive, negative and zero values. ℚ represents the set of rational numbers that may be expressed as a fraction of two integers.
Few built-in scalar types are int, float, complex, bytes, Unicode. In NumPy, there are 24 new fundamental Python types to describe different types of scalars. When it comes to a graphical representation, it will be a simple point on the graph More Info.
Defining Scalars and Few Operations in Python
# In-Built Scalars
a = 5
b = 7.5
print(type(a))
print(type(b))
print(a + b)
print(a - b)
print(a * b)
print(a / b)# Is Scalar Function
def isscalar(num):
if isinstance(num, generic):
return True
else:
return False>>> import numpy as mp
>>> np.isscalar(3.1)
True
>>> np.isscalar([3.1])
False
>>> np.isscalar(False)
TrueVectors are ordered arrays of single numbers and are an example of 1st-order tensor. Vectors are fragments of objects known as vector spaces. A vector space can be considered of as the entire collection of all possible vectors of a particular length (or dimension). The three-dimensional real-valued vector space, denoted by ℝ3 is often used to represent our real-world notion of three-dimensional space mathematically.
x = [x1 x2 x3 x4 ... xn]
To identify the necessary component of a vector explicitly, the ith scalar element of a vector is written as xi.
In deep learning vectors usually represent feature vectors, with their original components defining how relevant a particular feature is. Such elements could include the related importance of the intensity of a set of pixels in a two-dimensional image or historical price values for a cross-section of financial instruments.
Defining Vectors and Few Operations in Python
import numpy as np
# Declaring Vectors
x = [1, 2, 3]
y = [4, 5, 6]
print(type(x))
# This does'nt give the vector addition.
print(x + y)
# Vector addition using Numpy
z = np.add(x, y)
print(z)
print(type(z))
# Vector Cross Product
mul = np.cross(x, y)
print(mul)<class 'list'>
[1, 2, 3, 4, 5, 6]
[5 7 9]
<class 'numpy.ndarray'>
[-3 6 -3]
Matrices are rectangular arrays consisting of numbers and are an example of 2nd-order tensors. If m and n are positive integers, that is m,n ∈ ℕ then the m×n matrix contains mn numbers, with m rows and n columns.
The full m×n matrix can be written as:
It is often useful to abbreviate the full matrix component display into the following expression:
A=[aij]m×n
Defining Matrix and Few Operations in Python
>>> import numpy as np
>>> x = np.matrix([[1,2],[2,3]])
>>> x
matrix([[1, 2],
[2, 3]])
>>> a = x.mean(0)
>>> a
matrix([[1.5, 2.5]])
>>> # Finding the mean with 1 with the matrix x.
>>> z = x.mean(1)
>>> z
matrix([[ 1.5],
[ 2.5]])
>>> z.shape
(2, 1)
>>> y = x - z
matrix([[-0.5, 0.5],
[-0.5, 0.5]])
>>> print(type(z))
<class 'numpy.matrixlib.defmatrix.matrix'>Matrices can be added to scalars, vectors and other matrices. Each of these operations has a precise definition. These techniques are used frequently in machine learning and deep learning so it is worth familiarising yourself with them.
# Matrix Addition
import numpy as np
x = np.matrix([[1, 2], [4, 3]])
sum = x.sum()
print(sum)
# Output: 10C = A + B (shape of A and B should be equal)
# Matrix-Matrix Addition
import numpy as np
x = np.matrix([[1, 2], [4, 3]])
y = np.matrix([[3, 4], [3, 10]])
print(x.shape)
# (2, 2)
print(y.shape)
# (2, 2)
m_sum = np.add(x, y)
print(m_sum)
print(m_sum.shape)
"""
Output :
[[ 4 6]
[ 7 13]]
(2, 2)
"""Adds the given scalar to all the elements in the given matrix.
# Matrix-Scalar Addition
import numpy as np
x = np.matrix([[1, 2], [4, 3]])
s_sum = x + 1
print(s_sum)
"""
Output:
[[2 3]
[5 4]]
"""A of shape (m x n) and B of shape (n x p) multiplied gives C of shape (m x p)
# Matrix Multiplication
import numpy as np
a = [[1, 0], [0, 1]]
b = [1, 2]
np.matmul(a, b)
# Output: array([1, 2])
complex_mul = np.matmul([2j, 3j], [2j, 3j])
print(complex_mul)
# Output: (-13+0j)Multiplies the given scalar to all the elements in the given matrix.
# Matrix Scalar Multiplication
import numpy as np
x = np.matrix([[1, 2], [4, 3]])
s_mul = x * 3
print(s_mul)
"""
[[ 3 6]
[12 9]]
"""A=[aij]mxn
AT=[aji]n×m
# Matrix Transpose
import numpy as np
a = np.array([[1, 2], [3, 4]])
print(a)
"""
[[1 2]
[3 4]]
"""
a.transpose()
print(a)
"""
array([[1, 3],
[2, 4]])
"""The more general entity of a tensor encapsulates the scalar, vector and the matrix. It is sometimes necessary—both in the physical sciences and machine learning—to make use of tensors with order that exceeds two.
We use Python libraries like tensorflow or PyTorch in order to declare tensors, rather than nesting matrices.
To define a simple tensor in PyTorch
import torch
a = torch.Tensor([26])
print(type(a))
# <class 'torch.FloatTensor'>
print(a.shape)
# torch.Size([1])
# Creates a Random Torch Variable of size 5x3.
t = torch.Tensor(5, 3)
print(t)
"""
0.0000e+00 0.0000e+00 0.0000e+00
0.0000e+00 7.0065e-45 1.1614e-41
0.0000e+00 2.2369e+08 0.0000e+00
0.0000e+00 0.0000e+00 0.0000e+00
nan nan -1.4469e+35
[torch.FloatTensor of size 5x3]
"""
print(t.shape)
# torch.Size([5, 3])Few Operations on Tensors in Python
import torch
# Creating Tensors
p = torch.Tensor(4,4)
q = torch.Tensor(4,4)
ones = torch.ones(4,4)
print(p, q, ones)
"""
Output:
0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
1.6009e-19 4.4721e+21 6.2625e+22 4.7428e+30
3.1921e-09 8.0221e+17 5.1019e-08 8.1121e+17
8.1631e-07 8.2022e+17 1.1703e-19 1.5637e-01
[torch.FloatTensor of size 4x4]
0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
1.8217e-44 1.1614e-41 0.0000e+00 2.2369e+08
0.0000e+00 0.0000e+00 2.0376e-40 2.0376e-40
nan nan -5.3105e+37 nan
[torch.FloatTensor of size 4x4]
1 1 1 1
1 1 1 1
1 1 1 1
1 1 1 1
[torch.FloatTensor of size 4x4]
"""
print("Addition:{}".format(p + q))
print("Subtraction:{}".format(p - ones))
print("Multiplication:{}".format(p * ones))
print("Division:{}".format(q / ones))
"""
Addition:
0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
1.6009e-19 4.4721e+21 6.2625e+22 4.7428e+30
3.1921e-09 8.0221e+17 5.1019e-08 8.1121e+17
nan nan -5.3105e+37 nan
[torch.FloatTensor of size 4x4]
Subtraction:
-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00
-1.0000e+00 4.4721e+21 6.2625e+22 4.7428e+30
-1.0000e+00 8.0221e+17 -1.0000e+00 8.1121e+17
-1.0000e+00 8.2022e+17 -1.0000e+00 -8.4363e-01
[torch.FloatTensor of size 4x4]
Multiplication:
0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
1.6009e-19 4.4721e+21 6.2625e+22 4.7428e+30
3.1921e-09 8.0221e+17 5.1019e-08 8.1121e+17
8.1631e-07 8.2022e+17 1.1703e-19 1.5637e-01
[torch.FloatTensor of size 4x4]
Division:
0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
1.8217e-44 1.1614e-41 0.0000e+00 2.2369e+08
0.0000e+00 0.0000e+00 2.0376e-40 2.0376e-40
nan nan -5.3105e+37 nan
[torch.FloatTensor of size 4x4]
"""