In a very simplistic way, in shallow learning algorithms, input features are not "changed", so the algorithm try to search different parameters and/or way to separate the observation in the dataset so that the resulting model can separate and predict new observations by using the parameters and patterns learned whil training time. In other words, the number of parameters is mostly bounded by the number of feautures (input dimension space).
- Collobrative filtering
- decision tree and random forrest
- K-MEAN
- KNN
- Linear regression
- Logistic regression
- Naive Bayes
- PCA
- Polynomial regression
- SVM
Comparing deep learning to shallow learning definition given above, again in a very simplistic term, the input features are now changing in many ways during training, that means the algorithm keep trying to change the input features to have a better representation of the input data, and this process of learning new useful features is repeated as many time as we want. The repetition of this process is what lead us to depp learning because the algorithm is "always" trying to learn new features based on the precedent created features. For every new set of features, new parameters are created and learned through traininig, deep learning is "just" a fancy way of saying the algorithm has lot of parameters.