Lesson 1: Introduction to Machine Learning

• Using Sklearn for Iris dataset.
• Binary classification, multiclass classification.

Lesson 2: Linear classifier and stochastic gradient

• Stochastic gradient in practice.
• Maximum Likelihood and Regularization L1,L2.
• Find optimize regularization using LOO.

Lesson 3: Neural Networks: Gradient Optimization Techniques

• Autograd.
• MLP for MNIST.
• Tuning hyperparameters for MLP.

Lesson 4: Metric classification and regression methods

• kNN, kernel-kNN, Parzen window method, potential function method.
• Reference element selection, STOLP, Nadarai Watson formula.

Lesson 5: Support vector machine

• SVM , kernel-SVM for classification, regression.
• SVM feature.

Lesson 6: Multidimensional Linear Regression

• Multidimendional Linear Regression, SVD, regularization for MLR using SVD.
• Dependence of the approximation quality on the condition number.
• PCA on MNIST.
• PCA for images.

Lesson 7: Nonlinear Regression

• Non-linear regression example.
• Compare gradient descent, Newton-Raphson and Newton-Gauss.
• Generalized linear models: optimal sample size.
• Loss function for the problem of finding close sentences.
• Convergence visualization of the Newton Raphson method and the stochastic gradient.

Lesson 8: Model Selection Criteria and Feature Selection Methods

• Model quality assessment: external and internal criteria.
• Feature selection: exhaustive search, Add algorithm, Add-Del algorithm.
• Precision,Recall.
• Example of information retrieval task.

Lesson 9: Logical classification methods

• Logical classifier implementation.
• Informative criteria.
• Decision list, simple implementation.
• Decision tree.
• Random forest.

Lesson 10: Search for association rules

• Statement of the problem of association rules.
• Synthetic example.
• Example of real data from Kaggle.
• Apriori algorithm.
• FP-growth algorithm.
• Generalization for real data.
• Generalized association rules.

Lesson 11: Linear Ensembles

• DummyEnsemble.
• AdaBoost.
• Gradient boosting, XGBoost.
• An example of real data from kaggle.
• RandomForest.
• Mixture Of Expert.

Lesson 12: Advanced Ensembling Techniques

• ComBoost.
• Gradient Boosting.
• XGBoost.
• CatBoost.

Lesson 13: Bayesian theory of classification

• Maximum Likelihood Principle: Visualization.
• Density reconstruction from empirical data.
• Using LOO to select the window width.
• Naive Bayes classifier.

Lesson 14: Clustering and semi-supervised learning

• Clustering examples.
• K-means.
• DBSCAN.
• Hierarchical clustering.
• Semi-supervised learning.
• Self-training, 1970.
• Unlabeled data in deep learning.

Lesson 15: Deep Neural Networks

• CNN, RNN, Tensorboard, Transfer Learning, Interpretability of NN.

Lesson 16: AutoEncoder,GAN

• Autoencoder, Linear Autoencoder, Autoencoder using CNN, Variational autoencoder.
• Transfer learning from a pre-trained model.
• Generative adversarial networks.

Lesson 17: Tokenization,Word2Vec(Fasttext)

• An example of classifying tweets.
• Text tokenization.
• Word2Vec (based on the FastText model).
• FastText model (compressed to emb-dim=10 for lightness).
• Problems for unsupervised learning of vectorization models.

Lesson 18: Attention.Transformer

• Attention model RNN.
• Transformer.
• T2T translator.
• BPE tokenization.
• BERT.
• LaBSE.

Lesson 19: Modeling

• LDA.
• PLSA(bigartm).

Lesson 20: Homework

Lesson 21: Learning to rank

• Basic concept.
• An example of a ranking problem.
• An example of a recommender system.
• Training a search engine based on pyserini.

Lesson 22: Recommender Systems

• Constant model.
• Correlation system.
• SLIM.
• SVD.

Lesson 23: Time Series Analysis

• Autoregression model.
• Exponential smoothing.
• Cluster analysis of time series.

Lesson 24: Online Learning

Lesson 25: Reinforcement Learning

• Stationary multi-armed bandit.
• Non-stationary multi-armed bandit.
• Swim problem.

Lesson 26: Active Learning

• Active learning with a random additive element.
• Active learning with the addition of the element with the maximum variance.