{{ message }}

# roboticcam / machine-learning-notes Public

My continuously updated Machine Learning, Probabilistic Models and Deep Learning notes and demos (2000+ slides) 我不间断更新的机器学习，概率模型和深度学习的讲义(2000+页)和视频链接

Switch branches/tags
Nothing to show

## Files

Failed to load latest commit information.
Type
Name
Commit time
Sep 13, 2021
Sep 13, 2021

# Infinity in Deep Learning 深度学习“无限”精彩

• ### Neural Networks Gaussian Process and Neural Tangent Kernel Initialization

The detailed derivation of (1) Neural Network Gaussian process using central limit theorem (2) Neural Tangent Kernel (NTK) for initialization. I also tried to give people a brief introduction to what is Gaussian process and kernel method to make this tutorial more complete. 详细推导（1）使用中心极限定理的神经网络高斯过程（2）神经正切核（NTK）进行初始化. 我还尝试向大家简要介绍什么是高斯过程和内核方法，以使本教程更加完整。

• ### Infinite Depth: NeuralODE and Adjoint Equation

Discuss Neural ODE and in particular the use of adjoint equation in Parameter training 讨论神经ODE，尤其是在参数训练中使用伴随方程

# Sinovasinovation DeeCamp 创新工场DeeCAMP讲义

• ### DeeCamp 2019：Story of Softmax

properties of Softmax, Estimating softmax without compute denominator, Probability re-parameterization: Gumbel-Max trick and REBAR algorithm (softmax的故事) Softmax的属性, 估计softmax时不需计算分母, 概率重新参数化, Gumbel-Max技巧和REBAR算法

• ### DeeCamp 2018：When Probabilities meet Neural Networks

Expectation-Maximization & Matrix Capsule Networks; Determinantal Point Process & Neural Networks compression; Kalman Filter & LSTM; Model estimation & Binary classifier (当概率遇到神经网络) 主题包括：EM算法和矩阵胶囊网络; 行列式点过程和神经网络压缩; 卡尔曼滤波器和LSTM; 模型估计和二分类问题关系

# Video Tutorial to these notes 视频资料

• I recorded about 20% of these notes in videos in 2015 in Mandarin (all my notes and writings are in English) You may find them on Youtube and bilibili and Youku

# 3D Geometry Computer vision 3D几何计算机视觉

• ### 3D Geometry Fundamentals

Camera Models, Intrinsic and Extrinsic parameter estimation, Epipolar Geometry, 3D reconstruction, Depth Estimation 相机模型，内部和外部参数估计，对极几何，三维重建，图像深度估计

• ### Recent Deep 3D Geometry based Research

Recent research of the following topics: Single image to Camera Model estimation, Multi-Person 3D pose estimation from multi-view, GAN-based 3D pose estimation, Deep Structure-from-Motion, Deep Learning based Depth Estimation, 以下主题的最新研究：单图像到相机模型的估计，基于多视图的多人3D姿势估计，基于GAN的3D姿势估计，基于运动的深度结构，基于深度学习的深度估计

This section is co-authored with PhD student Yang Li 本部分与博士研究生李杨合写

# Deep Learning Research Topics 深度学习研究

• ### Variance Reduction

REBAR, RELAX algorithm and some detailed explanation of re-parameterization of Gumbel conditionals REBAR，RELAX算法以及对Gumbel条件概率重新参数化的一些详细说明

• ### New Research on Softmax function

Out-of-distribution, Neural Network Calibration, Gumbel-Max trick, Stochastic Beams Search (some of these lectures overlap with DeeCamp2019) 分布外、神经网络校准、Gumbel-Max 技巧、随机光束(BEAM)搜索（其中一些讲座与 DeeCamp2019 重叠）

• ### Mathematics for Generative Adversarial Networks

How GAN works, Traditional GAN, Mathematics on W-GAN, Info-GAN, Bayesian GAN GAN如何工作，传统GAN，W-GAN数学，Info-GAN，贝叶斯GAN

How Varational Autoencoder works, Importance Weighted Autoencoders, Normalized Flow via ELBO, Adversarial Variational Bayes, 变分自编码器的工作原理，重要性加权自编码器，通过ELBO的标准化流，对抗变分贝叶斯

• ### Bayesian Inference and Deep Learning (Seminar Talk)

This is a seminar talk I gave on some modern examples in which Bayesian (or probabilistic) framework is to explain, assist and assisted by Deep Learning. 这是我的演讲稿件。归纳了一些最近研究例子中，贝叶斯（或概率）框架来解释，帮助(或被帮助于)深度学习。

# Deep Learning Basics 深度学习基础

• ### Neural Networks

basic neural networks and multilayer perceptron 神经网络: 基本神经网络和多层感知器

• ### Convolution Neural Networks: from basic to recent Research

detailed explanation of CNN, various Loss function, Centre Loss, contrastive Loss, Residual Networks, Capsule Networks, YOLO, SSD 卷积神经网络：从基础到最近的研究：包括卷积神经网络的详细解释，各种损失函数，中心损失函数，对比损失函数，残差网络，胶囊网络, YOLO，SSD

• ### Restricted Boltzmann Machine

basic knowledge in Restricted Boltzmann Machine (RBM) 受限玻尔兹曼机(RBM)中的基础知识

# Reinforcement Learning 强化学习

• ### Reinforcement Learning Basics

basic knowledge in reinforcement learning, Markov Decision Process, Bellman Equation and move onto Deep Q-Learning 深度增强学习: 强化学习的基础知识，马尔可夫决策过程，贝尔曼方程，深度Q学习

• ### Monto Carlo Tree Search

Monto Carlo Tree Search, alphaGo learning algorithm 蒙托卡罗树搜索，alphaGo学习算法

Policy Gradient Theorem, Mathematics on Trusted Region Optimization in RL, Natural Gradients on TRPO, Proximal Policy Optimization (PPO), Conjugate Gradient Algorithm 政策梯度定理, RL中可信区域优化的数学,TRPO自然梯度, 近似策略优化(PPO), 共轭梯度算法

# Optimization Method 优化方法

• ### Quick note on Lagrangian Dual

A quick explanation of Lagrangian duality, KKT condition, support vector machines 关于拉格朗日对偶,对偶性和KKT条件,支持向量机的简单说明

A quick explanation of Conjugate Gradient Descend 共轭梯度下降的快速解释

# Natural Language Processing 自然语言处理

• ### Word Embeddings

Word2Vec, skip-gram, GloVe, Fasttext 系统的介绍了自然语言处理中的“词表示”中的技巧

• ### Deep Natural Language Processing

RNN, LSTM, Seq2Seq with Attenion, Beam search, Attention is all you need, Convolution Seq2Seq, Pointer Networks 深度自然语言处理：递归神经网络,LSTM,具有注意力机制的Seq2Seq，集束搜索，指针网络和 "Attention is all you need", 卷积Seq2Seq

# Data Science 数据科学课件

• ### 30 minutes introduction to AI and Machine Learning

An extremely gentle 30 minutes introduction to AI and Machine Learning. Thanks to my PhD student Haodong Chang for assist editing 30分钟介绍人工智能和机器学习, 感谢我的学生常浩东进行协助编辑

• ### Regression methods

Classification: Logistic and Softmax; Regression: Linear, polynomial; Mix Effect model [costFunction.m] and [soft_max.m]

• ### Recommendation system

collaborative filtering, Factorization Machines, Non-Negative Matrix factorisation, Multiplicative Update Rule 推荐系统: 协同过滤，分解机，非负矩阵分解，和期中“乘法更新规则”的介绍

• ### Dimension Reduction

classic PCA and t-SNE 经典的PCA降维法和t-SNE降维法

• ### Introduction to Data Analytics and associate Jupyter notebook

Supervised vs Unsupervised Learning, Classification accuracy 数据分析简介和相关的jupyter notebook，包括监督与无监督学习，分类准确性

# Probability and Statistics Background 概率论与数理统计基础课件

• ### Bayesian model

revision on Bayes model include Bayesian predictive model, conditional expectation 复习贝叶斯模型，包括贝叶斯预测模型，条件期望等基础知识

• ### Probabilistic Estimation

some useful distributions, conjugacy, MLE, MAP, Exponential family and natural parameters 一些常用的分布，共轭特性，最大似然估计, 最大后验估计, 指数族和自然参数

• ### Statistics Properties

useful statistical properties to help us prove things, include Chebyshev and Markov inequality 一些非常有用的统计属性可以帮助我们在机器学习中的证明，包括切比雪夫和马尔科夫不等式

# Probabilistic Model 概率模型课件

• ### Expectation Maximisation

Proof of convergence for E-M, examples of E-M through Gaussian Mixture Model, [gmm_demo.m] and [kmeans_demo.m] and [bilibili video]

• ### State Space Model (Dynamic model)

explain in detail of Kalman Filter [bilibili video], [kalman_demo.m] and Hidden Markov Model [bilibili video]

# Inference 推断课件

• ### Variational Inference

explain Variational Bayes both the non-exponential and exponential family distribution plus stochastic variational inference. [vb_normal_gamma.m] and [bilibili video] 变分推导的介绍: 解释变分贝叶斯非指数和指数族分布加上随机变分推断。[vb_normal_gamma.m][B站视频链接]

• ### Stochastic Matrices

stochastic matrix, Power Method Convergence Theorem, detailed balance and PageRank algorithm 随机矩阵，幂方法收敛定理，详细平衡和谷歌PageRank算法

• ### Markov Chain Monte Carlo

M-H, Gibbs, Slice Sampling, Elliptical Slice sampling, Swendesen-Wang, demonstrate collapsed Gibbs using LDA [lda_gibbs_example.m] and [test_autocorrelation.m] and [gibbs.m] and [bilibili video]

• ### Particle Filter (Sequential Monte-Carlo)

Sequential Monte-Carlo, Condensational Filter algorithm, Auxiliary Particle Filter [bilibili video] 粒子滤波器（序列蒙特卡洛）[B站视频链接]

• ### Bayesian Non Parametrics (BNP) and its inference basics

Dircihlet Process (DP), Chinese Restaurant Process insights, Slice sampling for DP [dirichlet_process.m] and [bilibili video] and [Jupyter Notebook]

• ### Bayesian Non Parametrics (BNP) extensions

Hierarchical DP, HDP-HMM, Indian Buffet Process (IBP) 非参贝叶斯扩展: 层次狄利克雷过程，分层狄利克雷过程-隐马尔可夫模型，印度自助餐过程(IBP)

• ### Completely Random Measure (early draft - written in 2015)

Levy-Khintchine representation, Compound Poisson Process, Gamma Process, Negative Binomial Process Levy-Khintchine表示，复合Poisson过程，Gamma过程，负二项过程

• ### Sample correlated integers from HDP and Copula

This is an alternative explanation to our IJCAI 2016 papers. The derivations are different from the paper, but portraits the same story. 这是对我的IJCAI2016论文 的一个不同解释。虽然写的方法公式推导不同，但描绘的是同一事情

• ### Determinantal Point Process

explain the details of DPP’s marginal distribution, L-ensemble, its sampling strategy, our work in time-varying DPP 行列式点过程解释:行列式点过程的边缘分布，L-ensemble，其抽样策略，我们在“时变行列式点过程”中的工作细节

# Special Thanks

• I would like to thank my following PhD students for help me proofreading, and provide great discussions and suggestions to various topics in these notes, including (but not limited to) Hayden Chang, Shawn Jiang, Erica Huang, Deng Chen, Ember Liang; 特别感谢我的博士生团队协助我一起校对课件，以及就课件内容所提出的想法和建议，团队成员包括（但不限于）常浩东，姜帅，黄皖鸣，邓辰，梁轩。

• I always look for high quality PhD students in Machine Learning, both in terms of probabilistic model and Deep Learning models. Contact me on YiDa.Xu@uts.edu.au 如果你想加入我的机器学习博士生团队或有兴趣实习, 请通过YiDa.Xu@uts.edu.au与我联系。

My continuously updated Machine Learning, Probabilistic Models and Deep Learning notes and demos (2000+ slides) 我不间断更新的机器学习，概率模型和深度学习的讲义(2000+页)和视频链接

## Releases

No releases published

## Packages 0

No packages published