tags: 課程共筆

AIAcademy 課程共筆

主頁

Machine Learning

Problem setting

= Classification and Regression

Algorithm

• Linear Regression(LR) and Multivariable Linear Regression(MLR)
• Logistic Regression
• KNN
• Decision Tree (CART)
  • Root node, Leaf node
  • Gini-index and Gini-table
• Random Forest
  • Bagging
• XGBoost
  • Gradient Boosting

Regularization

• L1 and L2 Norm
• Soft and hard margin

Verification

• Estimation error
• ROC

Deep Learning

Model Structure

• MLP
• *Tensors

Activation

• Sigmoid
• Softmax

Initialization

• *Xavier
• *HeNormal

Loss

• Categorical Cross Entropy
• Binary Cross Entropy
• Mean Square Error

Gradient Descent & Backpropagation

• Gradient
• Gradient Vanishing

Optimization

• SGD- learning rate/mini batch/momentum
• RMSProp
• Adam

Normalization

• Batch Normalization
• Instance Normalization

Training Control

• *Learning Rate Schedular