index.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sascha Spors,\n",
    "Professorship Signal Theory and Digital Signal Processing,\n",
    "Institute of Communications Engineering (INT),\n",
    "Faculty of Computer Science and Electrical Engineering (IEF),\n",
    "University of Rostock,\n",
    "Germany\n",
    "\n",
    "# Data Driven Audio Signal Processing - A Tutorial with Computational Examples\n",
    "\n",
    "Master Course #24512\n",
    "\n",
    "- lecture: https://github.com/spatialaudio/data-driven-audio-signal-processing-lecture\n",
    "- tutorial: https://github.com/spatialaudio/data-driven-audio-signal-processing-exercise\n",
    "\n",
    "Feel free to contact lecturer frank.schultz@uni-rostock.de\n",
    "\n",
    "This tutorial is still evolving, so major parts could be rearranged from year to year to fit the actual students' demands. We will prepare more audio examples in future..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework Task Template\n",
    "\n",
    "- [Homework Task Template](homework/homework.ipynb)\n",
    "\n",
    "# Planned Syllabus Winter Semester 2024/25\n",
    "\n",
    "- [Numerical Examples from the Slides](slides/ddasp_exercise_slides.ipynb)\n",
    "\n",
    "\n",
    "## Exercise : Motivation / Introducing a Toy Example for SVD / Regression\n",
    "- [Introduction to the Course](exercise01.ipynb)\n",
    "\n",
    "\n",
    "## Exercise : Singular Value Decomposition (SVD) / 4 Subspaces of a Matrix\n",
    "\n",
    "- [SVD and 4 Subspaces](exercise04_svd.ipynb)\n",
    "- [SVD and 4 Subspaces, above example as Matlab script](svd_four_subspaces.m) \n",
    "\n",
    "\n",
    "## Exercise: Left Inverse / Linear Regression with Ordinary Least Squares (OLS)\n",
    "- [SVD and Left Inverse](exercise04_leftinv.ipynb)\n",
    "- [SVD and Right Inverse](exercise04_rightinv.ipynb)\n",
    "- [Line Fit with Linear Regression](line_fit_linear_regression.ipynb)\n",
    "- [Linear Regression with OLS](ols.ipynb)\n",
    "\n",
    "\n",
    "## Exercise: SVD Factorization for Multitrack Audio Matrix\n",
    "- [exercise05.ipynb](exercise05.ipynb)\n",
    "\n",
    "\n",
    "## Exercise: Linear and Ridge Regression on Multitrack Audio Matrix\n",
    "- [L-curve to find optimum regularization parameter](lcurve.ipynb)\n",
    "- [exercise08_ridge_regression.ipynb](exercise08_ridge_regression.ipynb)\n",
    "- [exercise07_left_inverse_SVD_QR.ipynb](exercise07_left_inverse_SVD_QR.ipynb)\n",
    "\n",
    "## Exercise: Audio Features\n",
    "- [Audio Features I](exercise02.ipynb) (Segmentation, STFT, Spectrogram, Periodogram)\n",
    "- [Audio Features II](exercise03.ipynb) (Segmentation, RMS/(True)Peak/Crest Factor, R128 loudness)\n",
    "\n",
    "## Exercise: PCA\n",
    "- [pca_2D.ipynb](pca_2D.ipynb)\n",
    "- [pca_3D.ipynb](pca_3D.ipynb)\n",
    "- [pca_audio_features.ipynb](pca_audio_features.ipynb)\n",
    "- [pca_of_exam_grades.ipynb](pca_of_exam_grades.ipynb)\n",
    "\n",
    "## Exercise: Bias Variance Trade-Off vs. Model Complexity\n",
    "- [Bias-Variance Trade-Off vs. Model Complexity](bias_variance_linear_regression.ipynb)\n",
    "\n",
    "\n",
    "## Exercise: Bias Variance Trade-Off vs. Hyper Parameter Tuning\n",
    "- [Bias-Variance Trade-Off vs. Regularization](bias_variance_ridge_regression.ipynb)\n",
    "\n",
    "\n",
    "## Exercise: Gradient Descent along a 2D Surface\n",
    "- [Gradient Descent 1](gradient_descent.ipynb) with one saddle point\n",
    "- [Gradient Descent 2](gradient_descent2.ipynb) with saddle points, local maximum, local minima and a global minimum\n",
    "- [Gradient Descent with Momentum](gradient_descent_momentum.ipynb)\n",
    "- [Stochastic Gradient Descent for Least Squares Error](gradient_descent_on_least_squares.ipynb)\n",
    "- [Stochastic Gradient Descent for Complex Data Least Squares Error Using PyTorch](gradient_descent_on_complex_data_least_squares.ipynb)\n",
    "\n",
    "\n",
    "\n",
    "## Exercise: XOR as Non-Linear, Two-Layer Model\n",
    "- The XOR mapping is a popular example to motivate non-linearities in models, as linear regression  in [exercise10_xor_example.m](exercise10_xor_example.m) cannot solve this simple problem (coding this in Python on our own is a good practice for linear algebra handling and OLS)\n",
    "- The XOR mapping as as a simple **0/1 classification** using a **non-linear hidden layer** and **linear output layer** is coded in [regression_xor_twolayers.ipynb](regression_xor_twolayers.ipynb)\n",
    "\n",
    "We should not miss these brilliant resources to start with neural networks\n",
    "- [https://pythonalgos.com/create-a-neural-network-from-scratch-in-python-3/](https://pythonalgos.com/create-a-neural-network-from-scratch-in-python-3/)\n",
    "- [https://playground.tensorflow.org](https://playground.tensorflow.org)\n",
    "- https://www.tensorflow.org/tutorials/keras/overfit_and_underfit (and the other tutorials found there)\n",
    "\n",
    "\n",
    "\n",
    "## Exercise: Binary Logistic Regression with Only One Sigmoid Output Layer\n",
    "- [exercise10_binary_logistic_regression.py](exercise10_binary_logistic_regression.py)\n",
    "- With [Binary logistic regression, our implementation vs. Tensorflow](binary_logistic_regression_tf.ipynb) we compare an **own implementation against a TF model**\n",
    "\n",
    "\n",
    "## Exercise: Binary Logistic Regression with Hidden Layers and Sigmoid Output Layer\n",
    "- Next, we create more complex models in [binary_logistic_regression_tf_with_hidden_layers.ipynb](binary_logistic_regression_tf_with_hidden_layers.ipynb) using **hidden layers**, but still with **manually tuned hyper parameters**\n",
    "- It might be worth to spend time with this brilliant application https://playground.tensorflow.org/ to get a feeling how models get trained on rather simple data sets\n",
    "\n",
    "\n",
    "## Exercise: Multi-Class Classification with Hidden Layers and Softmax Output Layer\n",
    "\n",
    "- With [exercise12_MulticlassClassification_CategoricalCrossentropy.ipynb](exercise12_MulticlassClassification_CategoricalCrossentropy.ipynb) we expand the binary classification example [binary_logistic_regression_tf_with_hidden_layers.ipynb](binary_logistic_regression_tf_with_hidden_layers.ipynb)\n",
    "towards more classes (note that classes are exclusive)\n",
    "\n",
    "\n",
    "## Exercise: Hyper Parameter Tuning\n",
    "\n",
    "- With [exercise12_HyperParameterTuning.ipynb](exercise12_HyperParameterTuning.ipynb) we introduce\n",
    "    - data split into train, validate, test data sets\n",
    "    - hyper parameter tuning  \n",
    "    - one hot encoding\n",
    "    - training of best model with re-set weights using train / val data set\n",
    "    - final prediction on unseen test data set compared to predictions on train / val data sets\n",
    "    - confusion matrix and visualization of predictions\n",
    "    \n",
    "## Exercise: Simple Music Genre Classification Application\n",
    "   \n",
    "- Finally we apply all our knowledge so far to realize a music genre classification application in [exercise12_MusicGenreClassification.ipynb](exercise12_MusicGenreClassification.ipynb)\n",
    "    - feature design (loudness, crest, peak, rms, spectral weight)\n",
    "    - feature inspection / avoiding NaNs\n",
    "    - feature normalization\n",
    "    - balancing data set wrt class occurence\n",
    "    \n",
    "We could move on with dropout layers, regularization...\n",
    "\n",
    "We could also consider CNNs to work on STFT maps to solve this task...or combine CNN and DNN...nice homework project...\n",
    "\n",
    "And of course we might check the research literature, e.g. on [IEEE Xplore](https://ieeexplore.ieee.org/search/searchresult.jsp?newsearch=true&queryText=music%20genre%20classification), to figure the current state of the research.\n",
    "We then might realize that recent models are little more complicated and probably exhibit some unknown tools, but the fundamentals remain the same.\n",
    "Hence, we should be able to work in the literature after attending our courses and comprehending all the provided material.\n",
    "Please, do nice and useful things with ML :-)\n",
    "\n",
    "\n",
    "## Important Things We Might Not Cover in the Tutorials \n",
    "- [Audio Signal Fundamentals](audio_introduction.ipynb)\n",
    "- [QR factorization.ipynb](exercise07_QR.ipynb)\n",
    "- [linear_regression LS_vs_SVD](exercise07_linear_regression_LS_vs_SVD.ipynb)\n",
    "- [Tensorflow convolution vs correlation](tf_conv1D_vs_corr_conv.ipynb)\n",
    "- [Tensorflow 2D convolution](exercise13_CNN.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Textbook Recommendations\n",
    "Machine Learning (ML) using linear / non-linear models is a vivid topic and dozens of textbooks will be released each year.\n",
    "The following textbook recommendations are very often referenced in the field and brilliant to learn with.  \n",
    "- Kevin P. **Murphy**: *Probabilistic Machine Learning: An Introduction*, MIT Press, 1st. ed. [open source book and current draft as free pdf](https://probml.github.io/pml-book/book1.html)\n",
    "- J.A. **Fessler**, R.R. **Nadakuditi**: *Linear Algebra for Data Science, Machine Learning, and Signal Processing*, Cambridge University Press, 2024, 1st ed. [online ebook](https://ebookcentral.proquest.com/lib/ubrostock-ebooks/detail.action?docID=31691281)\n",
    "- Sebastian **Raschka**, Yuxi Liu, Vahid Mirjalili: *Machine Learning with PyTorch and Scikit-Learn*, Packt, 2022, 1st ed.\n",
    "- Gilbert **Strang**: *Linear Algebra and Learning from Data*, Wellesley, 2019, consider to buy your own copy of this brilliant book\n",
    "- Gareth **James**, Daniela Witten, Trevor Hastie, Rob Tibshirani: *An Introduction to Statistical Learning* with Applications in R, Springer, 2nd ed., 2021, [free pdf e-book](https://www.statlearning.com/)\n",
    "- Trevor **Hastie**, Robert Tibshirani, Jerome Friedman: *The Elements of  Statistical Learning: Data Mining, Inference, and Prediction*, Springer, 2nd ed., 2009, [free pdf e-book](https://hastie.su.domains/ElemStatLearn/)\n",
    "- Sergios **Theodoridis**: *Machine Learning*, Academic Press, 2nd ed., 2020, check your university library service for free pdf e-book\n",
    "- Ian **Goodfellow**, Yoshua Bengio, Aaron Courville: *Deep Learning*, MIT Press, 2016\n",
    "- Marc Peter **Deisenroth**, A. Aldo Faisal, Cheng Soon Ong: *Mathemathics for Machine Learning*, Cambridge University Press, 2020, [free pdf e-book](https://mml-book.github.io/)\n",
    "- Steven L. **Brunton**, J. Nathan Kutz: *Data Driven Science & Engineering - Machine Learning, Dynamical Systems, and Control*, Cambridge University Press, 2020, [free pdf of draft](http://www.databookuw.com/databook.pdf), see also the [video lectures](http://www.databookuw.com/) and [Python tutorials](https://github.com/dylewsky/Data_Driven_Science_Python_Demos)\n",
    "- Aurélien **Géron**: *Hands-on machine learning with Scikit-Learn, Keras and TensorFlow*. O’Reilly, 2nd ed., 2019, [Python tutorials](https://github.com/ageron/handson-ml2)\n",
    "\n",
    "ML deals with stuff that is actually known for decades (at least the linear modeling part of it), so if we are really serious about to learn ML deeply, we should elaborate concepts on statistical signal processing, maximum-likelihood, Bayesian vs. frequentist statistics, generalized linear models, hierarchical models...For these topics we could check these respected textbooks\n",
    "- L. **Fahrmeir**, A. Hamerle, and G. Tutz, *Multivariate statistische Verfahren*, 2nd ed. de Gruyter, 1996.\n",
    "- L. **Fahrmeir**, T. Kneib, S. Lang, and B. D. Marx, *Regression*, 2nd ed. Springer, 2021.\n",
    "- A. J. **Dobson** and A. G. Barnett, *An Introduction to Generalized Linear Models*, 4th ed. CRC Press, 2018.\n",
    "- J. F. **Monahan**, *A Primer on Linear Models*, CRC Press, 2008.\n",
    "- H. **Madsen**, P. Thyregod, *Introduction to General and Generalized Linear Models*, CRC Press, 2011.\n",
    "- A. **Agresti**, *Foundations of Linear and Generalized Models*, Wiley, 2015"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open Course Ware Recommendations\n",
    "\n",
    "- Online Course by Andrew **Ng** et al. at https://www.coursera.org/ and https://www.deeplearning.ai/\n",
    "- Online Course by Gilbert **Strang** et al. at https://ocw.mit.edu/courses/mathematics/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/\n",
    "- Online Course/Material by Aurélien **Géron** https://github.com/ageron\n",
    "- Online Course by Meinard **Müller** https://www.audiolabs-erlangen.de/resources/MIR/FMP/B/B_GetStarted.html (focus on music information retrieval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Autorship\n",
    "- current main authors\n",
    "    - University of Rostock\n",
    "        - [Frank Schultz](https://orcid.org/0000-0002-3010-0294)\n",
    "        - [Sascha Spors](https://orcid.org/0000-0001-7225-9992)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Copyright\n",
    "\n",
    "- the notebooks are provided as [Open Educational Resources](https://en.wikipedia.org/wiki/Open_educational_resources)\n",
    "- the text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/)\n",
    "- the code of the IPython examples is licensed under the [MIT license](https://opensource.org/licenses/MIT)\n",
    "- feel free to use the notebooks for your own purposes considering above licenses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Referencing\n",
    "- please cite this open educational resource (OER) project as *Frank Schultz, Data Driven Audio Signal Processing - A Tutorial Featuring Computational Examples, University of Rostock* ideally with relevant file(s), github URL https://github.com/spatialaudio/data-driven-audio-signal-processing-exercise, commit number and/or version tag, year."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "myddasp",
   "language": "python",
   "name": "myddasp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}