ECON622

This is a graduate topics course in computational economics, with applications in datascience and machine learning.

Course materials

• Get a GitHub ID and apply for the Student Developer Pack to get further free features
• Consider clicking Watch at the top of this repository to see file changes

Syllabus

See Syllabus for more details

Problem Sets

See problemsets.md.

Lectures

Paul

1. September 6: Environment and Introduction to Julia
   • Environment: read one or both of these on your own and install Julia, IJulia, and VSCode, preferrably before the first class
     • Julia Environment
     • MoJuWo: Writing your code
   • In class: Motivating econometric examples
   • Self-study: Introductory Examples
2. September 11: From Economic Models to Code
   • In class: Implementing random coefficient demand & numeric integration methods
   • Self-study: Julia Essentials
   • Self-study: Fundamental Types
3. September 13: Optimization
   • In class: discussion of types and using them to reorganize numeric integration code from last time and testing them
   • Optimization algorithms
   • Optimization packages
   • Self-study: Generic Programming
   • Self-study: Linear Algebra
4. September 18: Automatic Differentiation
   • In class: Overview of optimization packages and automatic differentiation packages
   • Self-study: Intro to AD
   • Self-study: Differentiation for Hackers
   • Self-study: Engineering Trade-Offs in Automatic Differentiation: from TensorFlow and PyTorch to Jax and Julia
   • Optional:
     • Understanding automatic differentiation (in Julia)
     • Forward and Reverse Automatic Differentiation In A Nutshell
5. Sepember 20: Extremum Estimation
   • Extremum estimation and inference
   • Empirical likelihood
   • Bootstrap
   • Self-study: General Packages
   • Self-study: Data and Statistical Packages
6. September 25: Testing and Package Development
   • In class: development of our random coefficients demand package as an example
   • Self-study: Testing and Packages
   • Self-study: Git and Github
7. September 27: Code Performance
   • Coding for performance be sure to look at the 2023 branch for the recent additions
   • GPU usage
   • Self-study: SIMDscan: since it briefly came up in class, and I was curious about it, I made a little package for calculating things like cumulative sums and autoregressive simulations using SIMD
   • Self-study: Need for speed
   • Self-study: Performance Tips

JESSE

Slides for the lectures can be found here

8. October 4th: Factorizations, Direct Methods, and Intro to Regularization

   • SLIDES: Factorizations and Direct Methods
   • Introduction to regularization and implicit bias of algorithms
   • Numerical Linear Algebra applying generic programming

9. October 11: Iterative Methods, Geometry of Optimization, and Rethinking LLS

   • SLIDES: Least Squares and Iterative Methods
   • Iterative Methods

10. October 12: Preconditioning and Overview of Machine Learning

    • SLIDES: Iterative Methods and Intro to ML
    • Finalize discussion of iterative methods and preconditioning
    • Introduce key concepts about supervised, unsupervised, reinforcement learning, semi-supervised, kernel-methods, deep-learning, etc.

11. October 16: Differentiable everything! JAX and Auto-Differentiation/JVP/etc.

    • SLIDES: Finish Intro to ML and start Differentiation
    • Core JAX transforms
    • Reverse-mode and forward-mode AD.
    • Jvps and vjps

12. October 18: AD of Implicit Functions, Optimization Methods for High-Dimensional Problems

    • SLIDES: Finish Differentiation and start Optimization
    • Implicit differentiation of systems of ODEs, linear systems, etc.
    • Gradient descent variations

13. October 23th: Stochastic Optimization Methods

    • SLIDES: SGD variations in Optimization
    • SGD and methods for variance reduction in gradient estimates
    • Using SGD-variants in practice within ML pipelines in JAX and Pytorch

14. October 25th: Machine Learning Pipelines, HPO, and ERM

    • SLIDES: Finished example code of pipelines in Optimization, W&B sweeps, and code in lectures/lectures/examples
    • Readings: Probabilistic Machine Learning: An Introduction Section 5.4 on ERM

15. October 30: Neural Networks, Representation Learning, Double-Descent

    • SLIDES: Deep Learning and Representation Learning and started Double-Descent and Regularization
    • Readings
      • Probabilistic Machine Learning: An Introduction Section 13.2.1 to 13.2.6 on MLPs and the importance of depth
      • Probabilistic Machine Learning: An Introduction Section 13.5.1 to 13.5.6 on regularization
      • Mark Schmidt's CPSC440 Notes on Neural Networks (see CPSC340 lectures for a more basic treatment of these topics)
      • Mark Schmidt's CPSC440 Notes on Double-Descent Curves (see CPSC340 lectures for a more basic treatment of these topics)
    • Optional Extra Material
      • Probabilistic Machine Learning: Advanced Topics Section 32 on representation learning

16. November 1 Finish Double-Descent and Intro to Kernel Methods and Gaussian Processes

    • SLIDES: Kernel Methods and Gaussian Processes and finish Double-Descent and Regularization
    • Readings
      • If you didn't do it already, read Mark Schmidt's CPSC440 Notes on Double-Descent Curves and Overparameterization (see CPSC340 lectures for a more basic treatment of these topics)
      • Probabilistic Machine Learning: An Introduction Section 17.1 and 17.2 on Kernel methods and Gaussian Processes
      • CPSC340 has some notes on the "kernel trick", and you can skip over the details on images. Also see more advanced notes on kernel methods
      • Finally, your problem set will involve running some simple Gaussian Processes with GPyTorch, which will become easier to understand after seeing the theory.
      • Probabilistic Machine Learning: Advanced Topics Section 18.1 to 18.3 on GPs and kernels
      • Researchers working in GPs love the online textbook Gaussian Processes for Machine Learning, so you may want to read the intro section on GP Regression

17. November 6 GUEST LECTURE GEOFF PLEISS FROM UBC STATS

PHIL

18. November 8 Graphical models introduction
    • Notes Course notes on graphical models
    • Readings Probabilistic Machine Learning: Advanced Topics Chapter 4.1-4.5
19. November 13 Fall break
20. November 15 Fall break
21. November 20 Using graphical models, Identification, Variable elimination, Undirected graphical models
    • Notes Course notes on graphical models
    • Slides: First set of slides, Second set
    • Readings:
      • Probabilistic Machine Learning: Advanced Topics Chapter 4.1-4.5
      • Probabilistic Machine Learning: Advanced Topics Chapter 9.5 (Variable elimination)
22. November 22 Sampling; Inverse CDF, Rejection/importance sampling, MCMC/Metropolis-Hastings algorithm
    • Notes Course notes on Monte-Carlo methods
    • Readings
      • Probabilistic Machine Learning: Advanced Topics Chapter 11
23. November 27 Local (Gibbs) sampling, Hamiltonian Monte-Carlo, ODE solvers, Probabilistic programming languages (PPLs)
    • Notes Course notes on Monte-Carlo methods
    • Readings
      • Probabilistic Machine Learning: Advanced Topics Chapter 12
      • Hidden Markov Models in Turing.jl and Pyro
24. November 29 NLP (short), Tuning foundation models, Graph theory, Graph spectrum, Network games
    • Readings
      • Course notes on network games
      • Datascience notebook on social & economic networks
      • Huggingface fine-tuning demo
      • Melissa Dell's packages LinkTransformer (for merging datasets) and LayoutParser (for OCR/digitization)
25. December 4 Student presentations 1
    • Eileen Bian - Application of linear programming
    • Xilin Song - Extension of BLP with product space similarities
    • Jacky Tan - Heterogeneous agent models of market microstructure
26. December 6 Student presentations 2
    • Yidan Chen - Digitization of patent data using NLP
    • Kisho Hoshi - Replication of discrete choice paper with recommender systems
    • Bruno Esposito - Optimizing the code from Balke and Lamadon (2022) using JAX
    • James Yu - Within-tier heterogeneity in directed search
27. Decemeber 20
    • Final Project due

Look under "Releases" for earlier versions of the course.