AI 351: A Mathematical Approach to Machine Learning
Title: A Mathematical Approach to Machine Learning with Pytorch
Instructor:Â Tony Shaska
Offered: Winter 2024 (Oakland Univ.)
Meeting Times: MWF 9:20-10:27
Prerequisites Linear Algebra, Calculus III
Most machine learning workflows involve working with data, creating models, using hyperparameters to optimize model, saving and inferencing the trained models. This class introduces you to mathematical foundations of machine learning and a complete machine learning (ML) workflow implemented in PyTorch, a popular ML framework for Python.
Resources
Part I: Fundamentals
Textbook:Â Mathematics for Machine Learning, Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong
Chap 2: Linear Algebra
Chap 3: Analytic Geometry
Chap 4: Matrix Decomposition
Chap 5: Vector Calculus
Chap 9: Linear Regression
Chap 12: Classification with Support Vector machines
Part II: Special Topics
KMeans
Gradient Descent and Stochastic Gradient Descent SGD
A mathematical approach to Neural Networks
Activation functions
Backpropagation method
Shallow and deep networks
Convolutional Neural Networks
Vanishing of the gradient
DenseNet: Densely Connected Convolutional Networks
Deep Learning
Degradation problem
Deep Residual Networks
Geometric Learning
Invariant theory and machine learning
Large Language Models
Automated Theorem Proving
An introduction to Lean
Projects
Linear Regression
KMeans
Implementing a Neural Network in Numpy
Implementing a Neural Network in Pytorch
A DenseNet example through Pytorch
Literature
Understanding Deep Learning, Simon J. D. Prince
Artificial Intelligence: A modern approach (4th edition)
Author's website:Â https://aima.cs.berkeley.edu/The mathematics of deep learning, https://arxiv.org/abs/2105.04026
Machine Learning book , S. Raschka, et al.Â
Bishop, Christopher. Neural Networks for Pattern Recognition. New York, NY: Oxford University Press, 1995. ISBN: 9780198538646.
Duda, Richard, Peter Hart, and David Stork. Pattern Classification. 2nd ed. New York, NY: Wiley-Interscience, 2000. ISBN: 9780471056690.
Hastie, T., R. Tibshirani, and J. H. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York, NY: Springer, 2001. ISBN: 9780387952840.
The Elements of Statistical Learning  (author's website)
MacKay, David. Information Theory, Inference, and Learning Algorithms. Cambridge, UK: Cambridge University Press, 2003. ISBN: 9780521642989. Available on-line here.
Mitchell, Tom. Machine Learning. New York, NY: McGraw-Hill, 1997. ISBN: 9780070428072.
Links of interests
Mathematics in Machine Learning, AMS Special session, April 19-21, 2024
Oxford Summer School of Machine Learning (May 7-10, July 6-14, 2024)