Instructor: Da Kuang
This course introduces numerical linear algebra from a data analysis perspective. Emphasis will be given to matrix computation arising from unsupervised clustering, dimension reduction, and optimization. In the first half of the course, students will work on mini-projects that relate numerical linear algebra to data analysis tasks; in the second half, students will read and implement a recent research paper on large-scale machine learning involving matrix computations. Overall, this course offers a solid understanding of the theory and practical implementation of matrix algorithms, which is important for effectively using existing machine learning tools and packages as well as creating new ones.
- MATH 115A (or equivalent) required.
- PIC 10B (or equivalent) or PIC 20A (or equivalent) or PIC 97 (Python) required.
- MATH 151A/B (or equivalent such as EE 133A) recommended.
- You will be expected to learn Python yourself by reading tutorials and working on projects.
- For the students who do not satisfy the programming prerequisites but still want to register, please complete this tutorial and send me an email with your answers to all the programming exercises in it, in order to gain an exception for registration.
- Singular value decomposition (SVD)
- Conditioning of a matrix
- Algorithms for least-squares fitting
- Matrix factorizations
- Overview of iterative solvers and eigensolvers
- Applications in data analysis
- Sparse coding
- Locally linear embedding
- Principal component analysis
- Nonnegative matrix factorization
- Spectral clustering
- Implementation issues
- Numerical software stack
- Sparse matrices and special structures
- Parallel matrix algorithms
Slides are available on your CCLE website.
- 40% Homework
- 40% Final project and presentation
- 20% Final exam
- No late homework allowed. However, there are no penalties for medical reasons or emergencies. You must submit a doctor's note or an official letter explaining the emergency.
- Attendance is not counted towards your final grade. However, if you missed a class without a valid reason, the instructor will not answer your questions that have been addressed in-class.
Each homework assignment includes 1~2 theoretical questions and a mini-project.
Please note that while discussion is allowed and encouraged, individual students must write up their own answers and computer programs.
All the students must observe the conduct code
In the final project, you will:
- Form groups of 2-3 persons
- Read a research paper and implement the algorithm in Python
- Reproduce the experiment results
- Apply the algorithm to a real data set
Each group should communicate with the instructor about the scope of experiments and the deliverables. You will very likely need a little literature search in order to understand the algorithms.
Example paper list: (for papers considered for implementation for the current quarter, please refer to the "Paper walk-through" slides on CCLE)
||Topics in Python
||Numerical software stack
||Vector and matrix norms
Singular value decomposition
|Singular value decomposition (cont'd)
HW1 due; HW2 out
|Principal component analysis
HW2 due; HW3 out
Team formation due
HW3 due; HW4 out
|Image classification pipeline
||Conditioning of a matrix
|(President's Day holiday)
Least squares algorithms
Project proposal due
|Techniques for introducing zeros
for dense LU, QR, SVD/EVD
|Locally linear embedding
||Locally linear embedding
||Nonnegative matrix factorization
|Iterative solvers and eigensolvers
||Iterative solvers and eigensolvers
Final exam: Mar 21 (Tuesday), 8:00-11:00am
Final report due: Mar 24 (Friday) 11:59pm