Instructor: Da Kuang

Course description:
This course introduces numerical linear algebra from a data analysis perspective. Emphasis will be given to matrix computation arising from unsupervised clustering, dimension reduction, and optimization. Students will read and implement recent research papers on large-scale machine learning involving matrix computations as final projects. Overall, this course offers a solid understanding of the theory and practical implementation of matrix algorithms, which is important for effectively using existing machine learning tools and packages as well as creating new ones.

Linear algebra (33A or equivalent) required. Introductory programming (PIC 10A or equivalent) required.
Linear algebra (115AB or equivalent) recommended. Familiarity with Matlab or Python (numpy) programming recommended. You will need to learn Python yourself by reading tutorials and working on projects.

Relationship with other courses:
In terms of numerical algorithms, this course is in some sense complementary to topics in 151A and 151B.
In terms of applications of numerical linear algebra, this course is devoted to data analysis, as opposed to solving PDEs in the graduate course MATH 270.

*New* Selected Final Project Report


Textbooks and references



Late Submissions Policy


Please note that while discussion is allowed and encouraged, individual students must write up their own answers and computer programs.
All the students must observe the conduct code.


Project Instructions

In the final project, you will:

Each group should communicate with the instructor about the scope of experiments and the deliverables. You will very likely need a little literature search in order to understand the algorithms.
If the implementation is available online, you should write it in a different programming language. Bonus points for significant improvement of the original algorithm/implementation.

Example paper list: (for papers considered for implementation for the current quarter, please refer to the "Paper walk-through" slides on CCLE)


Date Mon Wed Fri
Jan 4
Jan 6
Jan 8
Python learning: Set up
Matrix-vector multiplication
Reading: NLA, Lecture 1
Python learning: Introduction
Matrix-matrix multiplication
Range, subspace, norms
Reading: NLA, Lecture 1, 3
Python learning: Strings
Jan 11
Jan 13
Jan 15
Numerical software stack
Python exercises:
K-means clustering
HW1 out
Optional reading: MMDS, Chap 7
Python learning: Lists
Singular value decomposition
Reading: NLA, Lecture 2, 4
Python learning: Sorting
Jan 18
Jan 20
Jan 22
(MLK holiday) Singular value decomposition
HW1 due; HW2 out
Reading: NLA, Lecture 4, 5
Python exercises:
Singular value decomposition
Principal component analysis
Reading: NLA, Lecture 5
Python learning: Dicts and files
Jan 25
Jan 27
Jan 29
Principal component analysis
Code Data
Optional reading: ESL Chap 14.5.1
Python exercises:
Reading: NLA, Lecture 12
HW2 due; HW3 out
Python exercises:
Reading: NLA, Lecture 6
Team formation due
Python exercises:
Feb 1
Feb 3
Feb 5
Least squares problem
Reading: NLA, Lecture 11
Python exercises:
Techniques for introducing zeros
for computing LU, QR, SVD/EVD
Reading: NLA, Lecture 10
HW4 out
Python learning: Regular expressions
Project overview
(object recognition, movie recommendation,
Yelp challenge)
HW3 due
Python exercises: Baby names
Feb 8
Feb 10
Feb 12
Project overview
(paper walk-through)
Python learning: Utilities
Sparse coding
HW4 due
Python exercises: Copy files
Sparse coding
Python exercises: Log puzzle
Feb 15
Feb 17
Feb 19
(Presidents' Day holiday) Midterm Locally linear embedding
Feb 22
Feb 24
Feb 26
Locally linear embedding
Project proposal due
Sparse matrices Spectral clustering
Feb 29
Mar 2
Mar 4
Spectral clustering Nonnegative matrix factorization Random projection
Randomized SVD
Mar 7
Mar 9
Mar 11
Final presentation Final presentation Final presentation
Mar 14
Mar 16
Mar 18
Final project report due
(No final exam)