Course Information

Estimation Theory - Fall 2024

Instructor: Prof. Songhwai Oh (오성회)
Email: songhwai (at) snu.ac.kr
Office Hours: Friday 3:00-4:00PM
Office: Building 133 Room 403
Course Number: 430.714
Time: MW 5:00-6:15 PM
Location: Building 301 Room 201
TA: Junseo Lee (이준서)
Email: junseo.lee (at) rllab.snu.ac.kr
Office: Building 133 Room 610
 

Course Description

Week
Reading
Date
Lecture
Date
Lecture
1
 
9/2
  • Introduction
  • Review on linear system theory
9/4
  • Review on probability
2

Kay Ch. 1; Simon Ch. 1, Ch. 2

Kay Ch. 3.1 - 3.9

Kay Ch. 4, Ch. 5

9/9
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound (CRLB)
9/11
  • Linear models
  • Sufficient statistics
3
 
9/16
  • (Thanksgiving Holiday)
9/18
  • (Thanksgiving Holiday)
4

9/23
  • (No class)
9/25
  • (No class)
5
Kay Ch. 6, Ch. 7.1 - 7.6
9/30
  • Best linear unbiased estimators
  • Maximum likelihood estimation
10/2
  • Least squares
6
Kay Ch. 8, Ch. 10
10/7
  • Exponential family
  • Bayesian approach
10/9
  • (Holiday)
  • Bayesian approach
7
Kay Ch. 11
10/14
  • (No class)
  • Multivariate Gaussian
10/16
  • (No class)
  • Bayes risk, MMSE, MAP
8
Kay Ch. 12
10/21
  • Linear MMSE
10/23

Midterm

  • Time: 5:00 - 6:30 PM
  • Location: in class
9
 
10/28
  • Sequential linear MMSE
10/30
  • Bayesian filtering
10
Simon Ch. 5, Ch. 6
11/4
  • Kalman filter
11/6
  • Alternate Kalman filter formulations
11
Simon Ch. 7, Ch. 9
11/11
  • Kalman filter generalizations
11/13
  • Optimal smoothing (1)
  • Optimal smoothing (2)
12
Simon Ch. 9, Ch. 13
11/18
  • Optimal smoothing (2)
  • Nonlinear Kalman filtering
11/20
  • Unscented Kalman filter
13
Simon Ch. 14, Ch. 15
11/25
  • Particle filtering
11/27
 
14
 
12/2
  • Data association and multi-target tracking
12/4
  • Gaussian process regression
15
 
12/9
 
12/11

16
 
12/16

Final Exam

  • Time: 5:00 - 6:30 PM
  • Location: in class
 

 

This course introduces classical and modern topics in estimation theory to graduate level students. Topics include minimum variance unbiased estimators, the Cramer-Rao bound, linear models, sufficient statistics, best linear unbiased estimators, maximum likelihood estimators, least squares, exponential family, multivariate Gaussian distribution, Bayes risk, minimum mean square error (MMSE), maximum a posteriori (MAP), linear MMSE, sequential linear MMSE, Bayesian filtering, Kalman filters, extended Kalman filter, unscented Kalman filter, particle filter, data association, multi-target tracking, Gaussian process regression, and deep learning. Lectures will be in English.

Announcements

  • [12/04] The final exam will be held in class on 12/16 (Mon). The exam is closed-book but you can bring one sheet (A4) of handwritten notes on both sides. You have to turn in this cheat sheet with your exam.
  • [10/07] The midterm will be held in class on 10/23 (Wed). The exam is closed-book but you can bring one sheet (A4) of handwritten notes on a single side (the other side must be blank). You have to turn in this cheat sheet with your exam. 
  • [08/26] Please read Ethics of Learning.

Schedule

Textbooks

  • [Recommended] Steven M. Kay, "Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory", Prentice Hall, 1993.

  • [Recommended] Dan Simon, "Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches", Wiley-Interscience, 2006.

Prerequisites

  • Students must have a solid background in linear algebra, linear system theory, and probability.

Topics

  • Introduction and review of probability and linear system theory
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound
  • Linear models and sufficient statistics
  • Best linear unbiased estimators and maximum likelihood estimators
  • Least squares, exponential family, and Bayesian approaches
  • Multivariate Gaussian distribution
  • Bayes risk, minimum mean square error (MMSE), and maximum a posteriori (MAP)
  • Linear MMSE and sequential linear MMSE
  • Bayesian filtering
  • Kalman filtering
  • Advanced topics in Kalman filtering
  • Extended Kalman filter, unscented Kalman filter, and particle filter
  • *Data association and multi-target tracking
  • *Gaussian process regression
  • *Deep learning (*if time permits)