Course Information

Estimation Theory - Spring 2022

Instructor: Prof. Songhwai Oh (오성회)
Email: songhwai (at) snu.ac.kr
Office Hours: Friday 2:00-4:00PM
Office: Building 133 Room 403
Course Number: 430.714
Time: MW 2:00-3:15 PM
Location: Online (Building 301 Room 102)
TA: Minjae Kang (강민재)
Email: minjae.kang (at) rllab.snu.ac.kr
Office: Building 133 Room 610
 

Course Description

Week
Reading
Date
Lecture
Date
Lecture
1
 
 
 
3/2
  • Introduction
2

Kay Ch. 1; Simon Ch. 1, Ch. 2

3/7
  • Review on linear system theory
3/9
  • Review on probability
3
Kay Ch. 2, Ch. 3.1 - 3.9
3/14
  • Minimum variance unbiased estimators
3/16
  • Cramer-Rao lower bound (CRLB)
4
Kay Ch. 4, Ch. 5
3/21
  • Linear models
3/23
  • Sufficient statistics
5
Kay Ch. 6, Ch. 7.1 - 7.6
3/28
  • Best linear unbiased estimators
3/30
  • Maximum likelihood estimation
6
Kay Ch. 8, Ch. 10
4/4
  • Least squares
4/6
  • Exponential family
  • Bayesian approach
7
Kay Ch. 11
4/11
  • Multivariate Gaussian
4/13
  • Bayes risk, MMSE, MAP
8
Kay Ch. 12
4/18
  • Linear MMSE
4/20

Midterm

  • Time: 1:30 - 3:15PM
  • Location: 301-102
9
 
4/25
  • Sequential linear MMSE
4/27
  • Bayesian filtering
10
Simon Ch. 5, Ch. 6
5/2
  • Kalman filter
5/4
  • Alternate Kalman filter formulations
11
Simon Ch. 7, Ch. 9
5/9
  • Kalman filter generalizations
5/11
  • Optimal smoothing (part 1)
12
Simon Ch. 9, Ch. 13
5/16
  • Optimal smoothing (part 2)
5/18
  • Nonlinear Kalman filtering
13
Simon Ch. 14, Ch. 15
5/23
  • Unscented Kalman filter
5/25
  • Particle filtering
14
 
5/30
  • Data association and multi-target tracking
6/1
  • Gaussian process regression
15
 
6/6
 
6/8

Final Exam

  • Time: 1:30 - 3:15PM
  • Location: 301-102

This course introduces classical and modern topics in estimation theory to graduate level students. Topics include minimum variance unbiased estimators, the Cramer-Rao bound, linear models, sufficient statistics, best linear unbiased estimators, maximum likelihood estimators, least squares, exponential family, multivariate Gaussian distribution, Bayes risk, minimum mean square error (MMSE), maximum a posteriori (MAP), linear MMSE, sequential linear MMSE, Bayesian filtering, Kalman filters, extended Kalman filter, unscented Kalman filter, particle filter, data association, multi-target tracking, and Gaussian process regression. Lectures will be in English.

Announcements

  • [06/08] The final exam will be held in class on 6/8 (Wed). The exam is closed-book but you can bring one sheet (A4) of handwritten notes on both sides. You have to turn in this cheat sheet with your exam.
  • [04/06] The midterm will be held in class on 4/20 (Wed). The exam is closed-book but you can bring one sheet (A4) of handwritten notes on a single side (the other side must be blank). You have to turn in this cheat sheet with your exam. 
  • [02/28] Please read Ethics of Learning.

Schedule

Textbooks

  • [Recommended] Steven M. Kay, "Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory", Prentice Hall, 1993.

  • [Recommended] Dan Simon, "Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches", Wiley-Interscience, 2006.

Prerequisites

  • Students must have a solid background in linear algebra, linear system theory, and probability.

Topics

  • Introduction and review of probability and linear system theory
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound
  • Linear models and sufficient statistics
  • Best linear unbiased estimators and maximum likelihood estimators
  • Least squares, exponential family, and Bayesian approaches
  • Multivariate Gaussian distribution
  • Bayes risk, minimum mean square error (MMSE), and maximum a posteriori (MAP)
  • Linear MMSE and sequential linear MMSE
  • Bayesian filtering
  • Kalman filtering
  • Advanced topics in Kalman filtering
  • Extended Kalman filter, unscented Kalman filter, and particle filter
  • *Data association and multi-target tracking
  • *Gaussian process regression (*if time permits)