Course Information

Estimation Theory - Fall 2019
Instructor: Prof. Songhwai Oh (오성회)
Email: songhwai (at) snu.ac.kr
Office Hours: Friday 2:00-4:00PM
Office: Building 133 Room 405		 Course Number: 430.714
Time: MW 2:00-3:15 PM
Location: Building 302 Room 408
TA: Chanho Ahn (안찬호)
Email: chanho.ahn (at) rllab.snu.ac.kr
Office: Building 133 Room 610		  

Course Description

This course introduces classical and modern topics in estimation theory to graduate level students. Topics include minimum variance unbiased estimators, the Cramer-Rao bound, linear models, sufficient statistics, best linear unbiased estimators, maximum likelihood estimators, least squares, exponential family, multivariate Gaussian distribution, Bayes risk, minimum mean square error (MMSE), maximum a posteriori (MAP), linear MMSE, sequential linear MMSE, Bayesian filtering, Kalman filters, extended Kalman filter, unscented Kalman filter, particle filter, data association, multi-target tracking, and Gaussian process regression. Lectures will be in English.

Announcements

  • [11/20] The final exam will be held in class on 12/4 (Wed). The exam is closed-book but you can bring one sheet (A4) of hand-written notes on both sides. You have to turn in this cheat sheet with your exam.
  • [10/14] The midterm will be held in class on 10/23 (Wed). The exam is closed-book but you can bring one sheet (A4) of hand-written notes on a single side (the other side must be blank). You have to turn in this cheat sheet with your exam.
  • [08/26] Please read Ethics of Learning.

Schedule

Week Reading Date Lecture Date Lecture
1 Kay Ch. 1
Simon Ch. 1, 2		 9/2
  • Introduction
  • Review on linear system theory
 9/4
  • Review on probability
2


Kay Ch. 2, Ch. 3.1 - 3.9, Ch. 4

 9/9
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound (CRLB)
 9/11
  • Cramer-Rao lower bound (CRLB)
  • Linear models
3 Kay Ch. 5, Ch. 6

 9/16
  • Sufficient statistics
 9/18
  • Best linear unbiased estimators
4 Kay Ch. 7.1 - 7.6, Ch. 8 9/23
  • Maximum likelihood estimation
  • Least squares
 9/25
  • Least squares
  • Exponential family
5 Kay Ch. 10, Ch. 11, Ch. 12 9/30
  • Bayesian approach
  • Multivariate Gaussian
 10/2
  • Bayes risk, MMSE, MAP
  • Linear MMSE
6 Kay Ch. 12 10/7
  • Sequential linear MMSE
 10/9
  • Holiday
7 Simon Ch. 5 10/14
  • Bayesian filtering
 10/16
  • Kalman filter
8 Simon Ch. 6 10/21
  • Alternate Kalman filter formulations
 10/23
  • Midterm
    • in class
9 Simon Ch. 7 10/28
  • Kalman filter generalizations
 10/30
  • No class
10   11/4
  • No class
 11/6
  • No class
11 Simon Ch. 9 11/11
  • Optimal smoothing
 11/13
  • Optimal smoothing
12 Simon Ch. 13, Ch. 14 11/18
  • Nonlinear Kalman filtering
 11/20
  • Unscented Kalman filter
13 Simon Ch. 15 11/25
  • Particle filtering
 11/27
  • Gaussian process regression
14   12/2   12/4
  • Final Exam
    • Locaition: 302-408
    • Time: 12:30 - 3:25

Textbooks

  • [Recommended] Steven M. Kay, "Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory", Prentice Hall, 1993.
  • [Recommended] Dan Simon, "Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches", Wiley-Interscience, 2006.

Prerequisites

  • Students must have a solid background in linear algebra, linear system theory, and probability.

Topics

  • Introduction and review of probability and linear system theory
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound
  • Linear models and sufficient statistics
  • Best linear unbiased estimators and maximum likelihood estimators
  • Least squares, exponential family, and Bayesian approaches
  • Multivariate Gaussian distribution
  • Bayes risk, minimum mean square error (MMSE), and maximum a posteriori (MAP)
  • Linear MMSE and sequential linear MMSE
  • Bayesian filtering
  • Kalman filtering
  • Advanced topics in Kalman filtering
  • Extended Kalman filter, unscented Kalman filter, and particle filter
  • *Data association and multi-target tracking
  • *Gaussian process regression (*if time permits)