Course Information

Estimation Theory - Fall 2025

Instructor: Prof. Songhwai Oh (오성회)
Email: songhwai (at) snu.ac.kr
Office Hours: Friday 3:00-4:00PM
Office: Building 133 Room 503
Course Number: 430.714
Time: MW 5:00-6:15 PM
Location: Building 301 Room 201
TA: Hyeokjin Kwon (권혁진)
Email: hyeokjin.kwon (at) rllab.snu.ac.kr
Office: Building 133 Room 610
 

Course Description

Week
Reading
Date
Lecture
Date
Lecture
1
Kay Ch. 1; Simon Ch. 1, Ch. 2
9/1
  • Introduction
  • Review on linear system theory
9/3
  • Review on probability
2

Kay Ch. 3.1 - 3.9

9/8
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound (CRLB)
9/10
  • Cramer-Rao lower bound (CRLB)
3
Kay Ch. 4, Ch. 5, Ch. 6
9/15
  • Linear models
  • Sufficient statistics
9/17
  • Sufficient statistics
  • Best linear unbiased estimators
4

Kay Ch. 6, Ch. 7.1 - 7.6

9/22
  • Best linear unbiased estimators
  • Maximum likelihood estimation
9/24
  • Least squares
5

Kay Ch. 8, Ch. 10, Ch. 11

9/29
  • Exponential family
  • Bayesian approach
10/1
  • Multivariate Gaussian
  • Bayes risk, MMSE, MAP
6
 
10/6
  • (Holiday)
10/8
  • (Holiday)
7
Kay Ch. 12
10/13
  • Linear MMSE
10/15
  • Sequential linear MMSE
8
 
10/20
  • (No class)
10/22

Midterm

  • Time: 5:00 - 6:30 PM
  • Location: in class
9
Simon Ch. 5, Ch. 6
10/27
  • Bayesian filtering
10/29
  • Kalman filter
10
Simon Ch. 7
11/3
  • Alternate Kalman filter formulations
11/5
  • Kalman filter generalizations
11
Simon Ch. 9
11/10
  • Optimal smoothing (1)
11/12
  • Optimal smoothing (2)
12
Simon Ch. 13, Ch. 14
11/17
  • Nonlinear Kalman filtering
  • Unscented Kalman filter
11/19
  • Unscented Kalman filter
13
Simon Ch. 15
11/24
  • Particle filtering
11/26
  • Data association and multi-target tracking
14
 
12/1
  • Gaussian process regression
12/3

15
 
12/8
 
12/10

16
 
12/15

Final Exam

  • Time: 5PM-7PM
  • Location: 301-201
 

 

This course introduces classical and modern topics in estimation theory to graduate level students. Topics include minimum variance unbiased estimators, the Cramer-Rao bound, linear models, sufficient statistics, best linear unbiased estimators, maximum likelihood estimators, least squares, exponential family, multivariate Gaussian distribution, Bayes risk, minimum mean square error (MMSE), maximum a posteriori (MAP), linear MMSE, sequential linear MMSE, Bayesian filtering, Kalman filters, extended Kalman filter, unscented Kalman filter, particle filter, data association, multi-target tracking, Gaussian process regression, and deep learning. Lectures will be in English.

Announcements

  • [12/01] The final exam will be held in class on 12/15 (Mon). The exam is closed-book but you can bring one sheet (A4) of handwritten notes on both sides. You have to turn in this cheat sheet with your exam.
  • [10/01] The midterm will be held in class on 10/22 (Wed). The exam is closed-book but you can bring one sheet (A4) of handwritten notes on a single-side (the other side must be blank). You have to turn in this cheat sheet with your exam. 
  • [08/26] Please read Ethics of Learning.

Schedule

Textbooks

  • [Recommended] Steven M. Kay, "Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory", Prentice Hall, 1993.

  • [Recommended] Dan Simon, "Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches", Wiley-Interscience, 2006.

Prerequisites

  • Students must have a solid background in linear algebra, linear system theory, and probability.

Topics

  • Introduction and review of probability and linear system theory
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound
  • Linear models and sufficient statistics
  • Best linear unbiased estimators and maximum likelihood estimators
  • Least squares, exponential family, and Bayesian approaches
  • Multivariate Gaussian distribution
  • Bayes risk, minimum mean square error (MMSE), and maximum a posteriori (MAP)
  • Linear MMSE and sequential linear MMSE
  • Bayesian filtering
  • Kalman filtering
  • Advanced topics in Kalman filtering
  • Extended Kalman filter, unscented Kalman filter, and particle filter
  • *Data association and multi-target tracking
  • *Gaussian process regression
  • *Deep learning (*if time permits)