Course Outline

Introduction: Estimation in signal processing---the mathematical estimation problem---assessing estimator performance.

Minimum Variance Unbiased Estimation: Unbiased estimators---MVU criterion---existence of the MVU---finding the MVU---extension to vector parameter.

Cramer-Rao Lower Bound: CRLB---Fisher Information---general CRLB for signals in WGN---transformation of parameters---CRLB for vector parameters---transformation of vector parameters---asymptotic CRLB for WSS Gaussian processes---signal processing examples.

Linear Models: Definition and properties---linear model examples---extension to the linear model.

General MVU Estimation: Sufficient statistics---finding sufficient statistics by Neyman-Fisher factorization---using sufficiency to find the MVUE---RBLS Theorem---extension to vector parameter.

Best Linear Unbiased Estimator: Definition---finding the BLUE---extension to vector parameter---signal processing example.

Maximum Likelihood Estimation: Motivation---finding the MLE---properties of the MLE---MLE of transformed parameters: the principle of invariance---numerical determination of the MLE---extension to vector parameter---asymptotic MLE---signal processing examples.

Least Squares Estimation: Least squares approach---linear LS---weighted LS---geometrical interpretation---order-recursive LS---sequential LS---non-linear LS---signal processing examples.

Method of Moments: Definition---extension to vector parameter---statistical evaluation of estimators---signal processing examples.

Bayesian Estimation: Prior knowledge and estimation---choosing a prior PDF---Bayesian linear model---nuisance parameters---risk functions---MAP estimation---extension to vector parameter---performance description.