1 Introduction 1
1.1 Estimation in Signal Processing 1
1.2 The Mathematical Estimation Problem 7
1.3 Assessing Estimator Performance 9
1.4 Some Notes to the Reader 12
2 Minimum Variance Unbiased Estimation 15
2.1 Introduction 15
2.2 Summary 15
2.3 Unbiased Estimators 16
2.4 Minimum Variance Criterion 19
2.5 Existence of the Minimum Variance Unbiased Estimator 20
2.6 Finding the Minimum Variance Unbiased Estimator 21
2.7 Extension to a Vector Parameter 22
3 Cramer-Rao Lower Bound 27
3.1 Introduction 27
3.2 Summary 27
3.3 Estimator Accuracy Considerations 28
3.4 Cramer-Rao Lower Bound 30
3.5 General CRLB for Signals in White Gaussian Noise 35
3.6 Transformation of Parameters 37
3.7 Extension to a Vector Parameter 39
3.8 Vector Parameter CRLB for Transformations 45
3.9 CRLB for the General Gaussian Case 47
3.10 Asymptotic CRLB for WSS Gaussian Random Processes 50
3.11 Signal Processing Examples 53
3A Derivation of Scalar Parameter CRLB 67
3B Derivation of Vector Parameter CRLB 70
3C Derivation of General Gaussian CRLB 73
3D Derivation of Asymptotic CRLB 77
4 Linear Models 83
4.1 Introduction 83
4.2 Summary 83
4.3 Definition and Properties 83
4.4 Linear Model Examples 86
4.5 Extension to the Linear Model 94
5 General Minimum Variance Unbiased Estimation 101
5.1 Introduction 101
5.2 Summary 101
5.3 Sufficient Statistics 102
5.4 Finding Sufficient Statistics 104
5.5 Using Sufficiency to Find the MVU Estimator 107
5.6 Extension to a Vector Parameter 116
5A Proof of Neyman-Fisher Factorization Theorem （Scalar Parameter） 127
5B Proof of Rao-Blackwell-Lehmann-Scheffe Theorem （Scalar Parameter） 130
6 Best Linear Unbiased Estimators 133
6.1 Introduction 133
6.2 Summary 133
6.3 Definition of the BLUE 134
6.4 Finding the BLUE 136
6.5 Extension to a Vector Parameter 139
6.6 Signal Processing Example 141
6A Derivation of Scalar BLUE 151
6B Derivation of Vector BLUE 153
7 Maximum Likelihood Estimation 157
7.1 Introduction 157
7.2 Summary 157
7.3 An Example 158
7.4 Finding the MLE 162
7.5 Properties of the MLE 164
7.6 MLE for Transformed Parameters 173
7.7 Numerical Determination of the MLE 177
7.8 Extension to a Vector Parameter 182
7.9 Asymptotic MLE 190
7.10 Signal Processing Examples 191
7A Monte Carlo Methods 205
7B Asymptotic PDF of MLE for a Scalar Parameter 211
7C Derivation of Conditional Log-Likelihood for EM Algorithm Example 214
8 Least Squares 219
8.1 Introduction 219
8.2 Summary 219
8.3 The Least Squares Approach 220
8.4 Linear Least Squares 223
8.5 Geometrical Interpretations 226
8.6 Order-Recursive Least Squares 232
8.7 Sequential Least Squares 242
8.8 Constrained Least Squares 251
8.9 Nonlinear Least Squares 254
8.10 Signal Processing Examples 260
8A Derivation of Order-Recursive Least Squares 282
8B Derivation of Recursive Projection Matrix 285
8C Derivation of Sequential Least Squares 286
9 Method of Moments 289
9.1 Introduction 289
9.2 Summary 289
9.3 Method of Moments 289
9.4 Extension to a Vector Parameter 292
9.5 Statistical Evaluation of Estimators 294
9.6 Signal Processing Example 299
10 The Bayesian Philosophy 309
10.1 Introduction 309
10.2 Summary 309
10.3 Prior Knowledge and Estimation 310
10.4 Choosing a Prior PDF 316
10.5 Properties of the Gaussian PDF 321
10.6 Bayesian Linear Model 325
10.7 Nuisance Parameters 328
10.8 Bayesian Estimation for Deterministic Parameters 330
10A Derivation of Conditional Gaussian PDF 337
11 General Bayesian Estimators 341
11.1 Introduction 341
11.2 Summary 341
11.3 Risk Functions 342
11.4 Minimum Mean Square Error Estimators 344
11.5 Maximum A Posteriori Estimators 350
11.6 Performance Description 359
11.7 Signal Processing Example 365
11A Conversion of Continuous-Time System to Discrete-Time System 375
12 Linear Bayesian Estimators 379
12.1 Introduction 379
12.2 Summary 379
12.3 Linear MMSE Estimation 380
12.4 Geometrical Interpretations 384
12.5 The Vector LMMSE Estimator 389
12.6 Sequential LMMSE Estimation 392
12.7 Signal Processing Examples - Wiener Filtering 400
12A Derivation of Sequential LMMSE Estimator 415
13 Kalman Filters 419
13.1 Introduction 419
13.2 Summary 419
13.3 Dynamical Signal Models 420
13.4 Scalar Kalman Filter 431
13.5 Kalman Versus Wiener Filters 442
13.6 Vector Kalman Filter 446
13.7 Extended Kalman Filter 449
13.8 Signal Processing Examples 452
13A Vector Kalman Filter Derivation 471
13B Extended Kalman Filter Derivation 476
14 Summary of Estimators 479
14.1 Introduction 479
14.2 Estimation Approaches 479
14.3 Linear Model 486
14.4 Choosing an Estimator 489
15 Extensions for Complex Data and Parameters 493
15.1 Introduction 493
15.2 Summary 493
15.3 Complex Data and Parameters 494
15.4 Complex Random Variables and PDFs 500
15.5 Complex WSS Random Processes 513
15.6 Derivatives，Gradients，and Optimization 517
15.7 Classical Estimation with Complex Data 524
15.8 Bayesian Estimation 532
15.9 Asymptotic Complex Gaussian PDF 535
15.10Signal Processing Examples 539
15A Derivation of Properties of Complex Covariance Matrices 555
15B Derivation of Properties of Complex Gaussian PDF 558
15C Derivation of CRLB and MLE Formulas 563
A1 Review of Important Concepts 567
A1.1 Linear and Matrix Algebra 567
A1.2 Probability，Random Processes，and Time Series Models 574
A2 Glossary of Symbols and Abbreviations 583
INDEX 589