1 Basic Probability Theory 1
1.1 Introduction 1
1.2 Sample Spaces and Events 3
1.3 The Axioms of Probability 7
1.4 Finite Sample Spaces and Combinatorics 15
1.4.1 Combinatorics 17
1.5 Conditional Probability and Independence 27
1.5.1 Independent Events 33
1.6 The Law of Total Probability and Bayes’ Formula 41
1.6.1 Bayes’ Formula 47
1.6.2 Genetics and Probability 54
1.6.3 Recursive Methods 55
Problems 63
2 Random Variables 76
2.1 Introduction 76
2.2 Discrete Random Variables 77
2.3 Continuous Random Variables 82
2.3.1 The Uniform Distribution 90
2.3.2 Functions of Random Variables 92
2.4 Expected Value and Variance 95
2.4.1 The Expected Value of a Function of a Random Variable 100
2.4.2 Variance of a Random Variable 104
2.5 Special Discrete Distributions 111
2.5.1 Indicators 111
2.5.2 The Binomial Distribution 112
2.5.3 The Geometric Distribution 116
2.5.4 The Poisson Distribution 117
2.5.5 The Hypergeometric Distribution 121
2.5.6 Describing Data Sets 121
2.6 The Exponential Distribution 123
2.7 The Normal Distribution 127
2.8 Other Distributions 131
2.8.1 The Lognormal Distribution 131
2.8.2 The Gamma Distribution 133
2.8.3 The Cauchy Distribution 134
2.8.4 Mixed Distributions 135
2.9 Location Parameters 137
2.10 The Failure Rate Function 139
2.10.1 Uniqueness of the Failure Rate Function 141
Problems 144
3 Joint Distributions 156
3.1 Introduction 156
3.2 The Joint Distribution Function 156
3.3 Discrete Random Vectors 158
3.4 Jointly Continuous Random Vectors 160
3.5 Conditional Distributions and Independence 164
3.5.1 Independent Random Variables 168
3.6 Functions of Random Vectors 172
3.6.1 Real-Valued Functions of Random Vectors 172
3.6.2 The Expected Value and Variance of a Sum 176
3.6.3 Vector-Valued Functions of Random Vectors 182
3.7 Conditional Expectation 185
3.7.1 Conditional Expectation as a Random Variable 189
3.7.2 Conditional Expectation and Prediction 191
3.7.3 Conditional Variance 192
3.7.4 Recursive Methods 193
3.8 Covariance and Correlation 196
3.8.1 The Correlation Coefficient 201
3.9 The Bivariate Normal Distribution 209
3.10 Multidimensional Random Vectors 216
3.10.1 Order Statistics 218
3.10.2 Reliability Theory 223
3.10.3 The Multinomial Distribution 225
3.10.4 The Multivariate Normal Distribution 226
3.10.5 Convolution 227
3.11 Generating Functions 231
3.11.1 The Probability Generating Function 231
3.11.2 The Moment Generating Function 237
3.12 The Poisson Process 240
3.12.1 Thinning and Superposition 244
Problems 247
4 Limit Theorems 263
4.1 Introduction 263
4.2 The Law of Large Numbers 264
4.3 The Central Limit Theorem 268
4.3.1 The Delta Method 273
4.4 Convergence in Distribution 275
4.4.1 Discrete Limits 275
4.4.2 Continuous Limits 277
Problems 278
5 Simulation 281
5.1 Introduction 281
5.2 Random Number Generation 282
5.3 Simulation of Discrete Distributions 283
5.4 Simulation of Continuous Distributions 285
5.5 Miscellaneous 290
Problems 292
6 Statistical Inference 294
6.1 Introduction 294
6.2 Point Estimators 294
6.2.1 Estimating the Variance 302
6.3 Confidence Intervals 304
6.3.1 Confidence Interval for the Mean in the Normal Distribution with Known Variance 307
6.3.2 Confidence Interval for an Unknown Probability 308
6.3.3 One-Sided Confidence Intervals 312
6.4 Estimation Methods 312
6.4.1 The Method of Moments 312
6.4.2 Maximum Likelihood 315
6.4.3 Evaluation of Estimators with Simulation 322
6.4.4 Bootstrap Simulation 324
6.5 Hypothesis Testing 327
6.5.1 Large Sample Tests 332
6.5.2 Test for an Unknown Probability 333
6.6 Further Topics in Hypothesis Testing 334
6.6.1 P-Values 334
6.6.2 Data Snooping 335
6.6.3 The Power of a Test 336
6.6.4 Multiple Hypothesis Testing 338
6.7 Goodness of Fit 339
6.7.1 Goodness-of-Fit Test for Independence 346
6.7.2 Fisher’s Exact Test 349
6.8 Bayesian Statistics 351
6.8.1 Noninformative priors 359
6.8.2 Credibility Intervals 362
6.9 Nonparametric Methods 363
6.9.1 Nonparametric Hypothesis Testing 363
6.9.2 Comparing Two Samples 370
6.9.3 Nonparametric Confidence Intervals 375
Problems 378
7 Linear Models 391
7.1 Introduction 391
7.2 Sampling Distributions 392
7.3 Single Sample Inference 395
7.3.1 Inference for the Variance 396
7.3.2 Inference for the Mean 399
7.4 Comparing Two Samples 402
7.4.1 Inference about Means 402
7.4.2 Inference about Variances 407
7.5 Analysis of Variance 409
7.5.1 One-Way Analysis of Variance 409
7.5.2 Multiple Comparisons：Tukey’s Method 412
7.5.3 Kruskal-Wallis Test 413
7.6 Linear Regression 415
7.6.1 Prediction 422
7.6.2 Goodness of Fit 424
7.6.3 The Sample Correlation Coefficient 425
7.6.4 Spearman’s Correlation Coefficient 429
7.7 The General Linear Model 431
Problems 436
8 Stochastic Processes 444
8.1 Introduction 444
8.2 Discrete-Time Markov Chains 445
8.2.1 Time Dynamics of a Markov Chain 447
8.2.2 Classification of States 450
8.2.3 Stationary Distributions 454
8.2.4 Convergence to the Stationary Distribution 460
8.3 Random Walks and Branching Processes 464
8.3.1 The Simple Random Walk 464
8.3.2 Multidimensional Random Walks 468
8.3.3 Branching Processes 469
8.4 Continuous-Time Markov Chains 475
8.4.1 Stationary Distributions and Limit Distributions 480
8.4.2 Birth-Death Processes 484
8.4.3 Queueing Theory 488
8.4.4 Further Properties of Queueing Systems 491
8.5 Martingales 494
8.5.1 Martingale Convergence 495
8.5.2 Stopping Times 497
8.6 Renewal Processes 502
8.6.1 Asymptotic Properties 504
8.7 Brownian Motion 509
8.7.1 Hitting Times 512
8.7.2 Variations of the Brownian Motion 515
Problems 517
Appendix A Tables 527
Appendix B Answers to Selected Problems 535
Further Reading 551
Index 553