1 Introduction 1
1.0 Overview 2
1.1 Introduction 2
1.2 Discrete Random Variables and Generating Functions 6
1.3 Continuous Random Variables and Laplace Transforms 17
1.4 Some Mathematical Background 28
Problems 37
Bibliographic Notes 42
Appendix 43
References 43
2 Poisson Processes 47
2.0 Overview 47
2.1 Introduction 48
2.2 Properties of Poisson Processes 51
2.3 Nonhomogeneous Poisson Processes 56
2.4 Compound Poisson Processes 72
2.5 Filtered Poisson Processes 76
2.6 Two-Dimensional and Marked Poisson Processes 80
2.7 Poisson Arrivals See Time Averages(PASTA) 83
Problems 87
Bibliographic Notes 93
References 94
Appendix 95
3 Renewal Processes 97
3.0 Overview 97
3.1 Introduction 98
3.2 Renewal-Type Equations 101
3.3 Excess Life,Current Life,and Total Life 107
3.4 Renewal Reward Processes 118
3.5 Limiting Theorems,Stationary and Transient Renewal Processes 128
3.6 Regenerative Processes 132
3.7 Discrete Renewal Processes 144
Problems 146
Bibliographic Notes 154
References 155
Appendix 156
4 Discrete-Time Markov Chains 160
4.0 Overview 160
4.1 Introduction 161
4.2 Classification of States 167
4.3 Ergodic and Periodic Markov Chains 175
4.4 Absorbing Markov Chains 188
4.5 Markov Reward Processes 203
4.6 Reversible Discrete-Time Markov Chains 207
Problems 212
Bibliographic Notes 225
References 226
Appendix 227
5 Continuous-Time Markov Chains 238
5.0 Overview 239
5.1 Introduction 239
5.2 The Kolmogorov Differential Equations 245
5.3 The Limiting Probabilities 252
5.4 Absorbing Continuous-Time Markov Chains 256
5.5 Phase-Type Distributions 264
5.6 Uniformization 273
5.7 Continuous-Time Markov Reward Processes 277
5.8 Reversible Continuous-Time Markov Chains 284
Problems 298
Bibliographic Notes 313
References 314
Appendix 316
6 Markov Renewal and Semi-Regenerative Processes 321
6.1 Introduction 322
6.0 Overview 322
6.2 Markov Renewal Functions and Equations 331
6.3 Semi-Markov Processes and Related Reward Processes 339
6.4 Semi-Regenerative Processes 348
Problems 363
Bibliographic Notes 367
References 367
Appendix 368
7 Brownian Motion and Other Diffusion Processes 373
7.0 Overview 373
7.1 Introduction 374
7.2 Diffusion Processes 385
7.3 Ito's Calculus and Stochastic Differential Equations 396
7.4 Multidimensional Ito's Lemma 404
7.5 Control of Systems of Stochastic Differential Equations 409
Problems 417
Bibliographic Notes 419
References 420
Appendix 421
Appendix:Getting Started with MATLAB 427
Index 436