1 Sets and Events 1
1.1 Introduction 1
1.2 Basic Set Theory 2
1.2.1 Indicator functions 5
1.3 Limits of Sets 6
1.4 Monotone Sequences 8
1.5 Set Operations and Closure 11
1.5.1 Examples 13
1.6 Theσ-field Generated by a Given Class C 15
1.7 Borel Sets on the Real Line 16
1.8 Comparing Borel Sets 18
1.9 Exercises 20
2 Probability Spaces 29
2.1 Basic Definitions and Properties 29
2.2 More on Closure 35
2.2.1 Dynkin's theorem 36
2.2.2 Proof of Dynkin's theorem 38
2.3 Two Constructions 40
2.4 Constructions of Probability Spaces 42
2.4.1 General Construction of a Probability Model 43
2.4.2 Proof of the Second Extension Theorem 49
2.5 Measure Constructions 57
2.5.1 Lebesgue Measure on(0,1] 57
2.5.2 Construction of a Probability Measure on R witl Given Distribution Function F(x) 61
2.6 Exercises 63
3 Random Variables,Elements,and Measurable Maps 71
3.1 Inverse Maps 71
3.2 Measurable Maps,Random Elements,Induced Probability Measures 74
3.2.1 Composition 77
3.2.2 Random Elements of Metric Spaces 78
3.2.3 Measurability and Continuity 80
3.2.4 Measurability and Limits 81
3.3 σ-Fields Generated by Maps 83
3.4 Exercises 85
4 Independence 91
4.1 Basic Definitions 91
4.2 Independent Random Variables 93
4.3 Two Examples of Independence 95
4.3.1 Records,Ranks,Renyi Theorem 95
4.3.2 Dyadic Expansions of Uniform Random Numbers 98
4.4 More on Independence：Groupings 100
4.5 Independence,Zero-One Laws,Borel-Cantelli Lemma 102
4.5.1 Borel-Cantelli Lemma 102
4.5.2 Borel Zero-One Law 103
4.5.3 Kolmogorov Zero-One Law 107
4.6 Exercises 110
5 Integration and Expectation 117
5.1 Preparation for Integration 117
5.1.1 Simple Functions 117
5.1.2 Measurability and Simple Functions 118
5.2 Expectation and Integration 119
5.2.1 Expectation of Simple Functions 119
5.2.2 Extension of the Definition 122
5.2.3 Basic Properties of Expectation 123
5.3 Limits and Integrals 131
5.4 Indefinite Integrals 134
5.5 The Trarnsformation Theorem and Densities 135
5.5.1 Expectation is Always an Integral on R 137
5.5.2 Densities 139
5.6 The Riemann vs Lebesgue Integral 139
5.7 Product Spaces 143
5.8 Probability Measures on Product Spaces 147
5.9 Fubini's theorem 149
5.10 Exercises 155
6 Convergence Concepts 167
6.1 Almost Sure Convergence 167
6.2 Convergence in Probability 169
6.2.1 Statistical Terminology 170
6.3 Connections Between a.s.and i.p.Convergence 171
6.4 Quantile Estimation 178
6.5 Lp Convergence 180
6.5.1 Uniform Integrability 182
6.5.2 Interlude：A Review of Inequalities 186
6.6 More on Lp Convergence 189
6.7 Exercises 195
7 Laws of Large Numbers and Sums of Independent Raudom Variables 203
7.1 Truncation and Equivalence 203
7.2 A General Weak Law of Large Numbers 204
7.3 Almost Sure Convergence of Sums of Independent Random Variables 209
7.4 Strong Laws of Large Numbers 213
7.4.1 Two Examples 215
7.5 The Strong Law of Large Numbers for IID Sequences 219
7.5.1 Two Applications of the SLLN 222
7.6 The Kolmogorov Three Series Theorem 226
7.6.1 Necessity of the Kolmogorov Three Series Theorem 230
7.7 Exercises 234
8 Convergence in Distribution 247
8.1 Basic Definitions 247
8.2 Scheffé's lemma 252
8.2.1 Scheffé's lemma and Order Statistics 255
8.3 The Baby Skorohod Theorem 258
8.3.1 The Delta Method 261
8.4 Weak Convergence Equivalences；Portmanteau Theorem 263
8.5 More Relations Among Modes of Convergence 267
8.6 New Convergences from Old 268
8.6.1 Example：The Central Limit Theorem for m-Dependent Random Variables 270
8.7 The Convergence to Types Theorem 274
8.7.1 Application of Convergence to Types：Limit Distributions for Extremes 278
8.8 Exercises 282
9 Characteristic Functions and the Central Limit Theorem 293
9.1 Review of Moment Generating Functions and the Central Limit Theorem 294
9.2 Characteristic Functions：Definition and First Properties 295
9.3 Expansions 297
9.3.1 Expansion of eix 297
9.4 Moments and Derivatives 301
9.5 Two Big Theorems：Uniqueness and Continuity 302
9.6 The Selection Theorem,Tightness,and Prohorov's theorem 307
9.6.1 The Selection Theorem 307
9.6.2 Tightness,Relative Compactness,and Prohorov's theorem 309
9.6.3 Proof of the Continuity Theorem 311
9.7 The Classical CLT for iid Random Variables 312
9.8 The Lindeberg-Feller CLT 314
9.9 Exercises 321
10 Martingales 333
10.1 Prelude to Conditional Expectation：The Radon-Nikodym Theorem 333
10.2 Definition of Conditional Expectation 339
10.3 Properties of Conditional Expectation 344
10.4 Martingales 353
10.5 Examples of Martingales 356
10.6 Connections between Martingales and Submartingales 360
10.6.1 Doob's Decomposition 360
10.7 Stopping Tirmes 363
10.8 Positive Super Martingales 366
10.8.1 Operations on Supermartingales 367
10.8.2 Uperossings 369
10.8.3 Boundedness Properties 369
10.8.4 Convergence of Positive Super Martingales 371
10.8.5 Closure 374
10.8.6 Stopping Supermartingales 377
10.9 Examples 379
10.9.1 Gambler's Ruin 379
10.9.2 Branching Processes 380
10.9.3 Some Differentiation Theory 382
10.10 Martingale and Submartingale Convergence 386
10.10.1 Krickeberg Decomposition 386
10.10.2 Doob's(Sub)martingale Convergence Theorem 387
10.11 Regularity and Closure 388
10.12 Regularity and Stopping 390
10.13 Stopping Theorems 392
10.14 Wald's Identity and Random Walks 398
10.14.1 The Basic Martingales 400
10.14.2 Regular Stopping Times 402
10.14.3 Examples of Integrable Stopping Times 407
10.14.4 The Simple Random Walk 409
10.15 Reversed Martingales 412
10.16 Fundamental Theorems of Mathematical Finance 416
10.16.1 A Simple Market Model 416
10.16.2 Admissible Strategies and Arbitrage 419
10.16.3 Arbitrage and Martingales 420
10.16.4 Complete Markets 425
10.16.5 Option Pricing 428
10.17 Exercises 429
References 443
Index 445