Chapter 1．Basic Stochastic Calculus 1
1．Probability 1
1.1．Probability spaces 1
1.2．Random variables 4
1.3．Conditional expectation 8
1.4．Convergence of probabilities 13
2．Stochastic Processes 15
2.1．General considerations 15
2.2．Brownian motions 21
3．Stopping Times 23
4．Martingales 27
5．It？'s Integral 30
5.1．Nondifferentiability of Brownian motion 30
5.2．Definition of It？'s integral and basic properties 32
5.3．It？'s formula 36
5.4．Martingale representation theorems 38
6．Stochastic Differential Equations 40
6.1．Strong solutions 41
6.2．Weak solutions 44
6.3．Linear SDEs 47
6.4．Other types of SDEs 48
Chapter 2．Stochastic Optimal Control Problems 51
1．Introduction 51
2．Deterministic Cases Revisited 52
3．Examples of Stochastic Control Problems 55
3.1．Production planning 55
3.2．Investment vs.consumption 56
3.3．Reinsurance and dividend management 58
3.4．Technology diffusion 59
3.5．Queueing systems in heavy traffic 60
4．Formulations of Stochastic Optimal Control Problems 62
4.1．Strong formulation 62
4.2．Weak formulation 64
5．Existence of Optimal Controls 65
5.1．A deterministic result 65
5.2．Existence under strong formulation 67
5.3．Existence under weak formulation 69
6．Reachable Sets of Stochastic Control Systems 75
6.1．Nonconvexity of the reachable sets 76
6.2．Noncloseness of the reachable sets 81
7．Other Stochastic Control Models 85
7.1．Random duration 85
7.2．Optimal stopping 86
7.3．Singular and impulse controls 86
7.4．Risk-sensitive controls 88
7.5．Ergodic controls 89
7.6．Partially observable systems 89
8．Historical Remarks 92
Chapter 3．Maximum Principle and Stochastic Hamiltonian Systems 101
1．Introduction 101
2．The Deterministic Case Revisited 102
3．Statement of the Stochastic Maximum Principle 113
3.1．Adjoint equations 115
3.2．The maximum principle and stochastic Hamiltonian systems 117
3.3．A worked-out example 120
4．A Proof of the Maximum Principle 123
4.1．A moment estimate 124
4.2．Taylor expansions 126
4.3．Duality analysis and completion of the proof 134
5．Sufficient Conditions of Optimality 137
6．Problems with Statc Constraints 141
6.1．Formulation of the problem and the maximum principle 141
6.2．Some preliminary lemmas 145
6.3．A proof of Theorem 6.1 149
7．Historical Remarks 153
Chapter 4．Dynamic Programming and HJB Equations 157
1．Introduction 157
2．The Detcrministic Case Revisited 158
3．The Stochastic Principle of Optimality and the HJB Equation 175
3.1．A stochastic framework for dynamic programming 175
3.2．Principle of optimality 180
3.3．The HJB equation 182
4．Other Properties of the Value Function 184
4.1．Continuous dependence on parameters 184
4.2．Semiconcavity 186
5．Viscosity Solutions 189
5.1．Definitions 189
5.2．Some properties 196
6．Uniqueness of Viscosity Solutions 198
6.1．A uniqueness theorem 198
6.2．Proofs of Lemmas 6.6 and 6.7 208
7．Historical Remarks 212
Chapter 5．The Relationship Between the Maximum Principle and Dynamic Programming 217
1．Introduction 217
2．Classical Hamilton-Jacobi Thcory 219
3．Relationship for Deterministic Systems 227
3.1．Adjoint variable and value function:Smooth case 229
3.2．Economic interpretation 231
3.3．Methods of characteristics and the Feynman-Kac formula 232
3.4．Adjoint variable and value function:Nonsmooth case 235
3.5．Vcrification theorems 241
4．Relationship for Stochastic Systems 247
4.1．Smooth case 250
4.2．Nonsmooth case:Differentials in the spatial variable 255
4.3．Nonsmooth case:Differentials in the time variable 263
5．Stochastic Vcrification Theorems 268
5.1．Smooth case 268
5.2．Nonsmooth case 269
6．Optimal Feedback Controls 275
7．Historical Remarks 278
Chapter 6．Linear Quadratic Optimal Control Problems 281
1．Introduction 281
2．The Deterministic LQ Problems Revisited 284
2.1．Formulation 284
2.2．A minimization problem of a quadratic functional 286
2.3．A linear Hamiltonian system 289
2.4．The Riccati equation and feedback optimal control 293
3．Formulation of Stochastic LQ Problems 300
3.1．Statement of the problems 300
3.2．Examples 301
4．Finiteness and Solvability 304
5．A Necessary Condition and a Hamiltonian System 308
6．Stochastic Riccati Equations 313
7．Global Solvability of Stochastic Riccati Equations 319
7.1．Existence:The standard case 320
7.2．Existence:The case C=0，S=0，and Q,G≥0 324
7.3．Existence:The one-dimensional case 329
8．A Mean-variance Portfolio Selection Problem 335
9．Historical Rcmarks 342
Chapter 7．Backward Stochastic Differential Equations 345
1．Introduction 345
2．Linear Backward Stochastic Diffrential Equations 347
3．Nonlinear Backward Stochastic Differential Equations 354
3.1．BSDEs in finite deterministic durations:Method of contraction mapping 354
3.2．BSDEs in random durations:Method of continuation 360
4．Feynman-Kac-Type Formulae 372
4.1．Representation via SDEs 372
4.2．Representation via BSDEs 377
5．Forward-Backward Stochastic Differential Equations 381
5.1．General formulation and nonsolvability 382
5.2．The four-step scheme,a heuristie dcrivation 383
5.3．Several solvablc classes of FBSDEs 387
6．Option Pricing Problems 392
6.1．European call options and the Black-Scholes formula 392
6.2．Other options 396
7．Historical Remarks 398
References 401
Index 433