Bookstore

This textbook addresses postgraduate students in applied mathematics, probability, and statistics, as well as computer scientists, biologists, physicists and economists, who are seeking a rigorous introduction to applied stochastic processes. Pursuing a pedagogic approach, the content follows a path of increasing complexity, from the simplest random sequences to the advanced stochastic processes. Illustrations are provided from many applied fields, together with connections to ergodic theory, information theory, reliability and insurance. The main content is also complemented by a wealth of examples and exercises with solutions.

Applied Probability: From Random Sequences to Stochastic Processes

Notation iiiPreface ix1 Independent Random Sequences 11.1 Denumerable Sequences . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Sequences of Events . . . . . . . . . . . . . . . . . . . . . 71.1.2 Independence . . . . . . . . . . . . . . . . . . . . . . . . . 91.2 Analytic Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.2.1 Generating Functions . . . . . . . . . . . . . . . . . . . . 121.2.2 Characteristic Functions . . . . . . . . . . . . . . . . . . . 131.2.3 Laplace Transforms . . . . . . . . . . . . . . . . . . . . . 151.2.4 Moment Generating Functions and Cram´er Transforms . 171.2.5 From Entropy to Entropy Rate . . . . . . . . . . . . . . . 191.3 Sums and Random Sums . . . . . . . . . . . . . . . . . . . . . . . 231.3.1 Sums of Independent Variables . . . . . . . . . . . . . . . 231.3.2 Random Sums . . . . . . . . . . . . . . . . . . . . . . . . 271.3.3 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . 291.4 Convergence of Random Sequences . . . . . . . . . . . . . . . . . 301.4.1 Different Types of Convergence . . . . . . . . . . . . . . . 301.4.2 Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . 331.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 Conditioning and Martingales 512.1 Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512.1.1 Conditioning with Respect to an Event . . . . . . . . . . 522.1.2 Conditional Probabilities . . . . . . . . . . . . . . . . . . 532.1.3 Conditional Distributions . . . . . . . . . . . . . . . . . . 562.1.4 Conditional Expectation . . . . . . . . . . . . . . . . . . . 572.1.5 Conditioning and Independence . . . . . . . . . . . . . . . 632.1.6 Practical Determination . . . . . . . . . . . . . . . . . . . 652.2 The Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . 682.3 Stopping Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712.4 Discrete-Time Martingales . . . . . . . . . . . . . . . . . . . . . . 732.4.1 Definitions and Properties . . . . . . . . . . . . . . . . . . 732.4.2 Classical Inequalities . . . . . . . . . . . . . . . . . . . . . 782.4.3 Martingales and Stopping Times . . . . . . . . . . . . . . 812.4.4 Convergence of Martingales . . . . . . . . . . . . . . . . . 842.4.5 Square Integrable Martingales . . . . . . . . . . . . . . . . 862.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 893 Markov Chains 993.1 General Properties . . . . . . . . . . . . . . . . . . . . . . . . . . 993.1.1 Transition Functions with Examples . . . . . . . . . . . . 993.1.2 Martingales and Markov Chains . . . . . . . . . . . . . . 1073.1.3 Stopping Times and Markov Chains . . . . . . . . . . . . 1093.2 Classification of States . . . . . . . . . . . . . . . . . . . . . . . . 1113.3 Stationary Distribution and Asymptotic Behavior . . . . . . . . . 1163.4 Periodic Markov chains . . . . . . . . . . . . . . . . . . . . . . . 1233.5 Finite Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . 1273.5.1 Specific Properties . . . . . . . . . . . . . . . . . . . . . . 1273.5.2 Application to Reliability . . . . . . . . . . . . . . . . . . 1323.6 Branching Processes . . . . . . . . . . . . . . . . . . . . . . . . . 1353.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1394 Continuous Time Stochastic Processes 1534.1 General Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1534.2 Stationarity and Ergodicity . . . . . . . . . . . . . . . . . . . . . 1604.3 Processes with Independent Increments . . . . . . . . . . . . . . 1664.4 Point Processes on the Line . . . . . . . . . . . . . . . . . . . . . 1684.4.1 Basics on General Point Processes . . . . . . . . . . . . . 1694.4.2 Renewal Processes . . . . . . . . . . . . . . . . . . . . . . 1714.4.3 Poisson Processes . . . . . . . . . . . . . . . . . . . . . . . 1754.4.4 Asymptotic Results for Renewal Processes . . . . . . . . . 1774.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1815 Markov and Semi-Markov Processes 1895.1 Jump Markov Processes . . . . . . . . . . . . . . . . . . . . . . . 1895.1.1 Markov Processes . . . . . . . . . . . . . . . . . . . . . . . 1895.1.2 Transition Functions . . . . . . . . . . . . . . . . . . . . . 1925.1.3 Infinitesimal Generators and Kolmogorov's Equations . . 1955.1.4 Embedded Chains and Classification of States . . . . . . . 1975.1.5 Stationary Distribution and Asymptotic Behavior . . . . 2035.2 Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . 2075.2.1 Markov Renewal Processes . . . . . . . . . . . . . . . . . 2075.2.2 Classification of States and Asymptotic Behavior . . . . . 2105.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212Further Reading 225