Instructor:  Paul J. Smith, Statistics Program

Office:  MTH 4404
Telephone:  (301) 405-5104
E-mail:  pjs@math.umd.edu

Textbook:   Karlin, S. and Taylor, H. (1975).    A First Course in Stochastic Processes.   New York: Academic Press.

Prerequisite:  STAT 410.

Stochastic processes are random phenomena that evolve over time or space.  Some examples are the number of radioactive particles emitted by a sample of radium during the interval (0,t], the number of cells in a malignant tumor at age t, the position of a small particle suspended in a fluid at time t, the number of bacteria in a volume v of water, or the number of failures occurring in a repairable system in the last t years.  Mathematically, we have a collection of random variables, indexed by points in time or space.  Equivalently, we may regard a stochastic process as a random function whose domain is the time axis or a spatial region and whose range, the state space of the process, may be an arbitrary set.  The goal is to be able to study the probability behavior of such processes, particularly how they evolve probabilistically over time or space.

Stochastic processes can be classified according to their index, whether continuous or discrete, and according to their state space, which also may be continuous or discrete.  Most important stochastic process models are indexed by time, whether discrete or continuous, and these are important tools in biology, operations research, computer science, finance and physics.  Stochastic processes indexed by spatial co-ordinates are becoming increasingly important in applied fields such as meteorology, geophysics and pattern recognition.

STAT 650 will study stochastic processes from the modeling point of view.  The course will consider basic models in order to convey the flavor of the mathematics and will cover classes of models most important in applications.  The treatment of discrete state space processes will be more rigorous than the treatment of  continuous state space processes because of analytic difficulties, but the most important continuous time, continuous state process, Brownian motion, will be presented carefully.

The material in STAT 650 is included in the syllabus for the Written Examinations in Probability, both at the Master's and Ph.D. levels.

STAT 650 Topics:

• Markov Chains in Discrete Time: Transition probabilities, classification of states, recurrence, limiting distributions, examples and applications, including unrestricted random walk and random walk with absorbing or reflecting barriers.
• Markov Chains in Continuous Time:  The Poisson process, birth and death processes, forward and backward differential equations, finite state chains, absorbing chains.
• Renewal Theory:  Examples and connections with Markov processes, renewal equation, the Renewal Theorem, generalizations and applications, superposition.
• Martingales:  Random walks and generalizations, martingale property, submartingales and supermartingales, optional sampling, convergence of martingales.
• Brownian motion:  Brownian motion as a generalization of random walk, joint probabilities, path properties, calculation of distributions of functionals, applications to statistics and finance.
• Additional topics:  If time permits, some material concerning branching processes or continuous state, discrete time stationary processes will be presented.

• Midterms: Wednesday, February 25 and Wednesday, April 7.  These will be 45 minute in-class tests.
• Final:  Monday, May 17, 4-6 p.m.  Note the change in time.
• Homework:  Frequent problem sets from Karlin and Taylor will be assigned.  Students should not work together on homework assignments.  Click here for homework assignments.
• Grades:   Final course grades will be based on a weighted average, tentatively with weights of 30% for the final exam, 20% for each midterm and 30% for homework.

Bhattacharya, R. N. and Waymire, E. C.  (1990).  Stochastic Processes with Applications.  New York: J. Wiley.

Cinlar, E.  (1975).  Introduction to Stochastic Processes.  Englewood Cliffs, NJ: Prentice-Hall.

Durett, R.   (1999).  Essentials of Stochastic Processes.  New York: Springer.

Resnick, S. I. (1992).  Adventures in Stochastic Processes.  Boston: Birkhaeuser.

Ross, S. M. (1996).  Stochastic Processes.   New York: J. Wiley.