Introduction to Markov Decision Processes Markov Decision Processes A (homogeneous, discrete, observable) Markov decision process (MDP) is a stochastic system characterized by a 5-tuple M= X,A,A,p,g, where: •X is a countable set of discrete states, •A is a countable set of control actions, •A:X →P(A)is an action constraint function,

Target water bottles contigoIn the first few years of an ongoing survey of applications of Markov decision processes where the results have been implemented or have had some influence on decisions, few applications have been identified where the results have been implemented but there appears to be an increasing effort to model many phenomena as Markov decision processes.

Two sample applications of MDP and FPI Markov decision processes (MDPs) and policy iteration are powerful tools to solve dynamic decision problems. Here we give two application examples: 1. Dynamic RWA problem in Wavelength Routed Optical Networks