Simulation-Based Algorithms for Markov Decision Processes by Hyeong Soo ChangSimulation-Based Algorithms for Markov Decision Processes by Hyeong Soo Chang

Simulation-Based Algorithms for Markov Decision Processes

byHyeong Soo Chang, Jiaqiao Hu, Michael C. Fu

Hardcover | March 20, 2013

Pricing and Purchase Info

$196.72 online 
$234.95 list price save 16%
Earn 984 plum® points

Prices and offers may vary in store


In stock online

Ships free on orders over $25

Not available in stores


Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. Many real-world problems modeled by MDPs have huge state and/or action spaces, giving an opening to the curse of dimensionality and so making practical solution of the resulting models intractable. In other cases, the system of interest is too complex to allow explicit specification of some of the MDP model parameters, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based algorithms have been developed to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function. Specific approaches include adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search.
This substantially enlarged new edition reflects the latest developments in novel algorithms and their underpinning theories, and presents an updated account of the topics that have emerged since the publication of the first edition. Includes:
innovative material on MDPs, both in constrained settings and with uncertain transition properties;
game-theoretic method for solving MDPs;
theories for developing roll-out based algorithms; and
details of approximation stochastic annealing, a population-based on-line simulation-based algorithm.
The self-contained approach of this book will appeal not only to researchers in MDPs, stochastic modeling, and control, and simulation but will be a valuable source of tuition and reference for students of control and operations research.
Hyeong Soo Chang (SM'07 of the IEEE, Member of INFORMS) received the B.S. and M.S. degrees in electrical engineering and the Ph.D. degree in electrical and computer engineering, all from Purdue University,West Lafayette, IN, in 1994, 1996, and 2001, respectively. Since 2003, he has been with the Department of Computer Science and Engin...
Title:Simulation-Based Algorithms for Markov Decision ProcessesFormat:HardcoverDimensions:229 pages, 23.5 × 15.5 × 0.01 inPublished:March 20, 2013Publisher:Springer-Verlag/Sci-Tech/TradeLanguage:English

The following ISBNs are associated with this title:

ISBN - 10:144715021X

ISBN - 13:9781447150213

Look for similar items by category:


Table of Contents

Markov Decision Processes.- Multi-stage Adaptive Sampling Algorithms.- Population-based Evolutionary Approaches.- Model Reference Adaptive Search.- On-line Control Methods via Simulation.- Game-theoretic Methods via Simulation.

Editorial Reviews

From the book reviews:"The book consists of five chapters. . This well-written book is addressed to researchers in MDPs and applied modeling with an interests in numerical computations, but the book is also accessible to graduate students in operation research, computer science, and economics. The authors gives many pseudocodes of algorithms, numerical examples, algorithms convergence analysis and bibliographical notes that can be very helpful for readers to understand the ideas presented in the book and to perform experiments on their own." (Wieslaw Kotarski, zbMATH, Vol. 1293, 2014)