Understanding Markov Chains

Examples and Applications

Author: Nicolas Privault

Publisher: Springer

ISBN: 9811306591

Category: Mathematics

Page: 372

View: 3268


This book provides an undergraduate-level introduction to discrete and continuous-time Markov chains and their applications, with a particular focus on the first step analysis technique and its applications to average hitting times and ruin probabilities. It also discusses classical topics such as recurrence and transience, stationary and limiting distributions, as well as branching processes. It first examines in detail two important examples (gambling processes and random walks) before presenting the general theory itself in the subsequent chapters. It also provides an introduction to discrete-time martingales and their relation to ruin probabilities and mean exit times, together with a chapter on spatial Poisson processes. The concepts presented are illustrated by examples, 138 exercises and 9 problems with their solutions.

Stochastic Processes

with Applications to Reliability Theory

Author: Toshio Nakagawa

Publisher: Springer Science & Business Media

ISBN: 9780857292742

Category: Technology & Engineering

Page: 254

View: 5169


Reliability theory is of fundamental importance for engineers and managers involved in the manufacture of high-quality products and the design of reliable systems. In order to make sense of the theory, however, and to apply it to real systems, an understanding of the basic stochastic processes is indispensable. As well as providing readers with useful reliability studies and applications, Stochastic Processes also gives a basic treatment of such stochastic processes as: the Poisson process, the renewal process, the Markov chain, the Markov process, and the Markov renewal process. Many examples are cited from reliability models to show the reader how to apply stochastic processes. Furthermore, Stochastic Processes gives a simple introduction to other stochastic processes such as the cumulative process, the Wiener process, the Brownian motion and reliability applications. Stochastic Processes is suitable for use as a reliability textbook by advanced undergraduate and graduate students. It is also of interest to researchers, engineers and managers who study or practise reliability and maintenance.

Reshaping college mathematics

a project of the Committee on the Undergraduate Program in Mathematics

Author: Lynn Arthur Steen,Mathematical Association of America. Committee on the Undergraduate Program in Mathematics

Publisher: Mathematical Assn of Amer


Category: Mathematics

Page: 125

View: 6957



Examples in Markov Decision Processes

Author: A B Piunovskiy

Publisher: World Scientific

ISBN: 1908979666

Category: Mathematics

Page: 308

View: 6360


This invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the theorems on Markov Decision Processes. Many of the examples are based upon examples published earlier in journal articles or textbooks while several other examples are new. The aim was to collect them together in one reference book which should be considered as a complement to existing monographs on Markov decision processes. The book is self-contained and unified in presentation. The main theoretical statements and constructions are provided, and particular examples can be read independently of others. Examples in Markov Decision Processes is an essential source of reference for mathematicians and all those who apply the optimal control theory to practical purposes. When studying or using mathematical methods, the researcher must understand what can happen if some of the conditions imposed in rigorous theorems are not satisfied. Many examples confirming the importance of such conditions were published in different journal articles which are often difficult to find. This book brings together examples based upon such sources, along with several new ones. In addition, it indicates the areas where Markov decision processes can be used. Active researchers can refer to this book on applicability of mathematical methods and theorems. It is also suitable reading for graduate and research students where they will better understand the theory. Contents:Finite-Horizon ModelsHomogeneous Infinite-Horizon Models: Expected Total LossHomogeneous Infinite-Horizon Models: Discounted LossHomogeneous Infinite-Horizon Models: Average Loss and Other Criteria Readership: Advanced undergraduates, graduates and research students in applied mathematics; experts in Markov decision processes. Keywords:Markov Decision Processes;Optimal Control;Stochastic ModelsKey Features:This book is the first attempt to bring together the most interesting examples in Markov decision processesA standard reference for professional mathematiciansComplementary to standard student textbooks (M Puterman's Markov Decision Processes (Wiley, 1994), O Hernandez-Lerma and J B Lasserre's Discrete-Time Markov Control Processes (Springer, 1996) and others)Relevant to active researchers from other areas who will understand how to apply the optimal control theory in their fieldReviews:“This remarkable and intriguing book is highly recommended. Some examples are aimed at undergraduate students, whilst others will be of interest to advanced undergraduates, graduates and research students in probability theory, optimal control and applied mathematics, looking for a better understanding of the theory; experts in Markov decision processes, professional or amateur researchers. Active researchers can refer to this book on applicability of mathematical methods and theorems.”The European Mathematical Society “The book presents many interesting topics and results. This is an important book that will be particularly useful to students and researchers on MDPs. I recommend it to anyone interested in the theory of MDPs.”Mathematical Reviews

Monte Carlo

Author: George Fishman

Publisher: Springer Science & Business Media

ISBN: 9780387945279

Category: Business & Economics

Page: 698

View: 1469


Apart from a thorough exploration of all the important concepts, this volume includes over 75 algorithms, ready for putting into practice. The book also contains numerous hands-on implementations of selected algorithms to demonstrate applications in realistic settings. Readers are assumed to have a sound understanding of calculus, introductory matrix analysis, and intermediate statistics, but otherwise the book is self-contained. Suitable for graduates and undergraduates in mathematics and engineering, in particular operations research, statistics, and computer science.

Genetic Algorithms: Principles and Perspectives

A Guide to GA Theory

Author: Colin R. Reeves,Jonathan E. Rowe

Publisher: Springer Science & Business Media

ISBN: 0306480506

Category: Computers

Page: 332

View: 7441


Genetic Algorithms: Principles and Perspectives: A Guide to GA Theory is a survey of some important theoretical contributions, many of which have been proposed and developed in the Foundations of Genetic Algorithms series of workshops. However, this theoretical work is still rather fragmented, and the authors believe that it is the right time to provide the field with a systematic presentation of the current state of theory in the form of a set of theoretical perspectives. The authors do this in the interest of providing students and researchers with a balanced foundational survey of some recent research on GAs. The scope of the book includes chapter-length discussions of Basic Principles, Schema Theory, "No Free Lunch", GAs and Markov Processes, Dynamical Systems Model, Statistical Mechanics Approximations, Predicting GA Performance, Landscapes and Test Problems.

Mathematical Epidemiology

Author: Fred Brauer,Pauline van den Driessche,J. Wu

Publisher: Springer Science & Business Media

ISBN: 3540789103

Category: Medical

Page: 414

View: 8110


Based on lecture notes of two summer schools with a mixed audience from mathematical sciences, epidemiology and public health, this volume offers a comprehensive introduction to basic ideas and techniques in modeling infectious diseases, for the comparison of strategies to plan for an anticipated epidemic or pandemic, and to deal with a disease outbreak in real time. It covers detailed case studies for diseases including pandemic influenza, West Nile virus, and childhood diseases. Models for other diseases including Severe Acute Respiratory Syndrome, fox rabies, and sexually transmitted infections are included as applications. Its chapters are coherent and complementary independent units. In order to accustom students to look at the current literature and to experience different perspectives, no attempt has been made to achieve united writing style or unified notation. Notes on some mathematical background (calculus, matrix algebra, differential equations, and probability) have been prepared and may be downloaded at the web site of the Centre for Disease Modeling (www.cdm.yorku.ca).