Introduction to Markov Decision Processes
By Martin L. Puterman and Timothy C. Y. Chan
We are currently writing a book that will be an accessible and up to date introduction to Markov decision processes. Our target audience includes undergraduate students, graduate students and self-directed learners looking for a structured introduction that covers foundations, algorithms, and applications.
Drafts of certain chapters will be posted here as they become available. We welcome all feedback and suggestions.
This material will be published by Cambridge University Press as Introduction to Markov Decision Processes by Martin L. Puterman and Timothy C. Y. Chan. This pre-publication version is free to view and download for personal use only. Not for redistribution, resale, or use in derivative works. ©Martin L. Puterman and Timothy C. Y. Chan, 2021.