Multi-Objective Markov Decision Processes for Data-Driven Decision Support.

TitleMulti-Objective Markov Decision Processes for Data-Driven Decision Support.
Publication TypeJournal Article
Year of Publication2016
AuthorsLizotte, Daniel J., and Eric B. Laber
JournalJ Mach Learn Res
Date Published2016

We present new methodology based on Multi-Objective Markov Decision Processes for developing sequential decision support systems from data. Our approach uses sequential decision-making data to provide support that is useful to many different decision-makers, each with different, potentially time-varying preference. To accomplish this, we develop an extension of fitted- iteration for multiple objectives that computes policies for all scalarization functions, i.e. preference functions, simultaneously from continuous-state, finite-horizon data. We identify and address several conceptual and computational challenges along the way, and we introduce a new solution concept that is appropriate when different actions have similar expected outcomes. Finally, we demonstrate an application of our method using data from the Clinical Antipsychotic Trials of Intervention Effectiveness and show that our approach offers decision-makers increased choice by a larger class of optimal policies.

Alternate JournalJ Mach Learn Res
Original PublicationMulti-objective Markov decision processes for data-driven decision support.
PubMed ID28018133
PubMed Central IDPMC5179144
Grant ListP01 CA142538 / CA / NCI NIH HHS / United States
R01 AA023187 / AA / NIAAA NIH HHS / United States