iqLearn: Interactive Q-Learning in R.

TitleiqLearn: Interactive Q-Learning in R.
Publication TypeJournal Article
Year of Publication2015
AuthorsLinn, Kristin A., Eric B. Laber, and Leonard A. Stefanski
JournalJ Stat Softw
Date Published2015 Feb

Chronic illness treatment strategies must adapt to the evolving health status of the patient receiving treatment. Data-driven dynamic treatment regimes can offer guidance for clinicians and intervention scientists on how to treat patients over time in order to bring about the most favorable clinical outcome on average. Methods for estimating optimal dynamic treatment regimes, such as Q-learning, typically require modeling nonsmooth, nonmonotone transformations of data. Thus, building well-fitting models can be challenging and in some cases may result in a poor estimate of the optimal treatment regime. Interactive Q-learning (IQ-learning) is an alternative to Q-learning that only requires modeling smooth, monotone transformations of the data. The R package provides functions for implementing both the IQ-learning and Q-learning algorithms. We demonstrate how to estimate a two-stage optimal treatment policy with using a generated data set bmiData which mimics a two-stage randomized body mass index reduction trial with binary treatments at each stage.

Alternate JournalJ Stat Softw
Original PublicationiqLearn: Interactive Q-Learning in R.
PubMed ID26900385
PubMed Central IDPMC4760113
Grant ListP01 CA142538 / CA / NCI NIH HHS / United States