Proper Inference for Value Function in High-Dimensional Q-Learning for Dynamic Treatment Regimes.

TitleProper Inference for Value Function in High-Dimensional Q-Learning for Dynamic Treatment Regimes.
Publication TypeJournal Article
Year of Publication2019
AuthorsZhu, Wensheng, Donglin Zeng, and Rui Song
JournalJ Am Stat Assoc
Date Published2019

Dynamic treatment regimes are a set of decision rules and each treatment decision is tailored over time according to patients' responses to previous treatments as well as covariate history. There is a growing interest in development of correct statistical inference for optimal dynamic treatment regimes to handle the challenges of non-regularity problems in the presence of non-respondents who have zero-treatment effects, especially when the dimension of the tailoring variables is high. In this paper, we propose a high-dimensional Q-learning (HQ-learning) to facilitate the inference of optimal values and parameters. The proposed method allows us to simultaneously estimate the optimal dynamic treatment regimes and select the important variables that truly contribute to the individual reward. At the same time, hard thresholding is introduced in the method to eliminate the effects of the non-respondents. The asymptotic properties for the parameter estimators as well as the estimated optimal value function are then established by adjusting the bias due to thresholding. Both simulation studies and real data analysis demonstrate satisfactory performance for obtaining the proper inference for the value function for the optimal dynamic treatment regimes.

Alternate JournalJ Am Stat Assoc
Original PublicationProper inference for value function in high-dimensional Q-learning for dynamic treatment regimes.
PubMed ID31929664
PubMed Central IDPMC6953729
Grant ListP01 CA142538 / CA / NCI NIH HHS / United States
R01 GM124104 / GM / NIGMS NIH HHS / United States
R01 NS073671 / NS / NINDS NIH HHS / United States