Many randomized controlled trials (RCTs) to evaluate the effectiveness of complex care interventions are underpowered due to low participant engagement. As a result, the effect of the intervention is diluted in standard RCT analyses, and some programs may be falsely deemed ineffective despite a potential positive impact on study subpopulations. The Distillation Method, a novel approach for evaluating RCTs developed by researchers at Kaiser Permanente, can help address this challenge and may generate new evidence on the effectiveness of complex care programs that aim to reach patients with low or variable engagement in interventions.
This Better Care Playbook webinar, made possible through support from the Seven Foundation Collaborative, examined the challenges associated with RCTs in complex care and how the Distillation Method can help address these challenges. Researchers from the Camden Coalition of Healthcare Providers spoke about their experience applying the Distillation Method to the 2019 randomized controlled trial of their hotspotting intervention.
Agenda
I. Welcome and Introduction
Speaker: Emma Opthof, MPH, Communications Officer, Center for Health Care Strategies (CHCS)
II. Understanding Challenges in Randomized Controlled Trials and How the Distillation Method Works
Speakers: Anna Davis, PhD, Research Scientist, and John Adams, PhD, Principal Senior Statistician and Senior Director, Kaiser Permanente Research and Quality Measurement
A. Davis and J. Adams discussed the advantages of randomized experimental designs for evaluating complex care interventions and the analytic challenges that arise when intervention dilution occurs. They then presented an overview of the Distillation Method, a new strategy for analyzing treatment effects in RCTs where the intervention was diluted. Finally, they shared a case study of an RCT that was compromised by low uptake in the intervention group and present the results of applying this method to the case study.
III. An Application of the Distillation Method: The Camden Coalition’s Hotspotting Trial
Speakers: Dawn Wiest, PhD, Director of Research and Evaluation, and Qiang Yang, PhD, Data Scientist, Camden Coalition of Healthcare Providers
D. Wiest and Q. Yang discussed the rationale for applying the Distillation Method to the data from the Camden Coalition’s 2019 randomized trial, their approach to applying the method to the trial data, and early insight based on their use of the method.
IV. Moderated Q&A
Moderator: Logan Kelly, MPH, Senior Program Officer, CHCS