Speaker

Dr. Elizabeth Tipton\ Professor of Statistics and Data Science\ Northwestern University

Dr. Tipton is a Professor of Statistics and Data Science, the Co-Director of the Statistics for Evidence-Based Policy and Practice (STEPP) Center, and a Faculty Fellow in the Institute for Policy Research at Northwestern University. Dr. Tipton’s research focuses on the design and analysis of field experiments, with a particular focus on issues of external validity and generalizability in experiments; meta-analysis, particularly of dependent effect sizes; and the use of (cluster) robust variance estimation. Dr. Tipton earned her PhD in Statistics from Northwestern University in 2011. In 2005, she earned an M.A. in Sociology from the University of Chicago and in 2001 a B.A. in Mathematics from Transylvania University in Lexington, Kentucky. Prior to returning to Northwestern University, she was a member of the faculty at Teachers College, Columbia University for seven years.

Title

Generalizability and heterogeneity: Designing your study when treatment effects vary

Abstract

Randomized trials are now common in medicine, the social sciences, and in policy-related research. The primary goal of a randomized trial is to estimate the average treatment effect, which averages over unit-specific treatment impacts that might be quite heterogenous. Often the results of such a trial are to be used for making policy or practice decisions in a target population; this, paired with the fact that most such trials take place in samples of convenience, has led to an increased interest in methods for generalizing causal effects. This has included new methods for estimation of average treatment effects that take into account population data, as well as methods for designing sampling and recruitment strategies for such studies. But if treatment effects actually vary, often researchers seek to understand if such heterogeneity can be predicted or explained by unit characteristics. In this talk, I focus on this sampling problem in the face of heterogeneity. I show that methods for optimal designs found in response surface models can be useful, too, in designing sampling plans that result in increased power and precision for these moderator effects. I situate this in an example based on an evaluation of a school-based reading program.