Research

Stable Policy Learning with Harvey Barnhard and Rahul Singh [Draft] [Abstract]

In evidence-based policymaking, typically one experimental sample is observed, then a learned policy recommendation is implemented at scale. Policies learned from the experimental data can perform well in expected welfare, yet random sampling in the experiment can produce recommendations with poor welfare outcomes. In this paper, we ask: how should policy learning algorithms balance expected welfare against sampling risk? Our main contribution is to show that algorithmic stability plays a central role in characterizing and navigating the tradeoff. Intuitively, if a policy learning algorithm's recommendation remains stable when one experimental unit is replaced, then that algorithm has limited sampling risk. We propose a method for policy learning called policy-vote bagging, which learns treatment decisions on many subsamples then averages their votes into treatment probabilities. Relative to using one subsample, averaging across subsamples preserves expected welfare and improves expected utility for a risk-averse researcher. We derive sharp bounds linking estimation accuracy, subsample size, and welfare variation, including an exact guarantee under CARA utility.

Status: Collecting Feedback

Learning Where to Look: Delaunay Matching for Discovery and Data Collection [Draft] [Abstract]

A researcher observes a population in the status quo and wants to learn from a small, possibly selected sample exposed to an intervention, its effect on the rest of the population. How should they combine these sources of information? And who should receive the intervention next to improve on estimation the most? I focus on convex matching estimators and answer both questions by adopting a statistical decision theoretic perspective and drawing new connections with results from computational geometry. First, I show that interpolation over the Delaunay simplex containing a target point minimizes worst-case bias. Second, I provide robust confidence intervals and show that inference-optimal weighting rules lay on a frontier that connects the proposed estimator to Synthetic Controls. Finally, I characterize the minimax refinement problem and show that the Delaunay triangulation of the existing data provides sufficient information to solve it numerically and restricts the space where to look for optimal additions. In a semi-synthetic application in development economics, I show that Delaunay matching achieves a lower worst-case bias compared to commonly used estimators in practice (e.g. around 7 times smaller than synthetic controls) and that the minimax collection plan decreases it by around 90% within 20 additions.

Status: Collecting Feedback

Producing Policy Recommendations: from Statistical Decision Theory to Empirical Practice [Draft] [Abstract]

Applied research in economics is intrinsically motivated by broad normative objectives. However, it is not obvious how a researcher should direct their efforts to produce evidence toward such objectives. This paper reviews recent theoretical developments on research design for policy choice and provides new tools applied researchers can use to guide their design choices and communicate their policy recommendations. First, I focus on theoretical contributions in econometrics and provide a general framework that nests all the contexts and results reviewed using a coherent notation and narrative. Then, I present two diagrams applied researchers can use to navigate the theoretical literature starting from concrete scenarios to make thoughtful design choices. Finally, I introduce a new R package that produces one table and two figures applied researchers can plug in their `policy implications' section to provide evidence on the performance of different policy recommendations coming out of their study. The use of such tools is illustrated with an example in development economics.

Status: Collecting Feedback

Better Measurement or Larger Samples? Data Collection for Policy Learning with Unobserved Heterogeneity [Draft] [Slides] [Abstract]

Empirical research shows that individuals' responses to treatments vary along latent characteristics, such as innate ability or motivation. Therefore, a policymaker seeking to maximize welfare may consider designing policies based on observed characteristics and estimated latent traits. I characterize how the estimates' precision affects the worst-case performance of policies deriving rate-sharp regret bounds for assignment rules that include or exclude them, highlighting new trade-offs with the policy space complexity. I then study how a policymaker can solve such trade-offs by designing tailored data collections, and derive the minimax optimal collection plan. In an empirical application in development economics, I show that including a proxy for entrepreneurs' business skills in targeting cash transfers increases welfare by 5%, and halves the probability of generating welfare losses. Moreover, I estimate the optimal allocation of resources between improving the precision of the proxy via repeated measurements, and increasing sample size.

Winner of Unicredit Young Economist Best Presentation Award
Status: Submitted

BallotBot: Can AI Chatbots Lower Voter-Information Barriers? with Elliott Ash and Sergio Galletta [Draft] [VoxEU Column] [Abstract]

This study examines the potential of AI-powered chatbots to increase engagement with political information. We develop and evaluate BallotBot, an AI chatbot with access to official voter guide information from the November 2024 referendums in California. In a pre-registered three-wave survey experiment in the weeks around election day, participants (California voters) were randomly assigned to use either BallotBot or a traditional digital voter guide to answer questions about ballot initiatives. We find that BallotBot access lowered the perceived cost of acquiring information for less-informed participants, fostered greater engagement with political information, while improving participants' ability to correctly answer in-depth questions about ballot measures.

Status: R&R at The Economic Journal (2nd Round)