Experimentation in Political Science
IPR fellow leads conference for topic's first handbook

Fall, 2009, Volume 31, Number 1

Skip Lupia of Michigan (l.) talks to James Druckman about the forthcoming book.

On May 28–29, more than 100 political scientists and graduate students from across the country gathered at Northwestern University to discuss and critique papers that will be included as chapters in the first “Handbook of Experimental Political Science.”

IPR’s James Druckman was the conference organizer, and IPR was a co-sponsor. As one of the handbook’s co-editors and the first political scientist named as a lead editor of Public Opinion Quarterly, he is leading the charge to improve methodology in the field.

The participant list read like a who’s who of pioneers of experimental methods in the field—Shanto Iyengar and Paul Sniderman of Stanford, Alan Gerber and Donald Green of Yale, Diana Mutz of the University of Pennsylvania, and Arthur “Skip” Lupia of the University of Michigan, to name a few.

Druckman pointed out why Northwestern made such a great intellectual venue for the conference. “NU is where the father of quasi-experimentation, Donald Campbell, spent most of his career,” he said. He also noted the contributions of his father, Daniel Druckman of George Mason University, who played a role in the early development of experiments at Northwestern working with Campbell and Harold Guetzkow.

Fast forward several decades, IPR is still a vibrant intellectual center for research and training on experimental methods. Social psychologist Thomas D. Cook, Campbell’s longtime collaborator and a leading authority in his own right, continues to unravel questions of quasi-experimentation, particularly in the field of education. The Institute houses the Center for Improving Methods for Quantitative Policy Research, or Q-Center, led by statistician and education researcher Larry Hedges, an international authority on meta-analysis. Sociologist Jeremy Freese is leading a revamp of the online project Time-sharing Experiments for the Social Sciences, or TESS, that Mutz and Lupia launched in 2002 and for which Druckman is an associate PI (see p. 9).

“In fact, more than eight Northwestern alumni or faculty are involved in the volume,” Druckman said. Many at the conference marveled at how the trend in using experimentation has shifted. Today, it has become more acceptable—even “fashionable”—for papers to feature experimental methods. This stands in stark contrast to years, in the not too distant past, where reviewers often dismissed journal submissions outright for inclusion of such methods.

“We are no longer the lunatic fringe,” noted Don Kinder of the University of Michigan.

Still, many nagging questions about experiments remain. Thus, conference participants held a two-day “running conversation” on some of the pressing issues of experimental methods, covering not only the nuts and bolts of running
experiments—limitations of internal and external validity, causality, design and analytic challenges—but also specific considerations for experiments measuring particular phenomena, such as vote choice, elite bargaining, and political attitudes.

Kinder wrapped up the exhaustive two-day overview of political science experimentation with some general comments for the chapter authors.

“For any particular area of research, the more experiments included, the higher the chance for confusion on the reader’s part,” Kinder pointed out. “Literature reviews that try to cover everything are dangerous.”

Kinder suggested that the researchers concentrate on a few exemplary experiments that reveal unexpected findings. He also recommended that they try to be “embracingly ecumenical in spirit” and expand their discussions to include a variety of different methods, including field experiments, lab experiments, and observational studies.

“What can we learn from various forms of social science research?” Kinder queried.

Druckman was pleased with the conference proceedings. “We more than fulfilled the basic goals set out for the conference—to situate experiments in the broader discipline, present important methodological issues, and review the contributions of experimental research,” he said. “We are well set to meet the challenge of combining everything into a unified volume.”

Druckman is Payson S. Wild Professor in Political Science and an IPR fellow. “The Handbook of Experimental Political Science” will be published by Cambridge University Press.