Recent Performance Measurement and Rewards Research

Research Performance in Government and Nonprofits

Differences in Mixed Ownership Forms

Weisbrod has examined the reasons for expecting that in mixed industries such as hospitals, nursing homes, higher education, and book publishing, for-profit, public, religious nonprofit, and secular nonprofit organizations act the same in some dimensions—in markets where output is profitable—but differently in markets where outputs are unprofitable but socially desirable. His new working paper, “How Mixed Industries Exist: Modeling Output Choices in For-Profit, Public, Religious Nonprofit, and Secular Nonprofit Organizations, with Application to Hospitals,” uses a two-good model of organizational behavior to explain how ownership affects output choice, why nonprofits and for-profits choose outputs that are both alike and different, and how these forms can coexist despite differential constraints. Testing is for the mixed hospital industry. An appendix estimates the ability of differential CEO incentives—base salary, bonus, and their total—to explain the output differentials.

Dysfunction in Statistical Evaluations for Nonprofits

IPR statistician Bruce Spencer is starting a new project inspired by the IPR Performance Measurement Seminar Series. Many of the seminars reveal that performance measurement can lead to corrupting behavior in nonprofit organizations—and likely in many for-profit organizations as well. When performance is statistically assessed, efforts within the organization shift to enhance the statistical performance and diminish what is not being statistically assessed. For example, schools under No Child Left Behind place more emphasis on tested subjects than those that are not. In healthcare organizations, this might lead to resource-allocation shifts to improve medical report cards, even when such improvement comes at the cost of more direct healthcare priorities of the organization. Working with Dranove, Spencer plans to develop statistical models to analyze the interaction of the cost and quality of performance measurement, reward structures, and agents’ costs of modifying their performance under optimal behavior.

Learning How to Live with HIV

An HIV diagnosis is no longer a death sentence, with individuals now living for decades thanks to life-saving drug regimens. Yet many HIV-positive individuals grapple with how to transition from believing and behaving as though they have been sentenced to death to interpreting and coping with HIV as a chronic but manageable illness. As part of her Health, Hardship, and Renewal Study, IPR sociologist and African American studies researcher Celeste Watkins-Hayes interviewed 30 HIV-positive African American women between 2005 and 2008. In this project’s first published study, she and her coauthors reveal how nonprofit and government institutions, or “framing institutions,” play a critical role in helping these women to cope. These organizations provide conceptual frameworks for understanding what it means to have HIV, language to talk about their condition, and resources to begin restructuring their lives in the wake of a diagnosis. The authors also show how such institutions help the women to renegotiate their self-conceptions as black women after receiving another stigmatizing social marker. The study was written with Jean Beaman, an IPR postdoctoral fellow, and LaShawnDa Pittman-Gay, a postdoctoral fellow at Georgia State University and a former IPR graduate research assistant. It was published in Social Science & Medicine. The project receives support from the R.W. Johnson Foundation and National Science Foundation.

Healthcare Markets and Regulation

Individual Health Plan Exchanges

According to provisions of the Affordable Healthcare Act, states are supposed to have a “health insurance exchange” in place by January 1, 2014. Through these exchanges, individuals and employees of small firms will be able to select among plans deemed “qualified” by the federal government, to easily compare the benefits offered by these plans, and to determine how much the federal government will subsidize their premiums and cost-sharing. While much has been written about the costs associated with the exchanges, some of the benefits have been poorly quantified—including the “value of choice.” What would such choice mean in a nation where 80 percent of firms offering health insurance give workers only one “choice” of plans? While limited choice can spur cost savings and create better risk pools, it also prevents employees from selecting plans that best suit their needs. In a forthcoming article in the American Economic Journal: Economic Policy, healthcare economist and IPR associate Leemore Dafny, with economists Katherine Ho of Columbia University and Mauricio Varela of the University of Arizona, uses proprietary data on plan offerings and enrollment for more than 800 large employers to compare the choices offered by employers with the choices their employees would make on their own. Their main finding is that gains from offering more plan choices seem to outweigh potential premium increases associated with a transition from large-group to individual pricing.

Transparency and Medical Pricing

In the New England Journal of Medicine, Dafny and Harvard University economist David Cutler take on one of the few areas to enjoy bipartisan congressional support in the healthcare debates, price transparency. Around two-thirds of states have laws mandating full disclosure of pricing (via Sunshine laws) with the idea being they can help patients compare prices and make better care choices. But Dafny and Cutler argue that “the wrong kind of transparency could actually harm patients, rather than help them.” They point to a discouraging history of use of the “most favored nation” (MFN) clause in healthcare, which works to limit competition. Imagine a case where under an MFN contract, a hospital agrees not to charge a lower price to any other buyer. Thus, after signing an MFN clause with Insurer B, it could not offer a lower price to Insurer A, without offering the same to Insurer B. So prices to Insurer A would tend to rise. The researchers provide examples of where MFN and transparency have raised prices in healthcare and other industries such as ready-mix concrete. While the two laud efforts to raise consumer cost consciousness and advocate for “copayment transparency”—they worry that a pricing menu of negotiated payer-provider prices could raise costs instead of lowering them. 

Healthcare Markets, Regulators, and Certifiers

Discussing economist Kenneth Arrow’s seminal analysis of how uncertainty and lack of consumer information affect the market, healthcare economist and IPR associate David Dranove traces the evolution of research in the field. Better methodology now confirms that competition has driven costs down. He also tackles the performance of competitive healthcare markets, regulatory approaches to containing prices, the use and effectiveness of healthcare report cards as a quality assurance mechanism, and pay-for-performance schemes. Dranove, Walter McNerney Professor of Health Industry Management, argues that many misunderstood Arrow’s analysis and proposes that fundamental economic principals can successfully be applied in the field as long as one accounts for particular insurance and physician characteristics. He enumerates the research questions revolving around how to enhance efficiency—i.e., competitive versus regulated systems, compensation systems to address conflicts of interest, and third-party certification. The chapter was included in the Handbook of Health Economics, Vol. 2.

Hospital Infection Reporting and Rates

Healthcare-associated infections (HAIs) are often preventable, yet they kill about 100,000 people each year. In response, many states and the federal government require hospitals to publicly report their infection rates for one important infection type—central line-associated bloodstream infection (CLABSI). There is little evidence, however, on whether public HAI reporting, affects infection rates. Bernard Black, an IPR associate and Chabraja Professor of Law and Finance, and his colleagues are studying this question. In a case study of Pennsylvania, they find evidence that hospitals respond to public HAI reporting both by reducing infection rates and through time-inconsistent reporting (“gaming”). CLABSI rates in both Pennsylvania and control states, estimated from inpatient data (“inpatient rates”), rise at similar rates during the pre-reporting period (1998 to 2003). During the 2004–08 reporting period, Pennsylvania rates dropped by 15 percent, compared to control states. Hospitals did not report inpatient rates and had no direct incentive to manipulate them. Publicly reported CLABSI rates fell in Pennsylvania by 41 percent over 2005–07, much faster than inpatient rates. This difference suggests gaming of the public reports.

Risk, Innovation, and Technology

Regulating Risk in Nanotechnology

As development of nanotechnology zooms ahead, research on its effects on health and the environment lags. The Nanotechnology Challenge (Cambridge University Press, 2011), edited by Northwestern law professor and IPR associate David Dana, attempts to address this gap. The book offers views by legal scholars and scientists on how to assess the potential unknowns and risks of nanotechnology. It also examines the public perception of these risks and its influence on regulatory trends. Research in a chapter by Daniel Diermeier, an IPR associate and IBM Professor of Regulation and Competitive Practice, suggests that the public is largely ignorant about nanotechnology but nevertheless holds increasingly firm attitudes about it. IPR political scientist James Druckman, Payson S. Wild Professor of Political Science, and co-author Toby Bolsen of Georgia State University examine how scientific evidence links attitudes to behaviors using an experiment to survey perceptions of carbon nanotubes. Dana, who is Stanford Clinton Sr. and Zylpha Kilbride Clinton Research Professor, writes about adaptive definitions and environmental assurance bonds in nanotechnology regulation.

Accountability Measures for Service Industries

Consequences of Pay for Performance

No Child Left Behind (NCLB) legislation represents the nation’s most consequential education reform in decades—and its reach and impact are still being measured. A key component of the policy debate surrounding it has centered on whether the strong rewards for performance under NCLB—and its increasing reliance on measurable performance—should be expanded to other areas of government and the nonprofit world. IPR economist Burton Weisbrod, John Evans Professor of Economics, is writing a book, under contract with Stanford University Press, that will consider the unintended—but foreseeable—consequences of the rising tide of efforts to measure “performance” and then to reward it. Titled "The Perils of Pay for Performance: Not Just ‘a Few Bad Apples,’" the book will cover a wide array of public and nonprofit sector services, such as higher education, hospitals, policing, museums, private charities, and the federal judiciary in addition to K–12 education.

Closing Persistently Failing Schools

While many schools have succeeded at improving student performance and proficiency rates on the standardized tests mandated by the No Child Left Behind Act, others fail to reach proficiency year after year and face increasing penalties, culminating with required “reconstitution.” This necessitates either dramatically restructuring or completely closing the schools, which disproportionately serve disadvantaged and inner-city populations. Schanzenbach and colleagues Lisa Barrow and Kyung-Hong Park are analyzing longitudinal data on the standardized test scores of Chicago Public School students since 1993. They are investigating how school closures can have an impact on the students actually attending those schools, particularly whether the affected students perform better at their new, non-failing schools. The study will better inform school administrators trying to decide whether to close a persistently underperforming school. This project is supported by a grant from the Smith Richardson Foundation.

Accountability Pressure and Practice

While numerous recent authors have studied the effects of school accountability systems on student test performance and school “gaming” of accountability incentives, little attention has been paid to substantive changes in instructional policies and practices resulting from school accountability pressures. This lack of research is primarily due to the unavailability of appropriate data to carry out such an analysis. A study co-authored by IPR education economist David Figlio offers new evidence from a five-year survey conducted of a census of public schools in Florida, coupled with detailed administrative data on student performance. Figlio and his colleagues show that schools facing accountability pressure changed their instructional practices in meaningful ways. In addition, with Princeton’s Cecilia Rouse, Urban Institute’s Jane Hannaway, and Dan Goldhaber of the University of Washington, Figlio presents medium-run evidence of the effects of school accountability on student test scores, concluding that a significant portion of these test score gains can likely be attributed to the changes in school policies and practices that they uncover in their surveys. The study is forthcoming in the American Economic Journal: Economic Policy, and it was supported by grants from the Bill & Melinda Gates Foundation and Institute of Education Sciences.

School Accountability and Residency

Using U.S. Census microdata, Figlio and his colleagues are investigating whether school accountability systems affect families’ decisions about school choice and about where they reside. Exploiting time differences in the introduction of a state-level school accountability system, the researchers confirm evidence for school accountability systems increasing the likelihood that families will enroll their children in private schools. The researchers also show that accountability systems influence where new residents to a metropolitan area choose to reside. The results are particularly pronounced in those states with low assessment standards, where large fractions of students—and therefore, schools—pass the accountability standards. The results differ by family type. The project received funding from the John D. and Catherine T. MacArthur Foundation and Institute of Education Sciences.