How can researchers extract the rich lode of administrative data buried in state agencies that is so critical for their studies of welfare reform? How do they obtain it in useable form? And how will they assure the privacy and confidentiality of welfare clients and agencies?
These were among the "make or break" issues aired at a conference on "Evaluating State Policy: The Effective Use of Administrative Data," convened by the Poverty Center at Northwestern on June 16 and 17 and attended by more than 100 researchers and state administrators.
"With the new welfare reforms giving much greater discretion to states in the design of public assistance programs, it will become increasingly important to use such data in analyzing state-specific policies," said Poverty Center Director Rebecca Blank.
Researchers at the conference presented nine papers showing how state administrative data has been used to evaluate aspects of state social policies such as child support enforcement, welfare-to-work, and family planning programs. Woven through all the sessions were discussions of data access and quality, sampling problems, and linkages between data from different programs and between administrative data and other data sources.
Conference participants also heard a preliminary report from the national Advisory Panel on Research Uses of Administrative Data headed by V. Joseph Hotz of the University of Chicago. The Poverty Center will release the panel's report in August.
"The panel was motivated by the realization that there is considerable state-by-state variation in administrative data and that surveys may not provide adequate information or produce the kind of substate information that could prove useful," Hotz explained.
The Pluses. "Administrative data could change the study of poor families in very important ways over the next 20 years," said Berkeley professor Henry Brady who demonstrated how linked datasets offer researchers more variables, more cases, and more points in time.
Noting that researchers are already using such data for studies of child welfare, child well-being, child care, and child support, as well as welfare-to-work programs, Brady said this data could also inform about program participants, special populations, small demographic groups and small areas, successful county models, and referral services-and cost far less than surveys. Furthermore, linking information across time and across programs of various agencies with multiple objectives could help assess the impact of such policies as time limits on public assistance.
But these sources also are limited by the nature of their data-gathering, a paucity of control variables, and their failure to measure outcomes, Hotz pointed out. There is also a problem of selection bias: Clients are counted only when they remain in programs and are lost when they leave; others seeking similar services may not be visible to researchers if they enroll in a different program with similar objectives.
Key Issues. Hotz's panel report will probably focus on the following "make or break" issues:
Hotz also cautioned that administrative data may be more time-consuming and costly than researchers believe, especially since coding is not very standardized. Researchers must also address methodological problems, such as the limited number of explanatory variables, the timeliness of data delivery, and the predilection of clients to tell different stories to different agencies.
Follow the Leader. " To be successful, we must develop institutional trust between researchers and state data-gathering agencies," Hotz said. The model for many in the field is Brady's UCData project at Berkeley. That unit has joined forces with California's Department of Social Services to compile a large group of datasets from around the state and has made them readily accessible to researchers. UCData is now surveying 20 cities to determine what datasets are available in their administrative information systems. Central archives like UCData can mediate between the state and other researchers, and could shoulder some responsibility for data cleaning, and maintaining and establishing data links, Blank suggested.
The report will emphasize the need to build relationships with providers and push for common standards for data across states. It is also likely to recommend a clearinghouse, perhaps with a Web site, to provide help for states that have not yet developed information sources.
"This conference could not come at a more important time, " observed IPR Director Fay Lomax Cook. "With the devolution of Temporary Assistance to Needy Families (TANF) to the state level, good state-level data and good use of it are crucial to help us understand the effects of welfare reform in people's lives. The more we can encourage states to use similar measures in collecting data, the better we can make comparisons not only within states, but across states."
 |
IPR Home | Calendar | News | Events | Publications | People | Research Programs and Centers Institute for Policy Research • 2040 Sheridan Road • Evanston, IL, 60208-4100 T: 847-491-3395 • F: 847-491-9916 • ipr@northwestern.edu Last updated 03/16/2012
|