Attack on Milwaukee Voucher Research Comes Up Short
An attempt by a self-described "non-partisan" organization to discredit a voucher researcher's results showing higher graduation rates for voucher students came up short recently when it was pointed out that not only was the criticism unfounded, but the critic's organization was not a neutral observer on the issue of school choice. Graduation rates are important because they are a readily understood measure of the value of an education system or program.
Although the school voucher program in Milwaukee--the Milwaukee Parental Choice Program (MPCP)--has been in place for 14 years, the Wisconsin legislature has not yet established an official procedure for evaluating the program on an ongoing basis by a non-partisan group like the Wisconsin Legislative Audit Bureau. Pro-school choice groups and individuals, such as School Choice Wisconsin and Howard Fuller, have called for such an evaluation, which is strongly opposed by the state teacher union. Governor James Doyle (D) recently vetoed a bill establishing a voucher evaluation procedure.
To get some measure of the long-term value of school vouchers, School Choice Wisconsin retained researcher Jay Greene in 2004 to study graduation rates of MPCP students. Greene is a widely recognized education scholar whose work was cited in the U.S. Supreme Court's 2002 Zelman ruling on Cleveland vouchers. The Education Trust calls Greene "one of the most widely cited ... experts. His state-level graduation rate calculations are used by such policy organizations as the Education Commission of the States and the Alliance for Excellent Education."
Using a method widely accepted by education researchers, Greene found a 64 percent graduation rate for MPCP students in 2003, compared to 36 percent in the Milwaukee Public Schools. He obtained similar results using a different approach developed by the Harvard Civil Rights Project and Urban Institute. (See "Milwaukee Voucher Students Have Higher Graduation Rates," School Reform News, November 2004.)
On November 13, Public Policy Forum (PPF) researcher Anneliese Dickman published a critique of Greene's graduation rate study, saying his method for estimating graduation rates did not properly reflect student transfers in and out of the voucher schools. To evaluate the performance of voucher students, she called for placing the same strict accountability measures on choice schools as on public schools, something PPF has long recommended.
Characterizing her organization as "dedicated to non-partisan analysis," Dickman dismissed Greene's study as "propaganda," "unsound," "reckless," and not "mature." After completing her attack, she immediately called for a "truce" in the "war" between the two sides of the voucher debate.
Responding to what he called Dickman's only substantive but "mistaken" criticism, Greene noted his report had in fact described the concern she raised about student transfers as "the primary source of uncertainty in these estimates." To address them, he had used the Harvard/Urban Institute method to see if it produced different results, and it did not.
"This check should dismiss objections that the results are significantly biased by the transfer of students," noted Greene, "because the Harvard/Urban Institute method is much less sensitive to the transfer of students since it looks only at the movement of students in a one-year period."
Responding to Dickman's accusation that Greene's work was "propaganda posing as research," School Choice Wisconsin President Susan Mitchell noted Greene's respected standing as a researcher and pointed out that his method for estimating graduation rates is used by the national publication Education Week for its annual "Quality Counts" report.
Besides, added Mitchell, Dickman's organization "is hardly an adequate arbiter of research quality," citing a December 2002 PPF study that touted the superior performance of one group of students over another without using any controls--a no-no taught in Research 101.
Nor is PPF "non-partisan" when it comes to school choice issues, said Mitchell, noting PPF strongly supports a position that was rejected by the Wisconsin Supreme Court --putting rigid accountability systems in place for choice schools and treating them like public schools.
"While the PPF is entitled to this view, it makes the PPF a participant in the debate, not an objective observer," said Mitchell. "Indeed, major opponents of school choice believe more regulation will undermine the program by discouraging good private schools from participating."
One of the nation's leading education scholars, Paul Hill of the University of Washington, pulled out of a PPF project when he found its view was biased on this issue. He said, "[a]s the [PPF] reports were being drafted it became clear that the dominant people on the study ... had adopted a clear position in favor of using the same oversight processes for schools of choice as for conventional public schools."
George A. Clowes (firstname.lastname@example.org) is managing editor of School Reform News.
For more information ...
The School Choice Wisconsin Web site at http://www.schoolchoiceinfo.org offers not only a link to Jay Greene's study of graduation rates but also a wealth of information on school choice programs, school choice research, and ongoing school choice news.