Testing Experts Sees 'Illusions of Progress' Under NCLB (from Education Week)

FYI.
-----------------------------------------------------------------------------------
Education Week
Published Online: September 30, 2008
Published in Print: October 1, 2008

Testing Expert Sees ‘Illusions of Progress’ Under NCLB
By Scott J. Cech

Washington
Harvard University researcher Daniel M. Koretz has some good news and some bad news for policymakers looking ahead to the reauthorization of the federal No Child Left Behind Act: The nation’s K-12 students are now attending Lake Wobegon schools.

Because of rampantly inflated standardized test scores, Mr. Koretz contended at a panel discussion here last week, all children seem to be above average, as in the fictional town made famous by the radio personality Garrison Keillor—or at least better academically than they actually are.

“We know that we are creating, in many cases, with our test-based accountability system, illusions of progress,” Mr. Koretz told a roomful of policymakers and educators at a Sept. 22 forum hosted by the American Enterprise Institute, a Washington think tank.

Mr. Koretz, a professor of education at Harvard’s graduate school of education who has written a new book, Measuring Up: What Educational Testing Really Tells Us, said that under the No Child Left Behind law, widespread teaching to the test, strategic reallocation of teaching talent, and other means of gaming the high-stakes testing system have conspired to produce scores on state standardized tests that are substantially better than students’ mastery of the material.

“If you tell people that performance on that tested sample is what matters, that’s what they worry about, so you can get inappropriate responses in the classroom and inflated test scores,” he said.

Mr. Koretz pointed to research in the 1990s on the state standardized test then used in Kentucky, which was designed to measure similar aspects of proficiency as the National Assessment of Educational Progress, the federally sponsored testing program often called “the nation’s report card.”

Scores on both tests should have moved more or less in lock step, he said. But instead, 4th grade reading scores rose sharply on the Kentucky Instructional Results Information System test, which resembled an NCLB-test prototype, from 1992 to 1994, while sliding slightly on NAEP over the same period.

“If you’re a parent in Kentucky, what you care about is whether your kids can read, not how well they can do on the state test,” Mr. Koretz said. “And the national assessment told us that, in fact, the gains in the state test were bogus.”

Shortage of Studies
In his book, Mr. Koretz says a similar situation occurred during the “Texas miracle” of the 1990s: Minority students posted large gains on the high-stakes Texas Assessment of Academic Skills, but much smaller ones on NAEP.

Mr. Koretz said the relative dearth to date of comparative studies on large-scale state assessments isn’t for lack of trying. He said he and other scholars have often been rebuffed after approaching officials about the possibility of studying their assessment systems.

“There have not been a lot of studies of this,” Mr. Koretz said, “for the simple reason that it’s politically rather hard to do, to come to a state chief and say, ‘I’d like the chance to see whether your test scores are inflated.’?”

Bella Rosenberg, an education consultant and a longtime special assistant to Albert Shanker, the late president of the American Federation of Teachers, echoed many of Mr. Koretz’s critiques.

“This is officially sanctioned malpractice—if we did this in health, in medical practice, we’d all be dead by now,” she said of the testing system under the NCLB law. “The fact is, we’re doing a great job of obscuring instead of illuminating achievement gaps.”

But fellow panelist Roberto Rodriguez, a senior education adviser to U.S. Sen. Edward M. Kennedy, D-Mass., the chairman of the Senate Health, Education, Labor, and Pensions Committee who helped shepherd the NCLB law through Congress in 2001, cautioned against “throwing the baby out with the bath water.”

While a more effective accountability system “needs to be a goal, I would argue, of the upcoming reauthorization of the [Elementary and Secondary Education Act],” he said, that “summative assessment is a critically important area of accountability.” The No Child Left Behind law is the current version of the ESEA, first enacted in 1965.

On the tug of war between advocates of the current system and detractors who prefer that tests better reflect the course content that students are supposed to be mastering, Mr. Rodriguez said, “I think we can have our cake and eat it too.”

Incentives for Instruction
But Mr. Koretz said some parts of the nearly 7-year-old NCLB law need to be fundamentally reworked.

“I think we ought to mandate, as part of No Child Left Behind, independent evaluations of state accountability programs,” he said. “There is nobody involved in this system who has an incentive to look for good instruction anymore—all the incentives are lined up in one direction: Increase scores on the summative tests at any costs.

“We need to create a system in which somebody … has incentives to make sure we’re not just gaming the system.”

Williamson M. Evers, the U.S. Department of Education’s assistant secretary for planning, evaluation, and policy development, who attended the panel discussion, suggested that any changes to the ESEA law should preserve its essence.

“Some are proposing changes in accountability that remove consequences; this would make it accountability in name only,” Mr. Evers said in an e-mail. “Results should be comparable statewide, or parents, taxpayers, and teachers themselves can’t tell how students are doing.”

Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, praised the points Mr. Koretz makes about testing under NCLB in his book, and suggested steps to address the law’s flaws.

“Washington should first ensure that states use a common benchmark that defines proficient student achievement in reading, writing, and math,” Mr. Fuller, who is the director of the Policy Analysis for California Education research center based at Berkeley and Stanford University, said in an e-mail. “Second, neo-No Child policies should ensure that each state sets a valid way of tracking student growth over time, and provide states the resources to build necessary data systems.”

Vol. 28, Issue 06, Page 8

1 in 5 fail portion of Grade 10 MCAS (from Boston Globe)

I would be interested in a study that looks at the achievement of low-income Black and Latina/o students attending middle-class suburban schools. Once again, we are at a point across the country when the test scores from the spring will be publicly reported and the same old story will continue to emerge--there is an achievement gap by race and class. What is usually left out of the narrative is the opportunity gap and all the other factors that impact equity in education.
-Dr. Louie F. Rodriguez
--------------------------------------------------------------------------------------
1 in 5 fail portion of Grade 10 MCAS
Addition of science to exam a factor
By James Vaznis
Globe Staff / September 17, 2008

The percentage of sophomores who passed the MCAS exam on the first try this year declined for the first time because thousands of students failed the science section, a new graduation requirement, according to statewide scores released yesterday.
Globe Graphic MCAS trends
Twenty percent of the class of 2010 failed at least one portion of the test, compared with 13 percent last year, when sophomores needed to pass only the math and English portions.
One of the few bright spots in the latest results of the 10-year-old Massachusetts Comprehensive Assessment System exam was math, where scores hit a historic high for all grade levels. But even there, state education officials expressed concern that middle school math performance remained stubbornly sluggish. Subpar math scores have largely caused the state to designate two-thirds of the state's middle schools for improvement under a federal accountability law, according to a recent Globe analysis.
Overall, the results of the spring exams showed a persistent achievement gap, with white and Asian students outperforming other students at all grade levels, often by a wide margin, while reading scores for the youngest test-takers declined.
The mixed results prompted many state education advocates to highlight the urgent need to jump-start the state's 15-year-old effort to overhaul education, which they contend has sputtered in recent years.
"State policy makers are getting wobbly in their support for education and high standards," said Jamie Gass, director of the Center for School Reform at the Pioneer Institute.
The results could also provide fodder for next year's debate on Governor Deval Patrick's sweeping 10-year plan to better prepare students for college and jobs in the fields that drive the state's economy: biotechnology, engineering, healthcare, and other science-related fields.
State education officials were mostly upbeat about the results during a press conference yesterday morning. They applauded improvement in performance among student groups who historically struggle on the exam, such as black and Latino students, although the officials voiced frustration that the achievement gap remains wide.
For instance, in Grade 4 math, Latinos scoring in the top two categories improved by 4 percentage points, to 28 percent. White students improved by 2 percentage points, to 56 percent. The four scoring categories are advanced, proficient, needs improvement, and warning/failing.
"Students of color and low-income backgrounds have made more progress than their counterparts . . . but we need to do a better job," said Mitchell Chester, the state's commissioner of elementary and secondary education.
Yesterday the state released only statewide results for the exam, which is given each spring to students in grades 3 through 8 and in Grade 10. Individual district and school scores are scheduled for release next week. The exam, part of the 1993 Education Reform Act, was first given to students in 1998.
Students in the class of 2010, who took the MCAS this past spring as sophomores, will be the first group that must pass the science exam to graduate, adding to a five-year-old graduation requirement for passing the math and English exams. Students have the choice of testing in biology, chemistry, physics, or technology/engineering, and must take at least one of those exams either their freshman or sophomore year.
While the decrease in the percentage of students passing the test disappointed many educators and advocates, many believed that students did much better than expected on the science exam. The 17 percent of sophomores who failed the science exam this year represented a decrease from the 25 percent who flunked last year.
"This is a great start," said Jill Norton, executive director of the Rennie Center for Education Research and Policy. "As people become more familiar with the science exam, we'll see scores increase in the coming years."
However, many student groups who typically struggle in school are in jeopardy of not graduating because of the science exam, alarming many educators and advocates. Overall pass rates for English, math, and science show barely half of black and Latino 10th-graders and less than half of students with disabilities passed. Even more staggering, just 28 percent of students who speak limited English passed all three tests.
By contrast, 85 percent or more of Asian and white students passed.
"A tremendous amount of work remains," said Lance Hartford, executive director of the Massachusetts Biotechnology Education Foundation. "I'm increasingly concerned about the gap between inner-city students and what's going on with students in the rest of the state."
Boosting performance, educators and advocates said, may have to start as early as kindergarten to foster a genuine interest in the sciences among students. That, they said, will require devoting more time to the subject in elementary schools and more training for those teachers.
In secondary schools, the state is facing a critical shortage of qualified science teachers. Yesterday, educators and advocates said the state needs to do more to bolster the numbers by creating mentoring programs or paying those teachers more. Science labs, many of which date to the 1960s, also require updating.
In the short term, students who fail the science exam once can file an appeal with the state based on passing grades in a comparable high school course, under emergency rules adopted last week by the state Board of Elementary and Secondary Education. Students have to take the math and English exams three times before appealing.
Prospects for a successful appeal are slim. The department has granted only 2,800 appeals in the last five years, rejecting roughly 20 percent to 30 percent in recent years. A rejected appeal would force students to take the science test again.
Reading scores for younger students also raised concern. They dropped for grades 3, 4, and 5 after largely stagnating in recent years, prompting state education leaders once again to call for a renewed focus on the lower grades.
A student's ability to master reading is widely considered the best gauge of future academic success.
Chester said he believes schools are doing a good job in teaching the fundamentals of reading, such as letter and word identification, but more attention needs to be devoted to teaching students to read for meaning.
State Representative Patricia Haddad, a Somerset Democrat who is chairwoman of the Joint Committee on Education, said she found the flatness in the middle school scores in English to be the most worrisome.
"The middle school scores are a really good indicator of where kids are heading," she said.
Yesterday Chester and Education Secretary Paul Reville reaffirmed their support for MCAS testing as the governor embarks on a host of initiatives aimed at overhauling the education system. They were attempting to quash speculation within the state's education community that the less stringent appeals process on the science exam was a sign that the agency was softening its stance on MCAS as a graduation requirement.
"When you are adding a new requirement, like science," Chester said, "it's hard for me to see how that's backing off standards."
James Vaznis can be reached at jvaznis@globe.com.