Beyond Despair! New York State ELA Tests Are Failing Our Kids

by Fred Smith, retired administrative analyst with the New York City public school system, with Robin Jacobowitz, Director of Education Projects at the Benjamin Center

New York’s Common Core testing hasn’t worked.  The tests have consistently failed our children, especially the youngest kids, English Language Learners, students with disabilities and minorities. In our most recent research on this subject, we found that far too few students are able to tackle the written portion of the English Language Arts (ELA). For a closer look at this subject, see the entire series, New York State’s School Tests are an Object Lesson in Failure.

This series and report examined results for all 1.2 million students in grades 3-8 across New York State from 2012–2016 when Pearson, Inc. was the test publisher. More detailed information was provided for children in New York City by its Department of Education. Students there make up 37 percent of the test population. This allowed us to analyze data within subgroups for the questions that required students to construct a response.

Our analysis shows that a substantial percentage of children were unable to write comprehensible answers to five or more questions out of the nine or ten on each ELA exam. That is, they received a zero score on at least five of these questions, meaning that their responses were deemed to be “totally inaccurate, unintelligible, or indecipherable” by trained scorers.

We call this criterion the Threshold of Despair.

A dramatic change occurred when exams were aligned with the Common Core. NYC’s overall data show that in 2012 fewer than five percent of third and fourth graders crossed this threshold. But, with the advent of Common Core-aligned exams in 2013, the percentage more than doubled: yes, that means it got worse, not better.

From 2014 to 2016 up to 13 percent of third and fourth graders were unable to decipher and respond intelligibly to half or more of these constructed response questions.

The percentages crossing the Threshold of Despair on these ELA questions were much higher for subgroups. Up to 34 percent of English Language Learners and 36 percent of students with disabilities were not able to comprehend and/or give relevant answers to half or more of the nine or ten questions per test. The figure for minority students was 17 percent.

This is clear evidence that the ELA test left many eight- and-nine-year-olds in a virtual daze.

Further Analysis

“Five or more” zeroes is an arbitrary cutoff point. It is useful to give an extended view of the distribution of hardship these tests yielded by examining how many students scored at least three zeroes—that is, produced this score on approximately one-thirdof the written portion of the test.

The percentage of third graders who received zeroes on three or more questions doubled, from 14 percent in 2012 before the tests were aligned with Common Core, to 28 percent in 2013 after the alignment, and then remained at that level from 2014 through 2016.

Thirty-one percent of English Language Learners received three or more zeroes prior to the Common Core testing; the percentage at this dim level rose to 57 percent in 2016! The percentage for students with disabilities rose from 35 to 58 percent in this same time frame. The percentage of black and Hispanic children went from 17 to 34 percent. Their white counterparts also showed diminished performance, though they registered less than half of these percentages—going from 7 to 14 percent between 2012 and 2016 on at least one-third of the written portion of the exam.

The progression for fourth graders is not as sharp but follows a similarly gloomy trend. Overall, 23 percent of fourth-grade students fell into this category, once the tests were Core-aligned.

Please see the chart below for a deeper dive on the dismal facts and figures.

Fixing It. Now What?

What do we learn from this chronicle of failure? First, the State Education Department’s (SED) massive testing program problematically expected young kids to be able to handle material and a testing process that was over their heads. Students were made to sit for long hours on consecutive days, read lengthy passages, flip pages back and forth in a test booklet, and try to answer the questions. This cannot be the best way to assess progress toward standards for young students. We’re talking about eight- and nine-year-olds, not teenagers. And we’re talking about a disproportionately negative effect on students of color.

The Department dismissed early complaints from a small number of parents who questioned the appropriateness of the exams, mainly for the younger kids. The criticisms mounted, however, and by 2015, opting out of these exams became a movement that snowballed to more than 200,000 students whose parents and guardians rejected then.

The SED maintained that the growing numbers of objectors to this testing lacked evidence to support their views. Yet, SED tacitly acknowledged the need to fix basic underlying problems by promoting its own testing remedies, such as eliminating time limits in 2016 and decreasing the number of items on a test. But if a student cannot comprehend a question and write an acceptable response, more time on shorter tests won’t help. Timing and design issues should have been resolved before these flawed instruments were foisted on students. After-the-fact changes in administrative procedures do not transform a bad commercial test product into a good one.

Sadly, New York State still isn’t paying attention. In 2012, the State Education Department stopped releasing all of the testing data that it had posted in previous years. Questar, the current testing company, is now entering the third year of its $44 million contract.

Still there are no Truth-in-Testing laws—which would allow independent appraisal of test material and analysis of item-level statistics. So the process continues to operate without accountability. 

Testing, like death and taxes, is certain. With more rounds to come,it’s time for the State Education Department to release complete information about the 2017 and 2018 tests that Questar has produced, before we are led once again into another year of exams without proof of their worth.

To move forward, we propose an overhaul of the testing program—a staged three-year rescue plan to be piloted this year in the third and fourth grades, where the tests engender gloom and despair and convey little useful, interpretable information. We have evidence for this; there’s no excuse for waiting and debating.

With oversight from the Board of Regents, the new program would rely on teacher-made tests, performance- and project-based measures, and periodic district- or countywide exams. The pilot would become the platform for scaling up to grades 5 and 6 in 2020 and grades 7 and 8 in 2021.

The plan embraces both formative and summative aspects of evaluation. Measurement and observations of students would be designed to inform what happens in the classroom throughout the year, letting teachers and principals concentrate on creating the best learning environment.

To this end, testing should serve instructional purposes. Sores must not be given determinative weight in critical decisions affecting students, teachers, principals or school effectiveness. 

A well-coordinated assessment program would make teachers, principals and parents partners in its success. Testing companies won’t like this because it breaks their dominance. Too bad!

We have learned that the Common Core testing program has greatly under-served New York’s children. The evidence of student befuddlement that we uncovered constitutes an indictment of tests that treat children harshly and return little if any educational value. The testing program has brought forth no appreciable gains in achievement. That’s a system that isn’t working for anyone. It must be changed.