Do Reading Comprehension Assessments Deliver What They Promise?

by RSBSADMIN | Sep 7, 2018 11:36:06 AM

It is likely that your school district is trying to find an assessment tool to beef up its reading program and insure greater success on your state’s high stakes test. The search may have taken them to STAR Reading 360, I-Ready Adaptive Assessment Systems or the MAP Reading and Growth Skills Tests. All of these tests are a new breed of assessments, computer adaptive tests that promise far more than they can deliver. It is not their shortcomings that raise the greatest concern, it their incompatibility with what we know about reading comprehension and what we know about instruction that is troubling. In this blog article, Dr. Peter Dewitz discusses assessment and the Read Side by Side Reading Program.

Infographic on the reliability and validity of reading comprehension assessmentsAssessment tools are evaluated on two criteria: reliability and validity. Reliability determines whether the test is accurate and consistent. From one administration to another can you trust the consistency of the scores? These new tests are reliable. Validity asks a different question: does a test measure what it is supposed to measure? The tests listed above are built on a model of reading that is no longer compatible with the current thinking or research on reading comprehension. Comprehension is a process of constructing a mental model from the information in the text, and integrating it with prior knowledge. The reader is making inferences. The most widely accepted model of reading is that developed by Walter Kintsch and titled the Construction-Integration Model. Comprehension is not a series of discrete skills that can be arranged in a sequence and taught from September to May, but a gradual process of building and using knowledge.

construction integration model regarding reading comprehension assessmentsThese new computer adaptive assessments more reflect the skills model of reading than a Construction-Integration Model. When students take one of these new tests they respond to short passages, 300 words or less, they answer questions about main idea, cause and effect, theme, text organization, the meaning of words and word parts. These are pieces of the reading process, but the whole is greater than the sum of its parts. The computer feeds the students harder or easier questions depending on how they do and minimizes the time in front of the computer. These testing tools are efficient, but they sacrifice breadth and depth. No matter what their creators claim the diagnostic power of these tests is limited.

Dr. Peter Dewitz listing the essential qualities for becoming a mature readerIn the Read Side by Side Reading Program students read books and articles but mainly books. The students construct meaning across hundreds of pages, tracking long and sometimes integrated plots as they follow the development of characters and ideas. This is long term thinking not short-term skills application. Much of the students’ success is based on their desire, their motivation and their ability to work with others. What is essential to succeed with this program is not assessed with these new computer adaptive tests. Motivation, curiosity and perseverance may be more essential for becoming a mature reader than identifying a text organizational pattern or identifying the meaning of a prefix.

The new computer-based assessments will tell you if your students are reading on, above or below grade level, but you probably knew that about most of your students. Computer adaptive assessments do correlate with state assessments, but then all reading tests correlate with one another, because all share word identification, fluency and vocabulary in common. These computer adaptive tests fail to provide insights needed by teachers of the program or any teachers whose instructional focus is on books and vital themes.

Fortunately, the Read Side by Side Reading Program was designed to provide the data that any teacher would need. Let’s think about the vital components of reading. Hand any third grade student Jake Drake (Clements, 2012) Marvin Redpost (Sachar, 2011) or Beezus and Ramona (Cleary, 2009) and ask them to read 100 to 200 words and you will have strong information about their word identification and fluency. Read their retell summaries and you can assess their ability to construct a model of the text. Compare the book club retell summaries to the read-aloud summaries and you will know how well they have internalized the ability to construct and integrate information. Next, look at their turn and talk responses. Here you can gain insights into the ability to integrate text information and prior knowledge to draw inferences. The turn and talk summaries also tell you if the students can locate information in the text, a skill vital to success on the Smarter Balanced Assessment Consortium (SBAC). Finally, watch what they read on their own after you have read Shiloh (Naylor 1990), or Poppy (Avi, 1995). Then you will know if their motivation has changed. Notice if they can work well with a partner, a vital skill for any walk of life.

Computer adaptive assessments report what most teachers already knowComputer adaptive assessments are in vogue and their sales are climbing 30% a year. What these tests report most teachers already know. Relying on these tests undermines your confidence and distracts you from the most valuable data in your classroom – the thoughts and writings of your students.


Written by Peter Dewitz, Ph. D.

Director of Research

Read Side by Side Publications, LLC.

Dr. Peter Dewitz director of research headshot

Subscribe Now

Additional Reading