Overview of LD & RD Lesson 3: Readings (1) - previous page disabledreturn to mainnext page disabled
   

Using Curriculum-based Measurement for Assessing
Reading Progress and for Making Instruction Decisions

Stecker, P.M., Fuchs, L.S., & Fuchs, D. (1999).
Effective Reading Instruction for Individuals with Learning Disabilities Online Academy:
Teaching Reading to Individuals with Learning Disabilities.

Curriculum-based measurement (CBM; Deno, 1985, 1992) blends aspects of traditional, norm-referenced assessment with alternative assessment to provide a set of standard procedures that can be used for monitoring student growth in basic academic skills. With CBM, students take short tests on a frequent basis, usually twice weekly for students with disabilities or who are at risk, and teachers graph scores to document student progress over time. Progress toward a long-term goal is evaluated by applying standard decision-making rules to the graphed scores to determine if and when an instructional modification may be warranted. In this way, a teacher can chart student progress to determine whether the instructional program is having the desired effect for any particular student. The purpose of this paper is to describe the procedures for implementing CBM in reading. Specifically, the development and administration of reading measures and the rules for making instructional decisions based on graphed data are described.


Reading Measures

Selection of passages. An important component of CBM includes repeated measurement. Teachers make decisions about a student's progress over a period of time rather than judging a student's performance according to any one particular score. Consequently, multiple forms of the reading test need to be used. Additionally, in order to determine whether a student's reading achievement is actually improving (because the teacher's program is effective), these multiple measures should be comparable in terms of readability. Otherwise, student scores may fluctuate because the test materials vary in terms of difficulty. Another consideration is that students should read passages at their instructional level. If passages are too difficult, scores will be so low that a "floor effect" may be observed with the graphed scores. For example, at frustrational levels, the passages could be so difficult that student progress cannot be captured adequately by the CBM graph. On the other hand, if passages are extremely easy for a student, a "ceiling effect" may be observed, in which scores are so high that little progress could be charted over time. Reading passages to be used with CBM may be taken directly from the student's curriculum (i.e., reading series), or a standard set of passages may be derived from another source. A variety of readability formulas can be used to determine passage readability levels.

Reading passages should be selected randomly from the series to be used for measurement purposes. The passages should be several hundred words in length and should represent the type of reading in which the student is expected to be proficient by the end of the year. Less frequently used literary forms, such as plays and poems, may be omitted. Selecting passages from the actual reading curriculum used in the student's class is not essential for CBM. In fact, Fuchs and Deno (1994) argue that using passages directly from student's curriculum may artificially inflate student scores over time, as passages are more likely to be used with which the student is already familiar. To develop a pool of passages at a comparable difficulty level, the teacher may (a) apply a readability formula to selected passages from a reading series, (b) construct passages that reflect intact stories rather than story fragments and apply a readability formula to them, (c) order already developed passages at different readability levels from several different curricula from the University of Oregon CBM Network Web site (http://interact.uoregon.edu/dabcs/Academic/Graduate/order.html), or (d) use a computerized CBM data collection system (see Fuchs, Hamlett, & Fuchs, 1997) in which "complete" stories have been constructed and leveled (described below).

Oral reading rate. Once alternate forms of reading passages have been obtained, a teacher may administer a test on a twice-weekly basis by asking a student to read aloud for 1 minute. Research has documented the reliability and validity of using oral reading rate as a measure to document student change (see Deno, 1985). Because CBM data are used only as indicators of reading proficiency, each datum represents a student's overall reading achievement at a particular point in time. Measurement error is reduced, though, when data points are aggregated, or summarized, over time. Teachers can judge whether student performance is improving over time, staying the same, or decreasing. Oral reading rate is sensitive to student changes in performance. It also provides a gross measure of reading comprehension. Although the teacher may not learn much about a student's specific comprehension skills by listening to a student read aloud for 1 minute, oral reading rate is strongly correlated with standardized tests of reading comprehension (Deno, 1985). Positive correlations fall above .85, with most correlations around .90. Therefore, this strong, positive association between oral fluency and measures of comprehension indicates that, as students become more fluent readers, they also tend to comprehend more (and vice versa). Thus, oral reading rate can be used as an indicator that represents a student's overall reading proficiency. Teachers can use oral reading rate to judge reliably whether a student becomes a better reader over time. Additionally, as students become older, especially throughout the elementary school years, they tend to read faster and comprehend more. Normative data collected across students of different ages (and achievement levels) reflect these changes in oral reading rate.

Directions for administration. On a twice-weekly basis, a teacher administers a reading passage to each student whose progress needs to be monitored carefully. A passage (or textbook) is placed in front of the student. Another copy of the passage on which the teacher may note miscues needs to be prepared as well. The teacher asks the student to read aloud for 1 minute and follows along, marking any miscue that the student makes. If the title of the passage is included, it can be read aloud by the student but the scoring and timing should begin with the text of the passage. A slash should be placed through any word that is mispronounced. If a student pauses for 3 seconds on any word, the teacher may supply that word but counts it as incorrect. The student does not need to repeat the word. However, if the student self-corrects prior to teacher assistance (and within 3 seconds), the self-correction is scored as correct. A stopwatch should be used to keep an accurate account of time elapsed. At the end of 1 minute, the teacher tells the student to stop reading and places a double slash after the last word read.

Prior to having the student read, the teacher can read the following directions to the student.
When I say "start," begin reading aloud at the top of this page. Try to read each word. If you come to a word that you do not know, go ahead and try to say the word. If you wait too long on any word, though, I will tell you that word. At the end of 1 minute, I will tell you to stop reading. Be sure to do your best reading. Do you have any questions? Start.
The total number of words read correctly becomes the datum that is charted on the CBM graph. These quantitative scores help the teacher to determine (over time) whether the instructional program is effective for a particular student. If a program does not appear to be working, and the teacher wants to collect more specific information, he or she may elect to mark reading miscues on the passage protocol. For example, the teacher may indicate omissions, insertions, substitutions, and phonetic spellings for the mispronunciations the student makes. This qualitative information can help the teacher formulate the nature of an instructional modification for the student's program.

Long-term goals. A teacher determines whether a student is becoming a better reader by looking at the trend of student CBM data over time. If scores are gradually improving, the teacher can tell that a student is progressing. To determine whether a student is progressing at an adequate rate, however, the teacher compares student performance against an established goal line. This goal line connects student beginning performance (i.e., baseline, or current performance level) with an expected goal of reading proficiency by the end of the school year. The slope, or trend, of this goal line displays the rate of progress throughout the year that the student must exhibit in order to meet the expected goal.

A teacher can use his or her judgment to establish a realistic end-of-year goal. This goal should be higher than any of the data points from the baseline phase, yet it should be an ambitious expectation of reading proficiency by the year's end. Normative data across student achievement and grade levels indicate that students may improve as much as 1-3 correct words per minute each week, especially for students who are reading below a fourth-grade level. Therefore, a teacher can summarize current performance (usually 3-4 scores during baseline) by averaging scores or selecting the highest score the student attained. Then, the teacher counts the number of weeks left in the school year. The teacher selects a realistic, but ambitious weekly rate of increase, multiplies it by the number of weeks left in the school year, and adds the result to the current performance level in order to establish the goal. For example, if
  1. the student's current performance level is 20 correct words per minute,

  2. the teacher selects 2 correct words per minute increase as a realistic, but ambitious weekly rate of progress, and

  3. 25 weeks are left in the school year, the teacher will multiply 25 by 2 and add the result to the current performance level of 20 to establish a long-term goal of 70 correct words per minute ([25 x 2] + 20 = 70). An "X" or "G" can mark the spot on the graph to indicate the goal. The goal line connects the current performance level to the goal and aids the teacher in determining whether student progress throughout the year is sufficient to meet the expected goal.
Computer program. Fuchs et al. (1997) have developed computer software that facilitates the ease and efficiency with which CBM data can be collected and managed. Although the program cannot administer oral reading samples, oral reading rate data can be entered by the teacher (or aide or student), and the program will graph scores and apply standard decision rules for interpreting the graphed data. Class profiles can be provided that show classwide performance in addition to individual graphs. If, however, the teacher prefers to have the computer program also administer the reading tests, a maze procedure is used. Students read a passage at a particular readability level ("complete" story of about 400 words) from the computer screen in which every seventh word is deleted. For each blank, the student cycles through a presentation of three words and selects the correct word for the context. The story disappears after 2 1/2 minutes and shows the student the number of blanks for which a correct response was selected. Although the actual maze scores are lower than oral reading rate (i.e., potential of one correct response for every seven words), the maze scores mimic oral reading rate and correlate well with measures of reading comprehension. Therefore, the maze scores also can be used to track student progress across the year. Although decision rules are applied consistently to the graphed data, the teacher still needs to set a long-term goal. Following the same principles as in goal setting with oral reading rate, the teacher establishes a realistic, yet ambitious end-of-year goal. Based on typical student performance, a teacher may use .7 words correctly selected as an average weekly increase for these 2 1/2-minute maze measures.


Decision-Making Rules

Determining when to apply rules. A single CBM score is not adequate for informing teachers about student progress in the curriculum. The purpose of CBM is to document student change over time in order to determine whether the student is progressing appropriately in a particular instructional program. Teachers, then, must apply standard decision-making rules to the graphed data to determine if and when an instructional change is warranted. The following rules have evolved over a number of years of CBM research. These decision-making rules also are used in the CBM software.
  1. Collect baseline performance (at least 3-4 points) and set a realistic, yet ambitious end-of-year goal. Connect baseline performance to the goal to show the goal line (i.e., the student's anticipated rate of progress through the year in order to meet the goal).
  2. Draw a dotted vertical line following the last baseline point to indicate the establishment of the goal and the beginning of an instructional phase. Continue to monitor student performance on a twice-weekly basis and graph scores.
  3. Four-Point Rule: After 3 weeks of implementing an instructional program (at least six data points must be collected), examine the most recent four data points. If all four, consecutive data points fall above the goal line, draw another dotted vertical line and raise the goal. If all four, consecutive data points fall below the goal line, draw a solid vertical line and implement a teaching change.
  4. Trend Rule: If, however, the four, consecutive data points fall both above and below the goal line, keep collecting data. After collecting at least eight data points, a trend line can be drawn that represents a line of best fit through the data (see Appendix). (The computer program will draw a trend line through the most recent 8-10 data points.) This trend line shows the relative rate of progress the student is making during the most recent instructional phase. Compare the trend line to the goal line. If the trend line is steeper than the goal line, draw a dotted vertical line and raise the goal. If the trend line is less steep than the goal line, draw a solid vertical line and implement a teaching change. The four-point rule, however, always is applied first and supersedes the trend rule.
  5. Continue collecting CBM data on a twice-weekly basis and apply these standard decision-making rules for each phase of instruction.
Raising goals. Although a teacher makes an initial educated guess about an appropriate year-long goal, CBM data can be used to inform the teacher whether the goal may, in fact, be too low. Some research indicates that teachers may set inappropriately low goals, especially for students with disabilities, and that some teachers fail to raise their goals even if student performance data indicate that the goal would be reached or surpassed. Moreover, some research indicates that teachers who set high goals for their students tend to effect greater achievement among their students than teachers who set low goals (regardless of whether the students actually achieve those goals). Therefore, CBM decision-making rules include goal raising as a recommendation when student data indicate that the goal would be reached or surpassed. If the trend line through the current phase is extended to the end of the year, the teacher could set the new goal where the trend line suggests student performance should be by the end of the year if progress remains at the same high rate. Or, the teacher could be even more ambitious and set the new goal higher than the trend line suggests student performance will be. The teacher could work simultaneously to strengthen the instructional program, anticipating that he or she can stimulate even greater student growth.

Making instructional adjustments. If either the four-point rule or the trend rule indicates that the student is not progressing at the anticipated rate, an instructional adjustment should be made in the student's program. The teacher may be implementing a program that is, in fact, working well for other students but is not working well for a particular student. CBM decision rules, then, recommend that the teacher tailor the program to meet the individual needs of the student who is not progressing as anticipated. The teacher may vary the strategies or content of instruction, the arrangement of instruction (i.e., teacher-pupil ratio during instruction), the allocated time for instruction, the materials used, or the motivational strategies incorporated during instruction. Any one or combination of these variables may be altered to better meet the individual needs of the student whose CBM data indicate the necessity of an instructional adjustment. After the teacher determines the nature of this instructional change, the new plan should be implemented for at least 3 weeks prior to applying CBM decision-making rules to determine the success of the intervention.

Research documents that teachers who use CBM to monitor student growth and who make instructional adjustments when warranted effect greater student achievement than teachers who use their own methods of assessing student performance. By using CBM as a reliable and valid database, teachers can determine, in an ongoing fashion, whether an instructional program appears effective for particular students. By monitoring student progress over time, teachers also can compare the relative effectiveness of different instructional programs, thereby developing strategies and routines that may be stimulating the greatest growth among the very students who may exhibit the greatest need of differentiated instruction.


References

Deno, S. (1992). The nature and development of curriculum-based measurement. Preventing School Failure, 36 (2), 5-10.

Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219-232.

Fuchs, L. S., & Deno, S. L. (1994). Must instructionally useful performance assessment be based in the curriculum? Exceptional Children, 61, 15-24.

Fuchs, L. S., Hamlett, C. L., & Fuchs, D. (1997). Monitoring Basic Skills Progress (2nd ed.) [Computer software]. Austin, TX: PRO-ED.


Appendix

Directions for Drawing a Trend Line: Connecting Mid-dates and Mid-rates
  1. Draw a vertical line to divide the data set (i.e., one phase) in half.
  2. For each of the two halves of the data set, find:
  3. Place an "X" at both intersections of the mid-dates and mid-rates.
  4. Connect the two "X"s to show the trend line for the data set.

    Mid-date. First, divide the data set (i.e., the most current phase since the teaching change or goal raise) in half, using a vertical line. If there is an uneven number of data points, a vertical line will be drawn through one datum, that is, the point in the middle as the data are viewed from left to right. For example, if there are 9 scores in the current phase, a vertical line will be drawn through the fifth datum. Then, follow the same procedure for both sides of the data set. Draw a vertical line through the middle of the data points. If there is an even number of data points, the vertical line will pass through the middle of two points. For example, if there are four points, the vertical line will fall between the second and third scores collected.

    Mid-rate. The next step involves drawing a horizontal line through the middle of each half of the data set. This time "middle" refers to data from lowest to highest in value. If helpful, the scores could be numbered from low to high (i.e., 1, 2, 3, 4, etc.). These scores are referred to as "rate" because they indicate the number correct per period of time. Draw a horizontal line through the datum (or between two data if an even number of points) that falls in the middle in terms of value. For example, if the scores for one side of the data set were 32, 46, 37, 35, and 30, a horizontal line would be drawn through 35 because it is the middle score (i.e., the median score) for this group of data. If the scores were 32, 46, 37, and 35, a horizontal line would be drawn between 35 and 37 (at the value of 36) because 35 and 37 are the two middle scores.

    Trend. Place an "X" at the intersection of each of the two sets of vertical and horizontal lines. When the two "X"s are connected, the resulting line represents the trend for the data set in that phase.
  previous page disabledtop of pagenext page disabled