HMH Into Reading: 2020–2022 QED Study Research Results

Cobblestone Applied Research & Evaluation, Inc. conducted a quasi-experimental design (QED) study to determine the potential impact of the Into Reading program on student reading outcomes. The purpose of this ESSA Tier 3 Promising Evidence study was to answer one key research question: Do schools using Into Reading outperform comparable schools that use another ELA program?

Using variables from a range of relevant data sources, the Cobblestone research team used propensity score matching to find appropriate matches of Into Reading (treatment) and non-Into Reading (control) school sites across the state of Texas, yielding a final sample of 316 schools (140 treatment and 176 control). Utilizing statistical controls for selection bias, outcome analyses were conducted to assess potential differences between treatment and control conditions on the STAAR^® reading test. See Table 1 for the mean comparison of key demographic variables of the sample.

The analyses examined (a) the differences between Grade 3 students' pretest scaled reading scores (2020–2021) at the school-level, and their posttest scores (2021–2022, Grade 4); and (b) the differences between Grade 4 pretest scaled reading scores (2020-2021) at the school-level, and their posttest scores (2021–2022, Grade 5). Both comparisons indicated that Into Reading (treatment) schools had slightly higher average scaled scores.

The examination of the Grade 4 (2021–2022) STAAR posttest school-level scaled reading scores across conditions, controlling for Grade 3 pretest (2020–2021 ) scores, revealed that the Into Reading (treatment) schools had statistically significant higher average scaled scores at posttest (F [1, 184] = 4.81, p = .03; see Table 2] compared to students in the non-Into Reading (control) schools. These findings suggest that the Into Reading program significantly improved students’ reading skills in comparison to other programs.

It should be noted that an additional analysis examined the rates of Into Reading (treatment) students who were proficient (i.e., Levels 2, 3, and 4 indicate “Pass”) and not proficient (i.e., Level 1 indicates “Fail”) across conditions, for each grade level (i.e., Grades 3, 4, and 5). Findings revealed the percentage of Into Reading (treatment) students who did not meet the proficiency standard was notably lower than the percentage of non-Into Reading (control) students who did not meet the standard. The results for Grade 4 Into Reading students were marginally significant (p = .09) and Grade 5 Into Reading students approached the threshold for marginal significance (p = .11).

The Into Reading QED study was designed to determine the potential impact of the Into Reading program on student reading outcomes. Based on the results, the findings suggest that the Into Reading program significantly improved students’ reading skills in comparison to other programs. The study meets the ESSA Tier 3 Promising evidence standards (i.e., well-designed, well-implemented correlational study, with statistically controls for selection bias; and no strong negative findings from experimental or quasi-experimental studies).

Please download the Full Report to read more about the Into Reading 2020-2022 QED Study, including details related to the study design, propensity score matching procedures, study sample, and study results.