HMH Into Reading: Implementation Research Study Results 2019–2020

Literacy is the foundation of success in education and beyond. It is important that students develop into fluent readers and effective writers starting in the elementary grades. Literacy in the United States, however, is an ongoing concern. According to the National Assessment of Educational Progress (NAEP), fourth grade reading scores have remained stable since 2005 with only about a third of students showing proficiency in reading ability. In 2019, the NAEP scores dipped significantly by one point when compared to 2017. The gap between high- and low-performing students continues to increase, and racial and ethnic minority students continue to lag behind majority students in reading ability on NAEP (U.S. Department of Education, 2019). The Progress in International Reading Literacy Study (PIRLS) compared the reading ability of several countries, and the most recent assessment conducted in 2016 found that the United States is trailing many countries (Warner-Griffin et al., 2017). Because of the foundational nature of reading to general educational outcomes, it is imperative that reading education improve in the United States.

In 1997, the U.S. Congress created the National Reading Panel (NRP), charged with the task of reviewing the research on reading and writing to provide suggestions for both classroom practice and future research. To provide better reading instruction, the panel suggests a wide range of teaching practices including tutoring, small-group instruction, and large-group instruction. They recommend targeting phonics instruction and awareness, reading comprehension, and the use of technology in reading instruction (Cunningham, 2001). These various forms of instruction are not to be used independently, but instead support each other to create capable readers and writers (Camilli et al., 2003; Moore et al., 2016). As per the NRP’s report and follow-up research, a successful reading program will use many different approaches to provide instruction to students (Camilli et al., 2003; Cunningham, 2001).

Minority students, English learners, and students with disabilities are particularly likely to struggle attaining reading proficiency (U.S. Department of Education, 2018). A successful reading program will pull on the various forms of instruction to offer a differentiated approach, which can help provide high-quality education (Gibbs & Beamish, 2020). In this way, teachers may be able to decrease the gap between high- and low-performing students (Brozo, 2009). Understanding and addressing the differences among students can lead to improved learning outcomes for reading (Bui & Fagan, 2013; Reis et al., 2011). This differentiated reading instruction has been shown to be particularly successful in supporting striving readers' development of reading fluency (Förster et al., 2018). Hattie (2009) has provided support for continuous feedback between student and teacher. A teacher’s ability to illicit and respond to feedback from students is one of the greatest indicators of student success. Teachers who are able to tailor instruction individually provide positive outcomes and a better classroom experience for the student (Ismajli & Imami-Morina, 2018).

Another important aspect to student learning is individual mindset, which determines motivation and effort, leading to improved educational outcomes (Blackwell et al., 2007). Scaffolding techniques in instruction can be used to increase motivation, which helps to reinforce a growth mindset as students overcome challenges (Guthrie et al., 2004).

HMH Into Reading is a K–6 comprehensive literacy curriculum designed to improve literacy through student self-actualized learning. With a focus on supporting all learners, the curriculum is differentiated by design and aims to foster a culture of learning in the classroom and growth mindset in students so they can become independent learners.

HMH Into Reading includes whole-group and small-group instruction, as well as individual and collaborative print and digital learning activities. Integrated assessments and reports allow teachers to constantly be aware of student ability and progress so they can respond appropriately. Teachers are given support materials designed to aid in teaching individuals learning English or struggling with a disability.

The program supports both student and teacher experience in the classroom:

Student Experience

• Reading: Students have access to a large library of culturally relevant and ethnically diverse texts to build cross-disciplinary knowledge. Notice and Note guides direct students to the identification of key signposts to interpret texts for meaning.
• Integrated Instruction: Reading and writing lessons are implemented in tandem. Books are used as a springboard for writing and discussion.
• Scaffolding: Students receive initial support and practice that prepares them to become independent learners.
• Differentiation: Lesson plans are customizable to provide appropriate support for each student.

Teacher Experience

• Instructional Support: A fully integrated online platform allows teachers to plan, teach, assess, and differentiate instruction. Small-group routines, scaffolds, and strategies for English learners are provided.
• Professional Support: Getting Started courses and online coaching videos provide information on teaching strategies and techniques.

The HMH Into Reading implementation study was conducted by Cobblestone Applied Research & Evaluation, Inc., a third-party research firm, who collected and analyzed the data from one school district during the 2019–2020 school year. The purpose of the study was to understand how teachers implemented the curriculum and the extent to which students’ reading knowledge increased. In addition, the research team collected information regarding how the program was used in classrooms by teachers and their students to provide additional insight into usability of the curriculum components.

The results from the study revealed that students using HMH Into Reading made significant growth from the beginning of the year to the middle of the year as measured by the Renaissance Star Reading assessment. Student achievement gains occurred across all participating grades and subgroups, regardless of gender, ethnicity, special education status, English learner status, or Gifted/Talented status. Unfortunately, the study was terminated in the middle of March because of the global pandemic, which resulted in the closure of the school buildings, cancellation of standardized assessments, and implementation of travel restrictions. This early outcomes research provides initial evidence of the effectiveness of HMH Into Reading.

The HMH Into Reading implementation study was designed as a one-year program evaluation in which teachers implemented the HMH Into Reading curriculum for the first time. The study sought to answer the following research questions:

1. How is HMH Into Reading typically implemented in classrooms?
2. What are the initial effects of HMH Into Reading on student literacy achievement?
3. What are the teachers’ initial reactions to the HMH Into Reading program?

The study was conducted in a mid-sized suburban district in the western United States. During the 2019–2020 school year, the district enrolled approximately 10,000 students across pre-K–12. The majority (48%) of the students were White, 29% of the students were Hispanic or Latino, 10% were Multiracial, 9% were Black or African American, and 3% were from other racial/ethnic groups (Asian, American Indian, Alaskan Native, Native Hawaiian or other Pacific Islander). Thirty-three percent of the students were eligible for free or reduced-priced lunch, 13% received special education services, and 2% were English learners.

Site recruitment for the research study was conducted by HMH^®. The goal was to recruit three schools within the district and three to four teachers within each of the two selected grade levels. Prior to the study launch, HMH representatives confirmed that potential participants had adequate time, interest, and district support to participate in the research study. The final study sample included 10 teachers in Grade 3 and 10 teachers in Grade 5 for a total of 20 teachers, across three schools in the district. Participating teachers had an average of 11 years teaching experience and approximately 7.5 years teaching in their current grade level.

In total, 387 students participated across the three schools in the study, including 180 Grade 3 students and 207 Grade 5 students. Overall, the majority of the students (58%) were White, 30% were Hispanic or Latino, 9% were Black or African American, and 3% were from other racial/ethnic groups (Asian; Native Hawaiian or other Pacific Islander). Approximately 10% of the participating students received special education services, 6% were identified as gifted and talented, and 2% were English learners (see Table 1).

District implementation of HMH Into Reading during the 2019–2020 school year included the use of all program components (except for the writing component materials). To support implementation of this new program, all teachers participated in the HMH Into Reading Getting Started professional learning full-day in-person course, which introduced teachers to the reading components included in the program prior to implementing the curriculum (May 2019). To further support implementation, the district provided literacy coaches to directly support all teachers. The district literacy coaches provided direct support and guidance related to HMH Into Reading instruction and use of program materials. In addition, HMH provided participating teachers with a follow-up Getting Started implementation session (October 2019) and an in-person coaching visit (January 2020).

To ensure the safety and welfare of students, teachers, and all school personnel in the wake of the COVID-19 pandemic, the district pivoted to remote learning for the remainder of the school year (March to June 2020) and the remaining HMH professional learning sessions were not delivered.

Implementation Fidelity

To interpret student achievement outcomes appropriately, it was important to measure implementation within classrooms. The study assessed program implementation for the duration of the study. The breadth of content covered, as well as the quality of implementation, was assessed via classroom observations and online teacher implementation logs.

To understand how HMH Into Reading materials are used in classrooms, teachers completed monthly implementation logs via an online survey (August 2019 to March 2020). The purpose of tracking implementation via logs was to receive timely information regarding the extent to which teachers implemented the curriculum and instructional best practices as intended by HMH.

To validate and supplement information contained in the teacher implementation logs, Cobblestone Applied Research & Evaluation, Inc., conducted classroom observations during the fall of 2019 (the spring 2020 observations were canceled because of COVID-19 school building closures). Observation protocols focused on teacher implementation practices, program usage and fidelity, and classroom learning environment.

Student Assessment

Participating students completed the Renaissance Star Reading assessment (Star Reading assessment) at the beginning and middle of the school year. The Star Reading assessment is a standards-based Computer Adaptive Test (CAT) that measures reading comprehension and a broad range of other reading skills. The Star Reading assessment was selected as an objective measure of achievement because of its strong psychometric properties and because it was already part of the district assessment plan, therefore reducing additional assessment burden on teachers and their students.

Teacher Feedback

To capture teachers’ experience and satisfaction with the HMH Into Reading materials, teachers completed an online product satisfaction survey. To assess implementation during remote learning (April to June 2020), questions were added to the survey to assess the extent to which teachers continued to use HMH Into Reading resources after the school buildings closed.

Overall, curriculum implementation across study schools closely aligned to the HMH Into Reading Logic Model (Theory of Action). During classroom observations, researchers recorded how various components of the HMH Into Reading program were used. Classroom observations confirmed that teachers had the appropriate curriculum resources, materials, and environment (e.g., access to internet for digital components, sufficient student texts, classroom conducive to small-group instruction) and followed (or slightly modified) the suggested daily 120-minute instructional block.

Researchers recorded how various components of the HMH Into Reading program were used during classroom observations. Generally, teachers used the teacher and student textbooks and leveled readers in most observed classes, while some also used the digital HMH Into Reading resources. Observers noted that teachers displayed high levels of classroom management, rapport with their students, overall effectiveness, and preparedness for their classes. Most students were engaged throughout the classroom observations, during both individual and group work.

Together teachers within each grade level at each school closely mapped out each Module and lesson taught so that all students within the school were on the same implementation schedule. Lessons usually followed the suggested daily 120-minute instruction block: 10–20 minutes of whole-class instruction, 45 minutes of small-group instruction, 15–20 minutes of extended independent reading, and time for additional activities to build knowledge (e.g., read alouds; vocabulary cards).

The HMH Into Reading curriculum includes 12 Modules, each consisting of five unique lessons over the course of three weeks for a total of 15 lessons per Module. Teachers were able to complete approximately half of the curriculum from August 2019 to March 2020. Nearly all of the Grade 3 and Grade 5 teachers across the 3 schools covered 58% of the HMH Into Reading Modules, with the exception of School C Grade 5 teachers who covered 42% of Modules. The percentage of lessons taught within covered Modules was also calculated. In general, teachers completed between 62–78% of the lessons within each Module taught.

The Star Reading assessment was administered at the beginning of the school year to establish students' baseline scores and re-administered in December 2019 to determine achievement differences from the beginning of the year to the middle of the year. The original study design included an end-of-year administration (May 2020) but it was canceled because of COVID-19 school-closures.

A total of 387 students completed the beginning-of-year and middle-of-year Star Reading assessment. The number of questions each student answered correctly was converted to a scaled score (0–1,400). Growth from BOY to MOY was assessed using paired sample t-tests. Potential differences in scores by demographic characteristics were assessed with a repeated-measures ANOVA. Student scaled scores on the Star Reading assessment were compared from the beginning of the year to the middle of the year using a paired-samples t-test. Scaled scores on the Star Reading assessment increased significantly from beginning of year to middle of year for students in both Grades 3 and 5.

Grade 3 students’ scaled scores increased significantly from the beginning of the year to the middle of the year. The effect size of the increase was medium, indicating a practically significant effect (t(179) = 11.75, p < 0.001; Effect size: Cohen’s d = 0.51). Grade 5 student scaled scores increased significantly from the beginning of the year to the middle of the year. The effect size of the increase was small to medium, indicating a somewhat practically significant effect (t(206) = 8.97, p < 0.001; Effect size: Cohen’s d = 0.33) . (See Figure 1).

Teachers were asked how the HMH Into Reading Teacher’s Guide and digital resources were useful in helping them prepare their lesson plans and deliver instruction prior to their school building’s closure. For the Teacher’s Guide, many teachers relied on the “Week at a Glance” to help them plan for each day. One teacher wrote, “The ‘Week at a Glance’ helped me to understand what I needed to be teaching each day and what the main skills and strategies were for that week. It also helped me to know what the vocabulary words were going to be for each week.” Other teachers also noted that the “Week at a Glance” helped them manage their time. Teachers also used the Teacher’s Guide to help focus on the targeted skills for reading. Digital resources were also beneficial for teachers. They noted that the embedded program videos gave their students helpful background information at the start of each Module. The digital create feature was “great to keep us focused, and the display and engage were helpful to students.” Overall, the Teacher’s Guide and digital resources helped contribute to student learning both in the classroom and during remote learning.

Teachers reported satisfaction with HMH Into Reading and found it appropriate for different types of students. All (100% of) participating teachers believed that the curriculum was appropriate for on-grade and above-grade-level students, while 90% agreed that it was also appropriate for English learners and below-grade-level students (See Figure 2).

Overall, 90% of teachers Agreed or Strongly Agreed that they were satisfied with the HMH Into Reading program. Similarly, 90% of teachers were Satisfied or Very Satisfied with the amount of teacher preparation required to implement HMH Into Reading, and 90%of teachers reported that they look forward to using HMH Into Reading next year. In addition, 90% of teachers were Satisfied or Very Satisfied with the technology provided for student learning. Further, 85% of teachers Agreed or Strongly Agreed that HMH Into Reading provided useful information to effectively teach reading, and 80% of teachers Agreed or Strongly Agreed that HMH Into Reading was effective in increasing students' understanding of reading.

Teachers reported that their first year of implementing HMH Into Reading was a success. Specific successes included improving their own teaching skills and engagement with student learning. A few teachers reported they felt that they were better teachers by the end of implementation. For example, one teacher reported improved confidence: “I think my biggest success has been my confidence in teaching the skills to students. I really like how the digital resources are laid out and easy for teacher/student use. The anchor charts are wonderful! I felt confident going into each lesson with the knowledge needed to teach the skills effectively.” Teachers also reported success in student learning throughout the year. For example, teachers reported that students worked well in groups and became more interested in reading and their own learning.

Teachers reported high levels of satisfaction among students. Specifically, 90% of teachers Agreed or Strongly Agreed that their students were satisfied with HMH Into Reading. Teachers reported what their students liked about the HMH Into Reading program along with any changes in their motivation and engagement. For example, many students enjoyed the stories. One teacher wrote, “My students loved the variety and choices of the Modules they got to read. Every time they finished a Module they would say it was the best one, then we'd start another Module and same answer was given. They loved all the genres.” According to the teachers, students described the stories as “engaging,” “fun to read,” “interesting,” and “really funny.” Teachers noted that students were engaged with the stories because of their relevance and being able to annotate them on the computer. One teacher wrote, “Students really liked reading the stories, so they were engaged with them. Some of the questions really made them think, so that pushed my students to work harder to determine the answers.”

Teachers reported that students also enjoyed annotating their books on the computer. Although many teachers reported engagement and motivation with the digital components, one teacher wrote that it was a gradual change for this to happen: “At first, they were not eager to read and or write notes in their books. They were not used to having so much responsibility put on themselves. So it was a gradual change and excitement did happen.” Overall, the HMH Into Reading program helped engage students with reading through a digital platform, which was especially beneficial after the transition to remote learning.

Because of COVID-19, all schools within the district implemented remote learning starting in March 2020. All schools participating in the implementation study continued to use HMH Into Reading materials during this time. Document analysis of planning guides developed by participating teachers revealed that the schools taught HMH Into Reading approximately twice per week during remote learning. As a result, it took teachers much longer to cover material. For example, while the suggested time to cover each Module is typically three weeks (i.e., one lesson five days per week for three weeks), the remote learning timeline was expanded to approximately seven weeks, depending on the extent to which teachers completed each activity within the Module.

Teachers noted that use of HMH Into Reading digital resources helped to ease the transition from in-classroom to remote learning. For example, some teachers reported that their students were already adjusted to digital-based learning prior to schools closing. One teacher wrote, “Digital learning was important for all my students to learn how to navigate digital-based learning. It was very helpful after the closure, because they were already used to digital learning.”

To better understand teachers’ use of HMH Into Reading during remote learning, the posttest survey asked participants to indicate how often they continued to use student and teacher resources once in-classroom instruction had been suspended. In general, most teachers reported using student resources less often while engaging in remote learning. The student resources most commonly used during remote learning included the student myBook and VideoHub.

Most teachers used the Teacher's Guide and Teaching Pal just as frequently during remote learning. As expected, the print-only materials were not used during remote learning. Teachers also explained which of the HMH Into Reading resource(s) and Module/lesson components they found to be most beneficial during remote learning. The digital myBook was most helpful during this time as it allowed teachers to see students’ responses. In addition, the Teacher's Guide and Teaching Pal were also helpful to teachers; they used the resources to connect with students and ask them questions as they read from the student myBook. One struggle for teachers during remote learning, however, was students’ limited access to technology, as not all students had access to computers or the internet.

The current research study provided an initial understanding into the use of HMH Into Reading. Based on the research study results, the HMH Into Reading program provides good opportunities for students to learn reading skills with updated content using a variety of print and digital resources, which were especially helpful during remote learning.

Overall, curriculum implementation across study schools closely aligned to the HMH Into Reading Logic Model (Theory of Action). Teachers reported high levels of satisfaction with the HMH Into Reading program. They found most material easy to implement in their classes and appropriate for their students. Most teachers expressed that they were highly satisfied with the materials and look forward to using the curriculum in the future.

Students demonstrated positive reading growth during the study implementation, as evidence by their Star Reading assessment scaled scores, which increased significantly from the beginning of the year to the middle of the year for participating students.

Because of the COVID-19 pandemic, in March 2020 all district school buildings were closed, and the annual state test and the end of year Star Reading assessment were cancelled. As a result, the research team was not able to determine how student performance would have changed over the course of the whole school year. The district was able to pivot to remote learning and continue implementation of HMH Into Reading in a remote learning environment through the use of the programs’ digital components.

We are grateful for the participation of the research study teachers, students, and administrators. In addition to providing initial insights into the program’s implementation, teachers’ valuable feedback has already resulted in refinements to the print and digital components.

Thank you to our school district partner. We have learned so much from the teachers, students, and administrators that participated in this research study and are incredibly grateful for the opportunity to partner with such a dedicated group of educators.

Dr. Rebecca M. Eddy, PhD: Principal Investigator for HMH Into Reading Study, Cobblestone Applied Research & Evaluation, Inc.

Nicole Galport, MA: Project Manager for HMH Into Reading Study, Cobblestone Applied Research & Evaluation, Inc.

Courtney Koletar, MA: Database Manager for HMH Into Reading Study, Cobblestone Applied Research & Evaluation, Inc.

Renee Behring, EdM: Education Research Director, Houghton Mifflin Harcourt

Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246–263.

Brozo, W. G. (2009). Response to intervention or responsive instruction? Challenges and possibilities of response to intervention for adolescent literacy. Journal of Adolescent & Adult Literacy, 53(4), 277–281. https://doi.org/10.1598/jaal.5...

Bui, Y. N., & Fagan, Y. M. (2013). The effects of an integrated reading comprehension strategy: A culturally responsive teaching approach for fifth-grade students’ reading comprehension. Preventing School Failure: Alternative Education for Children and Youth, 57(2), 59–69.

Camilli, G., Vargas, S., & Yurecko, M. (2003). Teaching children to read: The fragile link between science and federal education policy. Education Policy Analysis Archives, 11(15), 1–51.

Cunningham, J. W. (2001). The National Reading Panel Report. Reading Research Quarterly, 36(3), 326–335. https://doi.org/10.1598/rrq.36...

Eddy, R. M., Galport, N., & Koletar, C. (2020, July). Houghton Mifflin Harcourt Into Reading implementation study report [Final Report]. Cobblestone Applied Research & Evaluation, Inc.

Förster, N., Kawohl, E., & Souvignier, E. (2018). Short- and long-term effects of assessment-based differentiated reading instruction in general education on reading fluency and reading comprehension. Learning and Instruction, 56, 98–109. https://doi.org/10.1016/J.LEAR...

Gibbs, K., & Beamish, W. (2020). Differentiated instruction: A programming tool for inclusion. In H. van Rensburg, & S. O’Neill (Eds.), Inclusive theory and practice in special education (pp. 174–191). IGI Global. http://doi:10.4018/978-1-7998-2901-0

Guthrie, J. T., Wigfield, A., & Perencevich, K. C. (2004). Scaffolding for motivation and engagement in reading. In J. T. Guthrie, A. Wigfield, & K. C. Perencevich (Eds.), Motivating reading comprehension: Concept-oriented reading instruction (pp. 55–86). Lawrence Erlbaum Associates Publishers.

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Ismajli, H., & Imami-Morina, I. (2018). Differentiated instruction: Understanding and applying interactive strategies to meet the needs of all the students. International Journal of Instruction, 11(3), 207-218. https://doi.org/10.12973/iji.2...

Moore, J., McClelland, S., Alef, E., & Vogel, E. (2016). The simplicity and complexity of reading comprehension. International Journal of Business and Social Science, 7(6), 1–7.

Reis, S. M., McCoach, D. B., Little, C. A., Muller, L. M., & Kaniskan, R. B. (2011). The effects of differentiated instruction and enrichment pedagogy on reading achievement in five elementary schools. American Educational Research Journal, 48(2), 462–501.

U.S. Department of Education. (2018). National Assessment of Education Progress 2017 reading report card. Institute of Education Sciences, National Center for Education Statistics. https://nces.ed.gov/nationsreportcard/

U.S. Department of Education. (2019). National Assessment of Education Progress 2019 reading report card. Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/highlights/reading/2019/

Warner-Griffin, C., Liu, H., Tadler, C., Herget, D., & Dalton, B. (2017). Reading achievement of U.S. fourth-grade students in an international context: First look at the Progress in International Reading Literacy Study (PIRLS) 2016 and ePIRLS 2016. NCES 2018-017, 36. Retrieved from https://search.proquest.com/do...