Ch. 4: Rating Frameworks and Ratings of CASEL SELect Programs

  • This describes the framework we used to evaluate the programs we reviewed and the ratings assigned to each. A few programs provide lessons for both preschool and elementary school students. For these programs, placement in just one set of tables or both was determined based on whether they conducted research studies documenting behavioral impacts in social or academic domains with preschool and/or elementary school students.

    Description of Program Design and Implementation Support Tables

    The tables provide information about seven topics:

    • Grade range covered. For each program we list the grade levels for which there are classroom lesson plans and training materials. In some cases this includes middle and high school. However, the ratings in this Guide are based on a review of only the preschool and/or elementary school materials.
    • Grade-by-grade sequence. Ideally, every student should receive planned, ongoing, systematic SEL education every year from preschool through elementary school. Some programs provide guidance and lesson plans for preschool through grade 5 while others only target a subset of grades or involve repetition if the materials are used in multiple grades. A check mark for this element indicates when the materials allow for sequenced programming for each grade level across the grade range covered. If this element is blank, we provide additional information about grade levels covered in the program description.
    • Average number of sessions per year. The program design table indicates the average number of sessions each year, where ”session” is defined as a set of activities designed to take place in a single time period. Programs vary in terms of the amount of class time they provide or require, ranging from 8 to 140 sessions annually. Some programs do not have a defined set of lessons, and instead enhance teacher practices and methods generally. For those approaches, number of sessions or length is not applicable, since the program is designed to change the overall climate and culture through ongoing classroom instruction. These programs receive a “not applicable” (n/a) rating for this element.
    • Classroom approaches to teaching SEL. We rated three primary research-based approaches to the classroom-based promotion of SEL.
      1. Explicit SEL skills instruction.
      2. Integration with academic curriculum areas.
      3. Teacher instructional practices.
    • Opportunities to practice social and emotional skills. Practicing newly learned behaviors is an essential component for developing social and emotional skills (Durlak et al., 2010; Durlak et al., 2011). Practice that takes place outside the lesson in real-world settings has the potential to be especially powerful. By definition all SELect programs provide students with opportunities to practice SEL skills. The rating for this element reflects the extent to which the programs provide active learning opportunities during or beyond classroom sessions. Programs received the highest rating if they provided consistent opportunities for practice of skills both within classroom lessons and beyond lessons in daily situations. Practice within the program typically includes role plays or guided self-management techniques. Practice beyond the program lessons includes applications of social and emotional skills to real-life situations, such as using self-calming or problem-solving skills during classroom or playground conflicts. The mid-level rating was given if programs provided these opportunities only during program sessions. Given the inclusion criteria, no programs received the lowest rating on this element.
    • Contexts that promote and reinforce SEL. Because of the importance of promoting and reinforcing SEL skills across multiple settings, each program was rated for the extent to which it provided practices for extending its concepts into four different contexts:
      (1) The classroom beyond the SEL program lessons
      (2) School-wide
      (3) The family, and
      (4) The community.
    • Assessment tools for monitoring implementation and student behavior. Programs sometimes offer tools to monitor implementation, either through teacher self-report measures or assessments completed by observers. Two columns in the table indicate whether or not each program provides these tools. The third column for this element indicates whether the program offers tools that can be used to assess the program’s impact on student behavior.

    Description of the Evidence of Effectiveness Tables

    These tables present information and ratings for four topics: grade range covered, characteristics of research sample, study design, and evaluation outcomes. Additional details about the program evaluations can be found in the individual program.

    • Grade range covered. For each program we list the grade levels for which there are classroom lesson plans and training materials. In some cases this includes middle and high school. However, the ratings in this Guide are based on a review of only the preschool and/or elementary school materials.
    • Characteristics of sample. The ratings for this element reflect four characteristics of the study sample in the qualifying evaluations: the grade levels, the geographic locations (urban, suburban, rural) where the studies were conducted, student race/ethnicity, and the percentage of students receiving free or reduced lunch included in the study samples. Evaluators sometimes used “diverse” to indicate the race/ethnicity of the sample, rather than providing specific information. Many of the preschool evaluation studies were conducted in Head Start programs. Given the income eligibility levels set by Head Start, we assumed all (100%) participants in those studies qualified for free and reduced lunch.
    • Study design. Ratings for this element are presented across two columns. They indicate whether there were quasi experimental or randomized clinical trials and how many of each.
    • Evaluation outcomes. Evaluation outcome ratings are based on the outcomes reported in at least one qualifying evaluation study. The ratings for this element are represented in four columns. They represent the outcome domains that were reviewed for program inclusion. SELect programs had to demonstrate a positive impact on a behavioral or academic performance indicator in at least one of the domains. A check mark indicates that a significant program effect was documented on an outcome in that domain as measured by observations, school records, or ratings made by teachers, parents, or students. The definitions of each outcome domain are:
      (1) Improved academic performance
      (2) Improved positive social behavior
      (3) Reduced conduct problems
      (4) Reduced emotional distress