Measuring school readiness through games: A focus on reliability, validity and outcomes

By Tammy Kwan, Cognitive ToyBox, New York University and Sandra Hull, Southwest Georgia Community Action Council, Inc.

Published October 2019

Background

A Head Start program in Southwest Georgia conducted a study to assess the effectiveness of Cognitive ToyBox (CTB) in improving school readiness. CTB is an innovative hybrid observation and game-based assessment platform designed specifically for early childhood educators and by combining observation and game-based data, it allows teachers to save time on assessments and access multiple sources of data. The tool generates automated reports that provide detailed information to facilitate instructional scaffolding and personalized student support. The study involved three four-year-old classrooms in the pilot group that used Cognitive ToyBox, while three four-year-old classrooms in the control group continued using their observation-only assessment tool. The Brigance screening tool was used to measure school readiness during the pilot period.

Study Goals

  1. To determine whether teachers found CTB helpful in adjusting and individualizing instruction

  2. To understand if the tool could enhance child outcomes

Challenges

Some challenges encountered during the study included variations in sample sizes for assessment objectives, limitations in test-retest reliability calculations due to adaptive assessments, and the need for including student-led assessments to improve reliability.

Outcomes

The reliability analysis indicated high test-retest reliability for the average CTB scores, Cognition Domain, and Language/Literacy Domain. The validity analysis demonstrated a moderate correlation between the aggregate Brigance® scores and the Average CTB scores. Teachers expressed positive feedback regarding the ease of use and engaging nature of the CTB tool, while education managers found the program reports helpful in tracking progress and supporting professional development.

Based on the promising results, the Head Start program plans to implement CTB for a full year with the pilot teachers. The program will gather further feedback to refine the assessments and explore the potential for scaling the hybrid assessment model across the counties they serve. Efforts will also be made to align the CTB data with the GKIDS Readiness Check to enhance the transition from Head Start to kindergarten and provide actionable information for kindergarten teachers.

Other Research

  • Using Cognitive ToyBox at Home during Covid-19 School Closures

    Los Angeles Unified School District

    See Study

  • Technology-Based Tools Enable Greater Collaboration Between Head Start and Public Schools

    The City of Chattanooga's Office of Early Learning

    See Study

  • Measuring school readiness through games: reliability, validity and outcomes

    Head Start Program

    See Study

  • CTB: Data-Based Assessment Tool to Accelerate Progress in Early Childhood Literacy

    Davie County School System

    See Study

  • HTKS-Kids: A tablet-based measure of self-regulation to equitably assess preschoolers

    Association for Psychological Science

    Click Here

  • Supporting family child care providers and networks with Cognitive ToyBox

    Avance

    See Research

  • Implementing Cognitive ToyBox in Louisiana Center-Based Pre-K Settings

    Louisiana DOE

    See Research

  • CTB Alongside a Traditional Approach to Assessment in the Early Classroom

    WestEd

    See Research

  • The role of developmental change and linguistics in the mutual exclusivity effect

    Cognition Journal

    See Research

  • Cognitive ToyBox Subject Matter Expert Review

    WestEd

    See Research

  • The Promise of Game-Based Assessment in Early Childhood Education

    Brooklyn Kindergarten Society

    See Research

  • Week-long practice matching 2D objects improves 3D shape bias and vocabulary growth

    Cognitive Science Society

    See Research

  • Hudson Guild Case Study

    Hudson Guild

    See Research