Elsevier

Early Childhood Research Quarterly

Volume 54, 1st Quarter 2021, Pages 86-98
Early Childhood Research Quarterly

Measuring preschool quality in low- and middle-income countries: Validity of the ECERS-R in Colombia

https://doi.org/10.1016/j.ecresq.2020.08.001Get rights and content

Highlights

  • We examined the psychometric properties of the ECERS-R in Colombia.

  • Results obtained with the ECERS-R were characterized by low scores and variability.

  • Principal component analyses did not identify seven factors, but three factors.

  • The effect sizes of the ECERS-R factors on the developmental gains were modest.

  • Small increases of quality showed potential to improve children's development.

Abstract

As preschool enrollment has spread across the globe, many researchers have turned to the Early Childhood Environment Rating Scale-Revised (ECERS-R) to assess the quality of preschool in low and middle-income (LAMI) countries. However, questions remain about the pertinence of this measure in LAMI countries due to evidence of low variability and extreme low scores across items. This study evaluated the psychometric properties of the ECERS-R in a Colombian sample of 174 classrooms from which 668 four-year-old children were selected for developmental assessment. Principal Component Analysis failed to identify the seven dimensions indicated in the original structure. Analyses instead found three dimensions in the Colombian sample: materials and spaces, interactions, and routines and practices. The predictive validity of these dimensions was not strong, but suggested that better routines and interactions are associated with children's development. Results and implications for the development of future measures are discussed.

Section snippets

Quality of early childhood education

A large number of studies shows that early childhood interventions can be cost-effective in reducing inequities and improving academic achievement (Heckman, 2011). Preschool programs, in particular, promote development in early childhood (Almond, Currie, & Duque, 2018; Heckman & Mosso, 2014; Yoshikawa et al., 2013). Results from experimental and quasi-experimental studies document the benefits of high-quality preschool programs (Black et al., 2017; Britto et al., 2017; Campbell & Ramey, 1994;

Using the ECERS-R to assess Early Education Quality

ECERS-R is one of the most commonly used instruments around the globe for assessing preschool quality. It is an observational measure that consists of 43 items to measure quality across seven dimensions: 1. space and furnishing, 2. personal care routines, 3. language and reasoning, 4. activities, 5. interaction, 6. program structure, and 7. parents and staff. Noteworthy, although most of the items assess availability of material resources (e.g. books, toys), they do not include some important

Preschool in Colombia

While for most of the period since the late 1980s access to public early education programs ranged from 20% to 40% for low income children in Colombia (Bernal & Camacho, 2011), this figure radically increased in recent years after the government launched the national early childhood strategy De Cero a Siempre (DCAS, From Zero to Forever) in 2011 (Bernal & Ramírez, 2019). This initiative was aimed at increasing access and quality of early childhood education offered to socio-economically

The present study

The goal of this study is to strengthen the knowledge about the psychometric properties of the ECERS-R in LAMI countries by considering how it performs in two studies in Colombia. Specifically, this study had two aims. First, we tested the structural validity of the ECERS-R in Colombia. Second, we examined whether the ECERS-R items could be arranged in a different factor structure with good predictive qualities. To achieve this second aim, we used theoretical and statistical criteria to develop

Sample

The present study included 147 preschool classrooms in Colombia from which 668 children were selected. The ECERS-R was used to assess preschool classroom quality in two studies. The first is the Early Education Quality in Bogota (EEQB) study (Maldonado-Carreño & Votruba-Drzal, 2014), which was conducted in Bogota, the capital of Colombia. The second is the Impact Evaluation of Child Development Centers (IACDC) study (Bernal et al., 2019), which was conducted in 14 medium-to-large cities across

Structural validity of the ECERS-R

Descriptive statistics for the ECERS-R items are shown in Table 2. The mean scores on each item were generally low, ranging from 1 to 2. This indicates that quality was “inadequate” according to ECERS-R standards. Although generally all items had low scores, it is important to highlight that items related with Interactions and Parents and staff had higher scores than the other items on the instrument.

The first goal of this study was to test whether the original ECERS-R structure of seven

Discussion

To our knowledge, this is the first study in Latin America to analyze the structural and predictive validity of the ECERS-R, although it has been previously used for program evaluation across the region (Aparicio, Wieler, & Elías, 2011; Araujo, López-Boo, Novella, Schodt, & Tomé, 2015; Kagan, 2011). First, this study considered whether the original structure of the ECERS-R was reproduced in the sample, finding no evidence of the seven-dimension in the Colombian context. Second, it examined

Limitations

It is important to note several limitations to this study. First, although our sample allowed for reasonable precision, our estimates would have been more precise with a larger sample. Additionally, both samples include direct assessments of only a few children from each classroom, and it is impossible to know how well these represent the experiences of other children in the classroom. Moreover, both of the studies included in this analysis are comprised of settings serving low-income families,

Conclusion

The findings of this study support the increasing attention that scholars and policymakers are giving to assess, adapt, and develop measures of preschool quality for global contexts (Anderson and Sayre, 2016, Unicef, 2019). Our results suggest that quality measurement needs to be broadened to conceptualize quality in high-poverty contexts. Developing measures that are responsive to the policies of LAMI countries and their material resource availability is vital to making informed decisions that

Credit author statement

Laura Betancur: Conceptualization, methodology, formal analysis, writing – original draft preparation

Carolina Maldonado-Carreño: Provision of study materials, methodology, validation, writing – reviewing and editing

Elizabeth Votruba-Drzal: Conceptualization, methodology, supervision, writing – reviewing and editing

Raquel Bernal: Provision of study materials, methodology, writing – reviewing and editing

References (98)

  • D.M. Early et al.

    Factor structure and validity of the Early Childhood Environment Rating Scale – Third Edition (ECERS-3)

    Early Childhood Research Quarterly

    (2018)
  • S. Holloway et al.

    Assessing child-care quality with a telephone interview

    Early Childhood Research Quarterly

    (2001)
  • K. Li et al.

    Quarterly Chinese Early Childhood Environment Rating Scale (trial) (CECERS): A validity study

    Early Childhood Research Quarterly

    (2014)
  • M.W. Lipsey et al.

    Effects of the Tennessee Prekindergarten Program on children's achievement and behavior through third grade

    Early Childhood Research Quarterly

    (2018)
  • L. Malmberg et al.

    Effects of a preschool intervention on cognitive development among East-African preschool children: A flexibly time-coded growth model

    Early Childhood Research Quarterly

    (2011)
  • D. Mayer et al.

    Examining the validity of the ECERS-R: Results from the German national study of child care in early childhood

    Early Childhood Research Quarterly

    (2016)
  • M. Perlman et al.

    Examining the psychometric properties of the Early Childhood Environment Rating Scale-Revised (ECERS-R)

    Early Childhood Research Quarterly

    (2004)
  • L. Phillipsen et al.

    The prediction of process quality from structural features of child care

    Early Childhood Research Quarterly

    (1997)
  • S. Sheridan et al.

    A cross-cultural study of preschool quality in South Korea and Sweden: ECERS evaluations

    Early Childhood Research Quarterly

    (2009)
  • R. Smith-Donald et al.

    Preliminary construct and concurrent validity of the Preschool Self-Regulation Assessment (PSRA) for field-based research

    Early Childhood Research Quarterly

    (2007)
  • J. Squires et al.

    Identification of social-emotional problems in young children using a parent-completed screening measure

    Early Childhood Research Quarterly

    (2001)
  • S. Wolf et al.

    Measuring and predicting process quality in Ghanaian pre-primary classrooms using the Teacher Instructional Practices and Processes System (TIPPS)

    Early Childhood Research Quarterly

    (2018)
  • D. Almond et al.

    Childhood circumstances and adult outcomes: Act II

    Journal of Economic Literature

    (2018)
  • K. Anderson et al.

    Measuring early learning quality and outcomes in Tanzania

    (2016)
  • A. Andrew et al.

    Preschool quality and child development

    (2019)
  • R. Aparicio et al.

    Resultados de la evaluación de efectividad del preescolar 2005

    (2011)
  • M.C. Araujo et al.

    Daycare services: It’s all about quality

  • M.C. Araujo et al.

    La calidad de los Centros Infantiles del Buen Vivir en Ecuador

    Resumen de Políticas, IDB-PB-248

    (2015)
  • E.K. Beller et al.

    Two measures of the quality of group care for infants and toddlers

    European Journal of the Psychology of Education

    (1996)
  • R. Bernal et al.

    The importance of early childhood programs in Colombia

    (2011)
  • R. Bernal et al.

    Early Childhood Policy in the Context of Equity and Social Mobility in Colombia

  • R. Bernal et al.

    Evaluación de impacto del Programa Hogares Comunitarios de Bienestar del ICBF

    Documento CEDE

    (2009)
  • J.B. Bowne et al.

    A meta-analysis of class sizes and ratios in early childhood education programs: Are thresholds of quality associated with greater impacts on cognitive, achievement, and socioemotional outcomes?

    Educational Evaluation and Policy Analysis

    (2017)
  • P. Britto et al.

    Quality of early childhood development programs in global contexts. Rationale for investment, conceptual framework and implications for equity

    SRCD Social Policy Report

    (2011)
  • R. Bull et al.

    Assessing quality of kindergarten classrooms in Singapore: Psychometric properties of the early childhood environment rating scale—revised

    International Journal of Early Childhood

    (2017)
  • M. Burchinal

    Differentiating among measures of quality: Key characteristics and their coverage in existing measures

    (2010)
  • M. Burchinal

    Measuring early care and education quality

    Child Development Perspectives

    (2018)
  • M. Burchinal et al.

    Quality rating and improvement systems: Secondary data analyses of psychometric properties of scale development

    (2016)
  • M. Burchinal et al.

    How well do our measures of quality predict child outcomes? A meta-analysis and coordinated analysis of data from large-scale studies of early childhood settings

  • F.A. Campbell et al.

    Effects of early intervention on intellectual and academic achievement: A follow-up study of children from low-income families

    Child Development

    (1994)
  • M.M. Campos et al.

    Early childhood education quality: A Brazilian study in six capitals

    Cadernos de Pesquisa

    (2011)
  • Comisión Intersectorial para la Atención Integral de la Primera Infancia; CIPI

    Estrategia de atención integral a la primera infancia. Fundamentos políticos, técnicos y de gestión

    (2013)
  • R.M. Clifford et al.

    What is pre-kindergarten? Characteristics of public pre-kindergarten programs

    Applied Developmental Science

    (2005)
  • J. Currie et al.

    School quality and the long-term effects of head start

    Journal of Human Resources

    (2000)
  • Departamento Nacional de Planeacion

    De Cero a Siempre. Informe de resultados de la evaluacion y tercera entrega de la documentacion de la base de datos de la evaluacion

    (2018)
  • P. Engle et al.

    Child development in developing countries: Strategies to avoid the loss of development potential in more than 200 million children in the developing world

    Lancet

    (2007)
  • M. Fenech

    An analysis of the conceptualisation of “quality” in early childhood education and care empirical research: Promoting “blind spots” as foci for future research

    Contemporary Issues in Early Childhood

    (2011)
  • L. Fabrigar et al.

    Evaluating the use of exploratory factor analysis in psychological research

    Psychological Methods

    (1999)
  • L.C.H. Fernald et al.

    Examining early child development in low-income countries

    (2009)
  • Cited by (13)

    • Family engagement in early learning opportunities at home and in early childhood education centers in Colombia

      2022, Early Childhood Research Quarterly
      Citation Excerpt :

      ICBF, 2020a). Regarding pedagogical quality in ECCE centers, on average, centers had a score of 2.3 on a scale of 1–4, indicating medium to low levels of quality that are consistent with previous assessments of process quality in smaller samples in Colombia using other tools such as the ECERS-R (Betancur et al., 2021). What are levels of home- and center-based family engagement in children's early learning opportunities in a national sample of public ECCE in Colombia?

    View all citing articles on Scopus
    View full text