July 2, 2008

An Exploratory Study of the Application of Early Childhood Environment Rating Scale Criteria

By Warash, Barbara G Ward, Corina; Rotilie, Sally

This study examined whether attending a one day training on the Early Childhood Environment Rating Scale-Revised (ECERS-R) corresponded to pre-k classroom changes. Teachers attended an ECERS- R module training and six months later completed a questionnaire to report any classroom changes. The questionnaire consisted of listing the subscales and items of the ECERS-R. The teachers marked items they changed and commented on how they improved their classroom. Although there were limited modifications, teachers reported changes in their classrooms after learning more about the ECERS-R. Small positive changes by teachers to improve their classroom environment are a step towards quality. Introduction

In 2005,61% of children in the United States under the age of 6 were enrolled in some type of nonparental child care (Federal Interagency Forum on Child and Family Statistics, 2006). However, many of these settings were not considered to be of high quality (Cryer, 1999). This is a concern because the quality of care young children receive has been shown to be linked to many developmental outcomes. The Cost, Quality, and Outcomes Study of Peisner-Feinberg et al. (1999) found that high quality care is positively related to school performance including cognitive, language, and social development.

In 2002, the West Virginia Legislature passed W. Va. Code [section]18-5-44, which requires the West Virginia Board of Education to fully implement a voluntary pre-k system by 2012-13 for all 4-year-olds. In response to the legislation, West Virginia Pre- K was established and beginning in September, 2002, a 10 year phase- in plan began. County school districts were given the flexibility to design their approach within the framework of West Virginia Board of Education Policy 2525, West Virginia's Universal Access to a Quality Early Education System (2005).

In order to support the maximization of existing resources, county school districts must contract for pre-k services with community programs, including Head Start and child care to provide a minimum of 50% of their pre-k services. This approach ensures the continued use of other programs that are designed to serve preschool children and their families. County collaborative teams are established that include, public school, Head Start, child care and private nursery school providers to develop annual plans specific to the needs and vision of each community.

According to National Institute for Early Education Research, West Virginia is ranked sixth in the nation for accessibility for 4- year-olds and 8th in the nation for accessibility for 3-year-olds. The state supports WV pre-k primarily through the public school funding formula and is ranked 10th in the nation for resources (Barnett, Hustedt, Robin, & Schulman, 2006). This state funding is supplemented by significant amounts of federal funding including Head Start, Title I and the Individual with Disabilities Act (IDEA).

The collaborative requirement is met through an agreement established by county pre-k planning teams that include, at a minimum, representation from: the county school system preschool program, the county school system preschool special needs program, a licensed community child care program in that county not operated by the county school system, and the Head Start program in that county. The teams must submit an annual plan that describes how they are moving toward full access for all 4-year-olds plus 3-year-olds with disabilities. This planning requirement supports the development of quality programming for preschool children in line with the standards set forth under Policy 2525.

Policy 2525 establishes requirements for quality programming, including standards for curriculum and assessment, health and safety, class size and teacher/child ratios, parent involvement, professional development and staff qualifications. A study published by the Edward Zigler Center for Child Development and Social Policy found that West Virginia's preschool teachers are among the most educated in the nation (Gilliam & Zigler, 2000). Sixty-six percent of preschool teachers in the state have at least a master's degree (Gilliam & Marchesseault, 2005).

To support the continuous quality improvement of classroom practices, Policy 2525, requires that each classroom be observed annually using the Early Childhood Environment Rating Scale-Revised (ECERS-R; Harms, Clifford, & Cryer, 1998). Scores are obtained for each classroom and submitted to the West Virginia Department of Education Coordinator for Early Childhood. An annual development plan must accompany each set of scores detailing the county's plan for improvement.

A three module training sequence was developed by a collaborative team of WV early childhood professionals to support local communities in their efforts to build cadres of valid and reliable ECERS-R observers. The developers of the ECERSR (Harms, Clifford, & Cryer, 1998) state that it is important to be accurate in using the ECERS-R whether it is used by individual teachers for self assessment or as an outside observer for program monitoring, program evaluation, program improvement, or research. Module One of the three training series, consists of small group discussions, handouts, demonstrations and modeling, observation of actual practice, and video presentations and are congruent with Wolfe's (1992) suggestions for adult pedagogy. Forms of trainings that meet the teachers' needs, receive support from administrators, and use different strategies in teaching have been found to be the most beneficial for improving classroom quality and teacher's beliefs and practices (Wolfe, 1992).

Module One training of the ECERS-R is offered to staff, teachers and administrators as a staff development opportunity. Module One of the training sequence involves a six hour session to:

* Develop an understanding of the structure and purpose of the ECERS-R.

* Develop an understanding of how the ECERS-R is scored.

* Practice observing and scoring by responding to video clips.

* Develop an understanding of the ECERS-R through an examination of the items using All About the ECERS-R (Cryer, Harms, & Riley, 2003).

* Develop an understanding of how the ECERS-R supports the implementation of the three approved state curriculums.

In order to enhance awareness and understanding of the ECERS-R and to gain knowledge of how to establish quality classroom environments, several WV Pre-K teachers chose to participate in Module One of the training sequence. Accurate use and familiarity of the scale is essential to ensure that teachers are using it correctly and therefore opening the opportunity to improve their classrooms according to the criteria of the ECERS-R. It's reasonable to assume if teachers are not familiar with the scale, it would be hard to make changes accordingly. Training teachers on the ECERS-R would seem to be a valid mechanism to improving classroom quality as research (Kendrick, 1994; Simpson, 2004; Nelson, 1998; Ruopp, Travers, Glantz, & Coelen, 1979; Bryant et al., 1997) indicates that training is directly related to quality. Even small amounts of specialized training have been shown to positively influence caregiver behavior which is one aspect of high quality care (Kendrick, 1994).

In a previous study, Warash, Markstrom, & Lucci, (2005), examined the effects of using the ECERS-R as a training tool to help improve the quality of early childhood classrooms. The purpose of this study was to determine if overall scores on the ECERS-R would improve after the initial scores and individual improvement plans for classrooms were reviewed by child care directors who subsequently reviewed them with their teachers. An individualized training plan was developed using the ECERS-R as a guide to show teachers what could be done to improve their scores on the scale. The individualized plan included the scores of the first ECERS-R and what would be required to improve them. Some of the suggestions were related to physical aspects of the room, such as adding a sand/ water table. On the other hand, some suggestions on the individualized plan required more specific training such as setting up a series of workshops that promoted age appropriate interactions. After six months a second ECERS-R was conducted so that pre- and posttest scores of the ECERS-R could be compared to determine if significant changes occurred. Results showed that there were significant increases in subscale scores for Personal Care Routines, Interaction, and Activities, as well as overall scores.

Policy 2525 requires that teachers obtain a minimum of 15 hours of ongoing training each year that is related to curriculum and inclusion. The ECERS-R training can be chosen as part of this requirement. In addition, there are classes the WV State Department of Education, the West Virginia Training Connections, West Virginia Department of Human Services and the Head Start State Collaboration Office provide for free or inexpensively at major colleges and universities. An initial study (Warash, Curtis & Morgan, in press) on the preschool inclusion classes held at three WV institutions found that early childhood personnel changed their pre-established attitudes about inclusion and became more focused on an "intentional" teaching approach to help all children become competent learners. Just as in the Warash et al. (2005) study, where directors used the ECERS-R scores to help teachers improve their classrooms, this study looked at the effectiveness of directly training the teachers about the ECERS-R. This exploratory study examined the reported changes that individual teachers made after completing Module One of the ECERS-R. This training was based theory of Wolfe's (1992) ideas on delivery of content for adult learners. Teachers need awareness and knowledge about what influences their work and how they can use opportunities to create challenging and stimulating environments for the children (Sheridan, 2001). The question in this exploratory study is, "After the completion of Module One of the ECERS-R, will teachers be prompted to make classroom changes.


Participants and Procedure

Data was collected from WV teachers working in Pre-K classrooms who participated in the Module One training of the ECERS-R. The types of classrooms represented in this sample were Head Start, Preschool Special Needs, Public School Pre-K, Child Care, or classrooms combining two or more of these characteristics.

Teachers and administrators of preschool classrooms were invited to attend the Module One of the ECERS-R training as a staff development opportunity. The trainer was taught by Dr. Thelma Harms, one of the authors of the ECERS R. Six months after attending the training, 35 participants were emailed a questionnaire designed by the trainer. Eleven teachers returned the questionnaire.

The training covered each subscale on the ECERS-R, gave guidance on how to accurately score the subscales, showed examples of what is considered appropriate, and indicated what is needed in each subscale to obtain the highest score. In addition, the six hour training covered the structure and purpose of the ECERS-R, with an in-depth look at the scale. The participants experienced each item on the ECERS-R by using the book, All About the ECERS-R (Cryer, Harms, & Riley, 2003). The training consisted of small and large group discussions, questions, and watching video clips to practice observing and scoring. Participants were emailed the questionnaire by the trainer of the module six months after the training. They were not told they would be receiving the questionnaire when they were at the training. Participants emailed the completed questionnaire back to the trainer. A reminder email was sent to the participants two months after the questionnaire was first sent out.


The Early Childhood Environmental Rating Scale-Revised (Harms et al., 1998) is a widely used tool designed to assess process quality in early childhood care settings such as daycares, preschools, child care facilities, and Head Start programs, as well as many other early childhood environments (Roscoe, 2005). The scale has been, and continues to be, utilized in a number of other ways such as: teacher training, overall quality of an environment, government regulations, and overall program improvement. ECERS-R has been used in many studies, has high reliability, high validity, and has a long history of research showing that quality as measured by the ECERS has predictive validity. Further, the scale has been shown to be reliable at the indicator and item level, as well as the total score level (Harms et al., 1998).

The ECERS-R consists of seven subscales that rate features of the early childhood environments that directly affect children and adults in those settings. These subscales include: space and furnishings, personal care routines, language-reasoning, activities, interaction, program structure, and parents and staff. There are a total of 43 items that are organized into the above subscales. Each item is scored on a 7 point scale with descriptors of what is needed for 1 (inadequate) up to 7 (excellent). When all items are scored, they can be either totaled separately for the seven subscales or all subscales together as an overall score.


A questionnaire was developed by the trainer/early childhood consultant who also helped to develop the training modules. It was designed to seek retrospective information from the teachers to determine if the training helped them become familiar with the ECERS- R and in turn make classroom changes. The questionnaire consisted of listing all of the subscales and scoring items from the ECERS-R. A space was provided next to each item so the participants could indicate specific changes that were made. At the end of the questionnaire there were questions concerning the characteristics of the classroom they taught in (Head Start, Preschool Special Needs, Child Care, Public Pre-K or a combination of two or more), how long they had been teaching, how long they had been teaching preschool, and what age group they were teaching. The completed questionnaires were returned via email. They were downloaded and names deleted from the questionnaires.


The 11 participants had been teaching for an average of 4.6 years. The number of years teaching preschool ranged from 1 year to 25 years. Two teachers were from Head Start, two from child care, two taught in public school pre-kindergarten and four taught in a combination of the preceding. The children in their classes were between the ages of 4 and 5.

Calculations were made by counting the number of items on each subscale, the number of possible responses, and the reported responses. Percentages were calculated based on the ratio between reported changes and possible changes to determine what items were marked as changed most frequently. This also allowed comparison of response frequencies between subscales. The comparison showed which subscales and what items were the most impacted after attending the training.

The Space and Furnishing subscale was reported as the highest changed subscale at 36.36%. The percentages for the remainder of the subscales are as follows: Personal Care Routine subscale at 21.21%, Activities at 20.91%, Program Structure at 18.18%, Language and Reasoning at 11.36%, Parents and Staff at 7.58%, and the least reported change was the Interaction subscale at 7.27% (see Table 1).

Changes were found in 36 of 43 items. Of the 43 items within all the subscales the most reported changed item occurred in item #3, furnishings for relaxation and comfort at 63.63%. Item #4, room arrangements for play, item #5, space for privacy, item #34, schedule, and item #28, promoting acceptance of diversity all had reported changes at 45.45%. Item #1, indoor space and item #13, health practices had reported changes at 36.36% (see Table 2). Table 3 lists the reported changes for the seven subscales and corresponding items in each of the subscales. The items that had no reported changes included: item #39, provisions for personal needs of staff, item #42, supervisions and evaluations of staff, item #43, opportunities for professional growth, item #30, general supervision of children, item #31, discipline, item #19 fine motor (activities), and item #23, sand/water (see Table 3).


The training was intended to help teachers improve the quality of their classrooms. It was found that teachers made some classroom changes after the attendance of Module One on the ECERS-R. Changes were reported on 36 of the 43 items of the ECERS-R by the teachers that responded to the questionnaire. Even though teachers reported relatively few changes to their classrooms, small modifications by the teachers were considered an attempt to improve the quality of their classrooms.

Most of the changed items were reported under the Space and Furnishing subscale. These included: item #3, furnishings for relaxation and comfort, item #4, room arrangements for play, item #5, space for privacy, and item #1, indoor space. Item #3, furnishings for relaxation and comfort, was reported as changed by over half of the teachers. This is not an unexpected finding due to the fact that changing physical items is easier to change than an item that requires more cognitive processes (Warash et al., 2005). For example, adding stuffed animals and "soft" items to an area, as the participants commented, is an easy task compared to changing item #17, using language to develop reasoning skills in the Language and Reasoning subscale. This takes more cognitive effort than just adding a stuffed animal to a room.

Items #4, #5, and #1, room arrangements for play, space for privacy, and indoor space, respectively, also fell under the Space and Furnishing subscale. Teachers commented that they had changed these items by arranging more open spaces, allowing for spaces between play areas for private time, and rearranging furniture to reduce the noise level. One teacher commented that she "added a 'box' work space," while another teacher was "in the process of ordering a canopy and some pillows to more clearly define the privacy area." The high frequency of reported change in this subscale may also be related to the fact that changing one area resulted in changes in another. For example, one teacher commented that she had changed her (item #1) indoor space by making more open space. By doing this she also changed her (item #4) room arrangements for play by maximizing the indoor space. While making open spaces, she also made sure that there was space between the play areas so the children could have private time, which is space for privacy, (item #5). Again, these changes are considered physical changes and are easier to accomplish.

Item #34, schedule, of the Program Structure subscale was changed by nearly 50% of the teachers. These teachers commented that they changed this item by increasing the amount of free play. Preschool children need quality free play as emphasized in the training video (Harms, & Cryer, 1999). Quality free play is an essential component of the approved preschool curriculums in West Virginia (Universal Access to a Quality Early Education System, 2005). All the teachers have had previous training on their respective curriculums, but the reported change of having more free play may have resulted from using a reliable and respected source such as the ECERS-R to help validate the necessity of play. The ECERS-R states that a substantial portion of the program must include free or unstructured play (Harms et al., 1998). The amount of free play in many preschools is limited, because some teachers do not always understand the value of play (Warash et al., 2005). Although increasing the amount of time for free play does not always occur in a quality environment, it is a start to the importance of play and it is a tangible item to fix. Promoting acceptance of diversity, item #28, under the Activities subscale, was also changed by half of the participating teachers. Teachers said that they changed this item by adding more multicultural dolls, books, and puzzles to their classrooms. Item #13, health practices, under the Personal Care and Routine subscale, was the last of the seven items with more than 30% of the participants reporting changes. Establishing a routine for hand washing and emphasizing good hygiene were the ways in which the item was modified. This was an area that was stressed in the video clips during the training module mainly because of its importance for the health of the teachers and the children (Harms, & Cryer, 1999). Hand washing is such a basic health issue and is also an easily implemented change to daily classroom routines. The two subscales that had the lowest reported changes were Parents and Staff at 7.58% and Interactions at 7.27%.

There were additional changes that teachers wrote as comments on their questionnaire. For example, item # 6, child-related display, was reported as changed by 27.27% of teachers by adding more 3D projects to the curriculum and displaying art work on the wall at children's eye level. Item #21, music/movement, also at 27.27%, was changed by adding music centers and allowing the use of more instruments. These changes are assumed to be the result of implementing the suggestions and ideas that were part of the training module.

The results of this study are similar to previous studies that have shown that trainings, even relatively short trainings, help teachers become aware of what is needed for high quality classrooms and positively influence caregiver behavior (Kendrick, 1994; Simpson, 2004; Nelson, 1998; Ruopp, Travers, Glantz, & Coelen, 1979; Bryant et al., 1997). Attending Module One of the ECERS-R could have prompted teachers to make changes to their classrooms.

Because this study used a self-report questionnaire, we can only assume that the teachers did actually change the items that they indicated on the questionnaire. Some classrooms changes might have occurred regardless of the training. Teachers who attended but did not fill out the questionnaire could have also changed some items within the six month period. We do not know if the other participants made changes.

Regardless of the sample size, those who completed Module One training of the ECERS-R did report classroom changes. The teachers who attended the training module and responded to the questionnaire, seemed to make small, positive changes in their classroom environments, as measured by the ECERS-R. This indicates that completion of the Module One training is associated, to some extent, with teachers' attempts to improve the quality of their classrooms.


Barnett, W.S., Hustedt, J.T., Robin, K.B., Schulman, K.L. (2006). The state of preschool: 2006 state preschool yearbook. New Brunswick, NJ: National Institute for Early Education Research.

Bryant, D., Maxwell, K., Burchinal, P., & Lowman, B. (1997). The effect of Smart Start on the quality of preschool child care. Report to the Department of Human Resources by the Smart Start Evaluation Team, Frank Porter Graham Child Development Center: University of North Carolina at Chapel Hill.

Cryer, D. (1999). Defining and assessing early childhood program quality. Annals of the American Academy of Political and Social Science, 563, 39-55.

Cryer, D., Harms, T., & Riley, C. (2003). All about the ECERS-R. New York: Teachers College Press.

Federal Interagency Forum on Child and Family Statistics. (2006). Population and Family Characteristics in America's Children in Brief: Key National Indicators of Well-Being 2006. Retrieved October 20, 2006 from http://www. childstats.gov/americaschildren/pop.asp.

Gilliam, W.S. & Marchesseault, C.M. (2005). Part 1: Who's teaching our youngest students? Teacher education and training, experience, compensation and benefits, and assistant teachers. Unpublished manuscript. Yale University Child Study Center. Retrieved from NIEER online: http://nieer.org/resources/files/ NPSteac hers.pdf

Gilliam, W.S. & Zigler, E.F. (2000). A critical meta-analysis of all evaluations of state-funded preschool from 1977-1998: Implications for policy, service delivery and program evaluation. Early Childhood Research Quarterly, 15, 441-473.

Harms, T., Clifford, R. M., & Cryer, D. (1998). Early childhood environment rating scale: Revised edition. New York: Teachers College Press.

Harms, T., & Cryer, D. (1999). Video observations for the ECERS- R. New York: Teachers College Press.

Kendrick, A. S. (1994). Training to ensure healthy child day- care programs. Pediatrics, 94, 1108-1110. http://nieer.org/ resources/research/ multistate/wv.pdf

Nelson, D. W. (1998). Confronting the child-care challenge. Governing, 12, 44-49.

Peisner-Feinberg, E.S., Burchinal, M.R., Clifford, R.M., Howes, C., Kagan, S.L., Yazejian, N., Byler, P., Rustici, J., & Zelazo, J. (1999). The children of the cost, quality, and outcomes study go to school: Technical report. Chapel Hill: University of North Carolina at Chapel Hill, Frank Porter Graham Child Development Center.

Roscoe, S. (2005). Introduction to the Harms, Clifford and Cryer early childhood environment rating scales. In Environment Rating Scales. Retrieved from http://www.fpg.unc.edu/~ ecers/

Ruopp, R.R., Travers, J., Glantz, F., & Coelen, C. (1979). Children at the center. Cambridge, MA: Abt Associates.

Sheridan, S. (2001). Quality evaluation and quality enhancement in preschool: A Model of Competence Development. Early Child Development and Care, 166, 7-27.

Simpson, A. (2004), Number of high-quality preschool and child care programs continues to grow. Retrieved October, 20, 2006 from http://www.naeyc.org/about/releases/2004120 2.asp.

Universal Access to a Quality Early Education System. [section] 126-28, West Virginia. (2005).

Warash, B., Markstrom, C., & Lucci, B. (2005). The Early Childhood Environment Rating Scale-Revised as a tool to improve child care centers. Education, 126, 240-250.

Warash, B., Curtis, R., & Morgan, K. (in press). Training on best practices: Impacting opinions of early childhood teachers on inclusive preschools. Journal of Early Education and Family Review.

West Virginia Department of Education. (2005). West Virginia Board of Education continues to strengthen early childhood education. Retrieved October 18, 2006 from http://wvde. state, wv.us/ news/1029/.

Wolfe, B. Presentation at the early childhood profession coming together, from the First Annual Conference of NAEYC's National Institute for Early Childhood Professional Development. June 4, 1992, Los Angeles.




Child Development and Family Studies

Department of Technology, Learning, and Culture

West Virginia University

Copyright Project Innovation Summer 2008

(c) 2008 Education. Provided by ProQuest Information and Learning. All rights Reserved.