Factor Structure, Internal Consistency, and Interrater Reliability of the Early Childhood Behavior Problem Screening Scale

By Nelson, Ron Epstein, Michael; Griffith, Annette; Hopper, John

The current study examined the initial psychometrics of the Early Childhood Behavior Problem Screening Scale (ECBPSS; Epstein & Nelson, 2006), namely the factor structure and associated internal consistency of factor items of parent and teacher versions as well as interrater reliability. Data came from samples of preschool- and kindergarten-age children from 2 medium-size cities in the U.S. midwest. Separate analyses of the parent and teacher data revealed internalizing and externalizing factors, with Cronbach’s alphas ranging from .83 to .95. Parent and teacher interrater reliability coefficients ranged from .32 to .37. Overall, the results suggest that the 12-item ECBPSS Parent and Teacher forms may be useful for screening young children who may be at risk for emotional and behavioral disorders. Traditional identification methods for special education are based on a “wait-to-fail” model that provides students with services only once they have begun to have difficulties in the classroom that lead to impairments in learning or social behavior (Albers, Glover, & Kratochwill, 2007). Fortunately, there has been a recent push toward identifying students early and providing them early intervention services before their difficulties lead to school failure (Fairbanks, Sugai, Guardino, & Lathrop, 2007). For students with emotional and behavioral disorders, early identification is particularly important, as it is often not until a child’s behavior becomes unmanageable by classroom teachers that referrals for emotional and behavioral disorders occur (Gresham, 1991). This is unfortunate, because problem behavior tends to become more severe and resistant to intervention over time (Kraemer et al., 1997). Furthermore, research has indicated that early problem behavior is highly related to school success (Gresham, Lane, & Lambros, 2000), and students who demonstrate problem behavior at school are likely to experience academic difficulties that begin early and continue throughout their school careers (Patterson, Reid, & Dishion 1992; Walker, Ramsey, & Gresham, 2004).

One result of this increased attention to early identification and intervention has been the adoption of threetiered models of behavior prevention (Severson, Walker, Hope-Doolittle, Kratochwill, & Gresham, 2007). Experts have recommended three-tiered models of behavior prevention as a way to assist schools in enhancing the social development of all students by addressing the range of behavioral challenges that are present in schoolage populations. One major premise of three-tiered models of behavior prevention is that within any school setting, one can identify three types of students: typical students who are not at risk for developing maladaptive behavior patterns (80%-85%), students who may be at risk for developing persistent maladaptive behavior patterns (10%-15%), and students who already display persistent maladaptive behavior patterns (l%-5%; Walker et al., 1996).

A second major premise of this model is that student members of each group are candidates for differing levels or types of prevention that represent greater specificity, comprehensiveness, expense, and intensity. Within three-tiered models, interventions are divided into levels (primary, secondary, and tertiary) in an attempt to provide specific, appropriate services for the three types of students. At the primary level, universal interventions are implemented that are intended to prevent problem behaviors before they emerge. These interventions are implemented on a schoolwide level and are intended to enhance the protective factors that will reduce the likelihood that students will develop the effects of maladaptive behavior patterns. At the secondary level, the type and intensity of the intervention is elevated to meet the needs of students who do not respond positively to the primary intervention. Secondary interventions provide more focused behavioral or academic support to students who display behavior that places them at risk for developing persistent maladaptive behavior patterns. Finally, at the tertiary level, particular interventions are chosen and implemented to meet the needs of individual students who already display persistent maladaptive behavior patterns. Interventions at this level are comprehensive, intensive, and long term.

Valid and reliable universal screening measures play a key role in the success of three-tiered behavior prevention models. Universal screening procedures are necessary to screen all students in a school and identify those students who are at risk for developing and those with persistent maladaptive behavior patterns. Students with persistent maladaptive behavior patterns are then administered more comprehensive measures. Fortunately, a number of psychometrically sound comprehensive measures are available to identify students who display maladaptive behavior patterns. The Scale for Assessing Emotional Disturbance (Epstein &c Cullinan, 1998), the Child Behavior Checklist (CBCL; Achenbach, 1991), and the Behavior Assessment System for Children (Reynolds & Kamphaus, 1992) are three measures that are frequently used to identify children appropriate for special education or tertiary services. These three measures have well-established research bases that document acceptable psychometric properties (i.e., representative norms, reliability, validity) to the point that the measures can be used to make important educational and placement decisions about students who display maladaptive behavior patterns.

Unfortunately, few psychometrically sound universal screening measures are available. This is particularly the case for the preschool and kindergarten grades when children are first entering the educational system. Universal screening measures should meet certain criteria. According to Walker et al. (2004), screening measures should be brief and easy to implement. Furthermore, screening measures should be appropriate for school entry at the preschool and kindergarten levels and should demonstrate valid and reliable psychometrics. One widely used universal screening measure is the Ages & Stages Questionnaires: Social Emotional (Squires, Bricker, & Twombly, 2003). Although this measure has demonstrated adequate to good psychometrics and is relatively brief to complete, it is intended for use by parents only. This is problematic, because parent and teacher ratings of problem behavior are often not highly correlated with each other. The Ages & Stages also does not provide information on internalizing versus externalizing types of behavior patterns that is of interest to teachers and other educators. Another commonly used measure is the Early Screening Project (Walker, Severson, & Feil, 1995), which involves three stages or decision points: teacher ranking, teacher rating of behavior on three measures, and direct observations. Although the Early Screening Project is psychometrically sound, completing the three stages is lengthy and time consuming for teachers. This potentially limits the acceptability of the Early Screening Project as a universal screening measure. Experts developed the Early Childhood Behavior Problem Screening Scale (ECBPSS; Epstein & Nelson, 2006) to address weaknesses in currently available universal screening measures (i.e., it has Parent and Teacher forms, it is user friendly, it targets internalizing and externalizing behavior patterns).

The ECBPSS is a universal screening measure designed to identify preschool and kindergarten children who are at risk for developing maladaptive behavior patterns at school or home. The ECBPSS is brief; it comprises 12 items that are rated on a scale 6f 0 to 3 (0 = not at all like the child, 1 = not much like the child, 2 = like the child, 3 = very much like the child). It also contains Parent and Teacher forms. Both the Parent and Teacher forms consist of the same 12 items and similar instructions. The purpose of the present study was to assess the initial psychometrics of the ECBPSS based on parent and teacher ratings of a sample of preschool- and kindergarten-age children. Specifically, we addressed three research questions:

1. What is the factor structure and associated internal consistency of the factor items for the ECBPSS Parent form?

2. What is the factor structure and associated internal consistency of the factor items for the ECBPSS Teacher form?

3. What is the interrater reliability of the ECBPSS Parent and Teacher forms?

METHOD

Item Selection Process

The content of the ECBPSS was developed in a multistep process. First, the authors examined the scholarly and professional literature on factors that place children at risk for maladaptive behavior patterns at school and home. This review identified 40 child developmental risks that could be grouped into 11 domains:

1. prenatal (e.g., emotional distress or medical problems)

2. natal (e.g., premature or unusual delivery)

3. postnatal (e.g., medical problems or a prolonged hospital stay)

4. externalizing behaviors (e.g., overactivity, impulsivity, or temper outbursts)

5. internalizing behaviors (e.g., fearful, socially withdrawn, or cautious)

6. childhood maladjustment (e.g., running away or being abusive to animals) 7. childhood maltreatment (e.g., physically or sexually abused)

8. antisocial and psychiatric family history (e.g., domestic violence or mental illness)

9. family structure and socioeconomic status (e.g., one-parent household or eligibility for free or reduced lunch)

10. family functioning and parent management (e.g., parental distress or a difficult child)

11. maternal depression

Second, the authors used these 40 child developmental risk factors to create a 123-item risk factor interview for use with the child’s primary caregiver. The interview used a dichotomous response format to indicate the presence (i.e., yes, no) of each risk factor. Special education and mental health practitioners and researchers evaluated the scale for redundancy, readability, and usability prior to its use. Final modifications were made to wording of the items. Third, the 123-item interview became part of the intake process in a secondary prevention program for children already identified as at risk for behavior problems. Data were collected on 156 kindergarten and first-grade children who were screened into the secondary prevention program. Finally, two logistic regression analysis procedures determined the most reliable and robust prediction of total behavior problems using the 123-item risk factor interview.

The target variables for each of the logistic regression analysis procedures were the dichotomized CBCL Total Problems broadband scores (i.e., behavior problems absent or behavior problems present) that had been completed on the 156 children. The purpose of the initial logistic regression analysis procedure was to identify the domains that were most predictive of problem behavior so as to include them in the final regression model. Each of the risk factor domains was included in a logistic regression analysis to predict total problem behavior. Because no outliers were identified in initial analyses, no corrections were made for outliers in additional analyses. For each at-risk domain entered as a block (e.g., in the postnatal domain, both medical problems and prolonged stay were entered together) into the analysis, an omnibus statistic needed to be statistically significant (p < .05) for the domain to be considered for the second stepwise logistic regression analysis. Only the individual risk factors within domains that were statistically significant were considered for the second stepwise logistic regression analysis procedure.

The purpose of the second stepwise conditional logistic regression procedure was to identify the individual risk factor variables that provided the most reliable and robust prediction of total behavior problems. The logistic regression analyses identified 12 risk factors that were highly predictive of total problem scores on the CBCL:

1. has difficulty adjusting to changes or new things

2. upsets me just to be mean

3. often cries or fusses over little things

4. does things that irritate or frustrate me

5. destroys own toys and things

6. often moody or irritable

7. has a bad temper

8. often does not do what is asked

9. easily upset and frustrated

10. physically abuses others

11. is easily upset

12. demands a lot of attention

The overall correct classification of true negatives and true positive cases based on CBCL results was 78% and 75%, respectively. Thus, our preliminary research resulted in 12 items that were highly correlated and predictive of behavior problems. The ECBPSS comprises these 12 items. Table 1 presents the item intercorrelations for the ECBPSS Parent and Teacher forms. Teacher item intercorrelations were larger than those for parents, indicating that the ECBPSS Teacher form may have more discriminating power than the Parent form. In general, the item intercorrelations for parents were small to moderate, whereas those for teachers were moderate to large.

Participants

Children were recruited from preschool and kindergarten classrooms located in two medium-size cities in the midwest. All of the kindergarten classrooms were located in a public school system. All of the preschool classrooms were part of a large nonprofit organization devoted to serving low-income children. Parents and teachers were asked to complete the ECBPSS Parent and Teacher forms, respectively.

Parents. Parents of 132 children completed the ECBPSS Parent form. Of the 132 parent participants, 47% were men and 53% were women; 40% of the children participants were at the preschool level, and 60% were at the kindergarten level; 6% had been previously identified as meeting requirements for special education services. The ethnicity of the children was as follows: Caucasian (74%), Hispanic/Latin American (8%), African American (3%), Asian (1%), Native American (1%), multiracial (12%), and unknown (l%).The home language of all child participants was English.

Teachers. Teachers (n = 31) of 149 children completed the ECBPSS form. Parents of 132 of the children completed the form. Of the 149 participants, 50% were boys and 50% were girls; 37% of the children were at the preschool level, and 63% were at the kindergarten level; 7% of the children had been previously identified as meeting requirements for special education services. The ethnicity of the children was as follows: 65% Caucasian, 7% Hispanic/Latin American, 3% African American, 1 % Asian, 1 % Native American, 11 % multiracial, and 12% unknown. The relatively large percentage of unknown responses was a function of teachers’ uncertainty regarding the specific ethnicity of students. The primary language of all children was English.

The gender of children in the study sample was representative of children ages 5 years or younger in the United States (U.S. Department of Education, 2006). Nationwide, 49% of children ages 5 or younger are boys, and 51% are girls. The ethnic breakdown of children in the study sample varied somewhat from the national population. Caucasian children in the sample were overrepresented, whereas African American and Hispanic children were underrepresented. The ethnicity of children nationwide aged 5 or younger is 56% Caucasian, 21% Hispanic/Latin American, 14% African American, 4% Asian, and 1% Native American.

Procedures

A total of 33 preschool and kindergarten teachers were approached and asked to participate in the study by sending home information packets to the parents of the children in their classes. In all, 31 teachers in 14 preschool and 17 kindergarten classes agreed to participate. Information packets, which contained an information letter and a consent form as per institutional review board procedures, were sent home to 321 parents. In all, 152 parents, a 47.3% response rate, provided signed consent for their child to participate.

Parents. Parents who provided consent received in the mail a package that contained a copy of the parent version of the ECBPSS and a letter with instructions on how to rate their child on the scale. Parents who had not returned the package within 2 weeks received a reminder phone call, and parents who had not responded within 1 month were mailed the ECBPSS in a second mailing. A response rate of 87% was obtained, with 132 completed packages returned.

Teachers. The teachers of each of the children who received signed parental consent were sent a package that contained a copy of the teacher version of the ECBPSS and a letter with instructions on how to rate the child using the scale. Teachers who had not returned the package within 2 weeks received a reminder phone call, and the ECBPSS was sent to them again. A response rate of 98% was obtained, with 149 completed packages returned.

RESULTS

What Is the Factor Structure and Associated Internal Consistency of Factor Items for the ECBPSS Parent Form?

Using a principal component factor analysis, we identified two factors with eigenvalues greater than 1.0. We rotated these two factors to a Promax solution. Promax, an oblique rotation procedure, is recommended when the interfactor correlations are more than .15 (see Table 1; DeVellis, 2003). We eliminated items if they failed to load above .40 on either factor or were redundant to an item with a higher loading. If any item loaded .40 on more than one factor, we assigned that item to the factor on which it loaded higher. This occurred for only one item (i.e., Has difficulty adjusting to changes or new things). Table 2 presents the factor loadings and eigenvalues for both factors. The two factors that emerged represented internalizing and externalizing constructs. Each factor contained six items.

To determine the internal consistency of the ECBPSS Parent form, we calculated Cronbach’s coefficient alphas for the overall scale and for the internalizing and externalizing subscales. We did calculations separately for the preschool and kindergarten samples. For the preschool sample, the Cronbach’s coefficient alpha for the overall scale was .87, for the internalizing subscale was .84, and for the externalizing subscale was .83. The Cronbach’s coefficient alphas for the kindergarten sample were as follows: overall = .90, internalizing = .87, and externalizing = .83. The internal consistency of the parent version of the ECBPSS overall scale and two subscales demonstrated acceptable reliability.

What Is the Factor Structure and Associated Internal Consistency of Factor Items for the ECBPSS Teacher Form?

Using a principal component factor analysis, we identified two factors. We also rotated these two factors to a Promax solution. We eliminated items if they failed to load above .40 on either factor or were redundant to an item with a higher loading. Table 2 presents the factor loadings and eigenvalues for both factors. As with the ECBPSS Parent form, two six-item factors emerged representing internalizing and externalizing constructs.

We calculated Cronbach’s coefficient alphas for the overall scale and for the internalizing and externalizing subscales so as to determine the internal consistency of the ECBPSS Teacher form. We determined coefficient alphas for the preschool and kindergarten samples separately. For the preschool sample, the Cronbach’s coefficient alpha for the overall scale was .95, for the internalizing subscale was .95, and for the externalizing subscale was .93. The Cronbach’s coefficient alphas for the kindergarten sample were as follows: overall = .89, internalizing = .83, and externalizing = .90. The teacher version of the ECBPSS demonstrated acceptable stability and reliability on the overall scale and two subscales. What Is the Interrater Reliability of ECBPSS Parent and Teacher Forms?

Table 3 presents the means and standard deviations of the internalizing, externalizing, and total problem scales for parents and teachers. Pearson product-moment correlation coefficients between the two raters ranged from .32 to .37.

DISCUSSION

The results from the factor analyses and the internal consistency analyses suggest that both the Parent and Teacher forms of the ECBPSS have technically adequate psychometrics. For each form, we identified two factors (internalizing behavior and externalizing behavior). These two factor structures are consistent with major theories and research of childhood psychopathology (Achenbach & Edelbrock, 1984). For each of these factors, and the scale overall, the internal consistencies were acceptable (greater than .83), providing confidence in the homogeneity of the questions within each factor.

The results from the interrater analyses indicate that the reliability between parents and teachers ranged from .32 to .37. Although the interrater reliability coefficients were less than would be preferred, the finding is consistent with those of previous studies that have reported only modest levels of agreement between raters of child problem behavior (McConaughy, Stanger, & Achenbach, 1992). Specifically, Achenbach, McConaughy, and Howell (1987) found that levels of agreement among informants varied depending on the roles of the raters. Different raters interact differently with children and observe children in different environments, and they therefore have differing perspectives on child behavior. Raters with similar roles (e.g., two parents) have higher levels of agreement than those with dissimilar roles (e.g., a parent and a teacher).

Furthermore, there were relatively large parentteacher discrepancies in the mean ratings of child behavior on the ECBPSS overall, externalizing, and internalizing subscales. Parents were more likely to indicate that their child exhibited maladaptive behavior patterns than were teachers. This finding is consistent with previous research indicating that parents and teachers often disagree about the presence or absence of psychiatric symptoms in children (Achenbach et al., 1987; Kolko & Kazdin, 1993; Wolraich et al., 2004). Previous studies have found that parent and teacher disagreements are associated with the lack of acceptance of the child by the parents (Kolko & Kazdin, 1993) and parental stress (Youngstrom, Loeber, & Stouthamer Loeber, 2000). Parent and teacher disagreements about the presence or absence of emotional and behavioral disorders on the ECBPSS suggest that school and mental health professionals should gather information from parents and teachers and attempt to obtain a problem definition that is shared by parents and teachers. A shared problem definition may enhance collaborative efforts between parents and teachers to further diagnose whether there is a problem and use parent-teacher discrepancies clinically. Obtaining a shared problem definition is critical because there is evidence that parent-teacher discrepancies predict poorer treatment prognosis (Ferdinand, van der Ende, &c Verhulst, 2004). Additionally, the lower item intercorrelations for parents relative to teachers indicate that parent ratings of their child behavior on the ECBPSS tend to have lower discriminating power. This finding is consistent with previous research (Achenbach et al., 1987).

We have discussed appropriate screening and identification of children who may be at risk for developing behavior problems as an essential component of the three-tiered model of behavior prevention. Although experts have developed several different screening methods to identify children for tertiary services, early universal screening to identify children for secondary services is still not being widely practiced in preschool and kindergarten settings. The reasons assessment strategies are not more widely practiced may be due to length, time requirements, and psychometric limitations on use with younger populations. To overcome these concerns, the authors designed the ECBPSS to be brief, requiring only a few minutes per child to implement, and to be used specifically with preschool- and kindergarten-age children. Although additional research will be needed to further assess the psychometrics, the ECBPSS Parent and Teacher forms appear promising for use as an early, universal screening measure to identify young children at risk for developing maladaptive behavior patterns.

We should note several limitations of the present research. First, the data were collected on a nonrandom, convenience sample of participants. Second, the samples in the studies were not very large and consisted of children and adults in only one state (i.e., Nebraska). Future researchers would be judicious in selecting larger, random samples of participants from various settings. Finally, the background characteristics of the preschool and kindergarten children (and their parents and teachers) were not representative of the larger national population. Specifically, the proportion of children and parents with cultural differences from the study sample varied somewhat from the proportion nationwide.

The current study has provided preliminary information regarding the factor structure, internal consistency, and interrater reliability of the ECBPSS. However, future research is needed to address the aforementioned limitations as well as other important psychometric issues. First, future studies should use larger and more culturally diverse samples of children with and without disabilities and their parents and teachers who are selected from all regions of the country. Future researchers will do well to conduct other types of reliability and validity studies, such as further studies of interrater reliability (e.g., teacher to teacher, parent to parent), short(2-week) and long-term (i.e., 6-month) test- retest reliability, construct validity, predictive validity, and discriminant validity (e.g., children with disabilities vs. children with behavior problems). Future research is also needed to clarify the implications of differences in parents’ and teachers’ ratings of child behavior. It is evident that understanding the implications of these differences is important not only to the further development of the ECBPSS, but also to clinical practice. Even though more research is clearly needed to further establish the psychometric properties of the ECBPSS, the scale appears to be a promising instrument that may be appropriate for use by school personnel as a universal screener in preschool and kindergarten programs. At this point in time, however, professionals should use the ECBPSS cautiously until the identified future research is conducted. We recommend that if professionals want to use the ECBPSS in its current form, they should administer it as a screener, assessing children who score in the highest quartile with an established behavior rating scale.

REFERENCES

Achenbach, T. M. (1991). Manual for the child behavior checklist/ 4-18 and 1991 profile. Burlington: University of Vermont Department of Psychiatry.

Achenbach, T. M., & Edelbrock, C. S. (1984). Psychopathology of childhood. Annual Review of Psychology, 35, 227-256.

Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232.

Albers, C. A., Glover, T. A., & Kratochwill, T. R. (2007). Introduction to the special issue: How can universal screening enhance educational and mental health outcomes? Journal of School Psychology, 45,113-116.

DeVellis, R. F. (2003). Scale development: Theory and applications. Thousand Oaks, CA: Sage.

Epstein, M. H., & Cullinan, D. (1998). Scale for assessing emotional disturbance. Austin, TX: PRO-ED.

Epstein, M. H., & Nelson, J. R. (2006). Early childhood behavior problem screening scale. Lincoln: Center for At-Risk Children’s Services, University of Nebraska.

Fairbanks, S., Sugai, G., Guardino, D., & Lathrop, M. (2007). Response to intervention: Examining classroom behavior support in second grade. Exceptional Children, 73,288-310.

Ferdinand, R. F., van der Ende, J., & Verhulst, F. C. (2004). Parent-adolescent disagreement regarding psychopathology in adolescents from the general population as a risk factor for adverse outcome. Journal of Abnormal Psychology, 113, 198-206.

Gresham, F. M. (1991). Conceptualizing behavior disorders in terms of resistance to intervention. School Psychology Review, 20,23- 36.

Gresham, F. M., Lane, K. L., & Lambros, K. M. (2000). Comorbidity of conduct problems and ADHD: Identification of ‘fledgling psychopaths.’ Journal of Emotional and Behavioral Disorders, 8, 83- 93.

Kolko, D. J., &c Kazdin, A. E. (1993). Emotional/behavioral problems in clinic and nonclinic children: Correspondence among child, parent and teacher reports. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 34, 991-1006.

Kraemer, H. C, Kazdin, A. E., Offord, D. R., Kessler, R. C, Jensen, P. S., & Kupfer, D. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54, 337-343.

McConaughy, S. H., Stanger, C, & Achenbach, T. M. (1992). Three- year course of behavioral/emotional problems in a national sample of 4- to 16-year-olds: I. Agreement among informants. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 932-940. Patterson, G. R., Reid, J., & Dishion, J. (1992). Antisocial boys. Eugene, OR: Castalia Press.

Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior assessment system for children. Circle Pines, MN: American Guidance Service.

Severson, H. H., Walker, H. M., Hope-Doolittle, J., Kratochwill, T. R., & Gresham, F. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practice. Journal of School Psychology, 45, 193-223.

Squires, J., Bricker, D., & Twombly, E. (2003). Ages & stages questionnaires: Social emotional. Baltimore: Brookes.

U.S. Department of Education, National Center for Education Statistics. (2006). The condition of education 2006 (NCES Report No. 2006-071). Washington, DC: Author.

Walker, H. M., Horner, R. H., Sugai, G., Bullis, M., Sprague, J. R., Bricker, D., et al. (1996). Integrated approaches to preventing anti-social behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders, 4, 194-209.

Walker, H. M., Ramsey, E., & Gresham, F. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Belmont, CA: Wadsworth/Thomson Learning.

Walker, H. M., Severson, H. H., & Feil, E. G. (1995). The early screening project. Longmont, CO: Sopris West.

Wolraich, M. L., Lambert, E. W, Bickman, L., Simmons, T., Doffing, M. A., & Worley, K. A. (2004). Assessing the impact of parent and teacher agreement on diagnosing attention-deficit hyperactivity disorder. Journal of Developmental and Behavioral Pediatrics, 25, 41-47.

Youngstrom, E., Loeber, R., & Stouthamer Loeber, M. (2000). Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. Journal of Consulting and Clinical Psychology, 68, 1038-1050.

Ron Nelson

Michael Epstein

Annette Griffith

John Hopper

University of Nebraska-Lincoln

Address: Ron Nelson, Department of Special Education, University of Nebraska-Lincoln, 202 Barkley Center,

Lincoln, NE 68510: e-mail: rnelson8@unl.edu

Copyright PRO-ED Journals Fall 2007

(c) 2007 Topics in Early Childhood Special Education. Provided by ProQuest Information and Learning. All rights Reserved.