Outcomes From Wraparound and Multisystemic Therapy in a Center for Mental Health Services System-of-Care Demonstration Site

Written By: Sam Savage

Published Date: October 2, 2007 Last Edited: October 2, 2007

By Stambaugh, Leyla Faw Mustillo, Sarah A; Burns, Barbara J; Stephens, Robert L; Et al

This study examined outcomes for 320 youth in a Center for Mental Health Services system-of-care demonstration site. Youth received wraparound-only (n = 213), MST-only (n = 54), or wraparound + MST (n = 53). Participants were 12 years old on average and mostly White (90%), and 75% were Medicaid-eligible. Service use and functional and clinical outcomes were examined at 6-month intervals out to 18 months. All three groups improved over the study period. The MST- only group demonstrated more clinical improvement than the other groups. Functional outcomes did not differ significantly across groups. Youth in wrap + MST had higher baseline severity and experienced less clinical and functional change than the other two groups, despite more mental health service use. Targeted, evidence- based treatment may be more effective than system-level intervention alone for improving clinical symptoms among youth with serious emotional disorders served in community-based settings. New or amended approaches may be needed for youth with the most severe disorders. Wraparound and multisystemic therapy (MST) are two contemporary, community-based interventions for youth with serious emotional disorders (SED). Although MST intervenes at the individual level, wraparound is a process for developing individualized service plans at the system level. Both interventions have seen widespread dissemination into community mental health settings around the United States over the last decade. MST is heavily based in theory and research, and favorable outcomes have been reported from controlled evaluations. Wrap-around has spread quickly as a promising intervention, but standards have been slow to develop, and rigorous research has lagged as a result.

Following brief descriptions of the two approaches and an update on the research evidence, this article will present results of an examination of the effectiveness of wraparound and MST in one system of care for children and adolescents with SED. Although not a controlled trial, the study is an attempt to measure outcomes from the two interventions as delivered in a naturalistic setting. The study is, thus, an answer to the call of the President’s New Freedom Commission on Mental Health (2003) and the National Advisory Mental Health Council’s Workgroup on Child and Adolescent Mental Health Intervention Development and Deployment’s “Blueprints for Change” report (2001) to collect evidence from real-world practice to guide research on interventions that are both effective for youth and acceptable to practitioners in community- and clinic-based settings.

WRAPAROUND PROCESS

Principles

Wraparound is “an approach to implementing individualized, comprehensive services within a system of care for youth with complicated multidimensional problems” (Burns & Goldman, 1999). As such, it is viewed as a system-level intervention that quite literally aims to “wrap” existing services around youth and their families to address their problems in an ecologically comprehensive way. Wraparound is intended for youth involved in mental health, education, juvenile justice, and child welfare. The wraparound philosophy posits that direct intervention in the service system to provide individualized service planning will lead indirectly (via specific services) to positive change within the child and family. Targeted outcomes include increasing behaviors that facilitate functioning in the community (e.g., positive family and peer relationships, school achievement) and eliminating behaviors that place the child at risk for removal from his or her family or community (e.g., aggression, theft, vandalism, self-injury). A 1998 survey of U.S. states and territories estimated that approximately 1 80,000 youth were receiving wraparound at that time (Faw, 1999).

Research

Reviews of the wraparound evidence base describe wraparound as promising, having shown positive results from three randomized trials and multiple quasi-experimental studies (Burns, Schoenwald, Burchard, Faw, & Santos, 2000; Farmer, Dorsey, & Mustillo, 2004). Study populations have included youth in child welfare, juvenile justice, and system of care. Outcomes have been studied in various domains, producing mixed funding across domains. For youth in child welfare, increases in both clinical symptoms and functioning have been observed compared to treatment as usual (Clark et al., 1998; Evans, Armstrong, Kuppinger, Huz, & Johnson, 1998). Studies with youth in juvenile justice have found improvements in school performance and decreased instances of running away from home (Carney & Buttell, 2003; Pullman et al., 2006). Findings on recidivism (re-arrest) were mixed. Finally, one randomized trial including youth in a system of care found no difference in clinical outcomes for wraparound versus usual treatment, despite higher service use for youth in wraparound (Bickman, Smith, Lambert, & Andrade, 2003). In addition to these experimental and quasi- experimental findings, a recent study reported a positive association between wraparound fidelity and child clinical outcomes (Bruns, Suter, Force, & Burchard, 2005).

MULTISYSTEMIC THERAPY

Principles

Multisystemic therapy is an intensive home- and communitybased family treatment model for children and adolescents at imminent risk for out-of-home placement because of serious emotional and behavioral problems (Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 1998). MST is a time-limited intervention that lasts 3 to 5 months. MST therapists use principles of family therapy to intervene directly with children and families. Although MST began as a treatment model for juvenile offenders, it has been adapted for youth in child welfare, psychiatric inpatients, and youthful violent sex offenders. Targeted outcomes include decreased behavior problems and improved family, peer, and school functioning.

Research

MST has been the subject of nine randomized trials and at least one quasi-experimental study (Aos, Phipps, Bamoski, & Leib, 2001; Curtis, Ronan, & Borduin, 2004; Hoagwood, Burns, Kiser, Ringeisen, & Schoenwald, 2001). Comparison conditions have included individual counseling (Borduin, Henggeler, Blaske, & Stein, 1990), typical juvenile justice services (Leshied & Cunningham, 2002), typical mental health services (Rowland et al., 2005), psychiatric hospitalization (Henggeler et al., 2003), and typical child welfare services (Ogden & Halliday-Boykins, 2004). Many positive outcomes have been reported, including decreased aggressive behavior, fewer arrests, fewer placements, and improved family functioning. Long- term follow-up findings have also been positive (Schaeffer & Borduin, 2005). A recent meta-analysis criticized findings on MST’s effectiveness in child welfare as well as the fact that research on MST has been conducted almost exclusively by MST developers (Littell, 2005). Overall, however, MST is a well-researched treatment model for youth with behavior problems and has begun to emerge as an effective model for other child mental health disorders.

STUDY RATIONALE

Both wraparound and MST target the child’s ecology and aim to keep the child in his or her home community. Although MST is a short- term clinical intervention, wraparound is an unconditional, often long-term process for planning and coordinating services. The history of wraparound and MST is a somewhat adversarial one. Proponents of MST argue that treatment should be heavily grounded in theory and research and that brief, behaviorally targeted intervention with the child and family is most likely to produce improvement. Proponents of wraparound argue that a rigorous process for tailoring individualized service plans for children and families based on their unique strengths and needs and an unconditional commitment are most likely to produce lasting improvements. These two fundamentally different approaches have never been studied in parallel. From a policy perspective, it is important to compare the utility of brief, treatment-level intervention versus longer-term, system-level intervention.

The provision of both wraparound and MST at a site where data on children and families were routinely collected as part of a large- scale national evaluation offered an ideal opportunity to conduct an observational study comparing outcomes from the two interventions. Moreover, because both wraparound and MST creatively draw upon resources in the community, the availability of community resources was held constant across intervention groups. Absent such parity, any study comparing the two interventions would be fatally confounded. The study’s aim was to analyze clinical and functional outcomes from wrap- around versus MST for youth with SED. Both interventions were expected to predict improvement in clinical symptoms and functioning over an 18-month follow-up period.

METHOD

Study Site

The study included data from families who enrolled in a Center for Mental Health Services (CMHS) system-of-care site that was funded between 1999 and 2003. The site covers a rural and frontier area of Nebraska and targets youth with SED who are at risk for out- of-home placement. Referrals are taken from the schools, child welfare agencies, the state department of health, juvenile justice agencies, the departments of parole and probation, and other child- serving agencies. Treatment Enrollment

To qualify for wraparound, a youth had to be under 21 years old, demonstrate a diagnosable and persistent mental health disorder as defined by the Diagnostic and Statistical Manual of Mental Disorders- Fourth Edition (DSM-IV; American Psychiatric Association, 1994), demonstrate functional impairment in two or more areas, be at risk for restrictive placement, and be at risk for dropping out of school or involvement in the juvenile justice system. In addition, families or the state (in the case of a ward of the court) had to agree to participate in treatment. Additional criteria were required for enrollment in MST. Specifically, in addition to meeting the enrollment criteria for wraparound, a child had to demonstrate physical aggression in the home, at school, or in the community; school truancy or failure; verbal aggression; criminal or delinquent behavior; association with delinquent peers; or substance abuse. Participants were excluded from MST if they had committed a sex offense in the absence of other delinquent or antisocial behavior or if they were actively suicidal, homicidal, or psychotic.

These procedures for group enrollment would suggest that youth assigned to MST had more severe behavioral problems than youth assigned to wraparound. However, the baseline sample descriptives (described later) suggest this was not the case, implying that enrollment protocols were set as an ideal but were not always met. When the protocol was not followed, this was most likely due to nonsystematic fluctuations in the availability of wraparound facilitators and MST therapists to take on new cases. Both wraparound and MST were provided to youth who did not respond to the original intervention attempt. These youth were considered a separate intervention group for all study analyses. Most of the youth who received both interventions received wraparound first and then MST.

Treatment Discharge

To be discharged from wraparound, a child and family had to show that they had met the treatment goals identified at intake, that the child no longer had significant problems, and that the family had been functioning reasonably well for 3 months. For a youth to be discharged from MST, treatment gains had to be sustained for 3 weeks, or the treatment had to have reached a point of diminishing returns, evidenced by the youth having no significant clinical problems, making a reasonable educational or vocational effort, and getting involved with prosocial peers and minimally with deviant peers.

Treatment Implementation

Wraparound providers at the site were initially trained using an adapted version of the VanDenBerg and Grealish ( 1996) model followed by continuous trainings that included classroom sessions, shadowing, and monthly booster sessions. A train-thetrainer model was used whereby on-site staff members trained new staff members as they came on board and provided monthly booster sessions to all wraparound staff. Wraparound facilitators were required to have a bachelor’s degree and carry a maximum caseload of 10 families.

The Wraparound Fidelity Index (WFI; Bruns, Burchard, Suter, Leverentz-Brady, & Force, 2004) provided monthly feedback to staff on wraparound adherence. Version 2.1 was used initially, followed by Version 3 when it became available. Prior analyses from the site demonstrated that high fidelity was maintained across the 11 essential elements of wraparound measured by the WFI (DeKraai et al., 2004). Site-level scores were consolidated across respondents (child, parent, care coordinator, and team member) and across school- and clinic-based programs. The average WFI score across sites was 80.3% in Year 1, 82.7% in Year 2, and 83.2% in Year 3.

Although fidelity data were not available for the study, the standard protocol for MST fidelity was implemented at the site. The protocol as described in Henggeler, Schoenwald, Liao, Letoumeau, and Edwards (2002) involves an initial 40-hr training session to provide grounding in the theoretical and empirical bases of MST, weekly on- site clinical supervision by an MST clinical supervisor, weekly phone consultations with an MST expert, and quarterly booster trainings on special topics. These methods have been positively associated with therapist adherence and child and family outcomes (Schoenwald, Henggeler, Brondino, & Rowland, 2000). MST therapists had at least a master’s degree and carried a caseload of four to six families. In the current study, 57 MST participants had received more than 1 week of MST prior to baseline study enrollment. As such, the results provide a conservative estimate of MST effects.

Data Collection Procedures

The study site was a demonstration site in the National Evaluation of the Comprehensive Community Mental Health Services (CMHS) for Children and Their Families Program. Begun in 1993, this program disseminates evidence-based practice across the United States, via CMHS systems of care, to children with mental health problems and their families (see Holden, Friedman, & Santiago, 2001, for description). The National Evaluation (conducted by Macro International, Inc.) was established to examine these demonstration sites. Main goals of the evaluation include describing the children and families served by CMHS systems of care, examining whether the system-of-care model leads to positive outcomes for youth, and describing patterns of child and family service utilization. Each site is funded for 5 to 6 years.

The Nebraska site also participated in a longitudinal substudy in which additional data on service use and clinical outcomes were collected at 6-month intervals up to 36 months following system-of- care enrollment. The current study includes longitudinal data through 18-month follow-up. This follow-up point was chosen to maintain a high degree of data retention for our analyses and because we felt it was reasonable to expect clinical change within this time period. In addition, due to rolling enrollment procedures, this was the longest follow-up wave for which all cohorts had the opportunity to provide data.

Study Procedures

The observational study followed three treatment groups from enrollment through three consecutive 6-month follow-up assessments. Informed consent procedures were followed as specified for the National Evaluation. Consent was obtained by (a) completing intake forms and descriptive information when families presented to services, (b) determining eligibility for the longitudinal outcome study, (c) recruiting eligible families for the longitudinal outcome study, (d) obtaining informed consent (and youth assent where appropriate) to participate in the longitudinal outcome study, and (e) identifying appropriate respondents to participate in data collection interviews.

Instruments

Although the National Evaluation employs many measures covering a wide array of child and family variables, instruments focused on child service use, and clinical and functional outcomes were chosen for the current study. These measures provided a broad scale for detecting mental health change while also maintaining a parsimonious conceptual model.

Child Behavior Checklist. The CAiW Behavior Checklist (CBCL; Achenbach, 1991) is a parent-report instrument that assesses behavioral problems and social competencies of children ages 4 to 18. The CBCL contains scales of Externalizing (aggressive and delinquent) and Internalizing (withdrawn, anxious/depressed, somatic complaints) symptoms, which when combined yield a Total Problems score. The CBCL has demonstrated strong convergent validity with the Conners Parent Questionnaire (.56 to .86) and the Quay-Peterson Revised Behavior Problem Checklist (.52 to .88; Achenbach, 1991). One-week test-retest reliability of .93, as well as interparent reliability of .66 for Internalizing and .80 for Externalizing, has been established (Achenbach, 1991). Content and criterion validity are supported by the ability of CBCL items to discriminate between matched referred and nonreferred youth (Achenbach, 1991).

Child and Adolescent Functional Assessment Scale. The Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 1996), completed by the youth’s caregiver or clinician for children ages 7 to 18, measures the degree to which a mental health disorder affects a child’s everyday life across eight domains: school, home, community, behavior toward others, moods/emotions, self-harmful behavior, substance use, and thinking. The total subscale scores for each life domain are summed to provide a total scale score corresponding to the child’s global functioning. The CAFAS has demonstrated adequate internal consistency reliability (range = .63- .68), high interrater reliability (.92), and convergent validity with the CBCL (.42-.49) and the Child Assessment Schedule (.52-.56; Burns & Kutash, 2000).

Multisector Service Contact Questionnaire. The Multisector Service Contact Questionnaire (MSSC) was created by Macro International, Inc., to examine the types and frequencies of services received by children and families in the National Evaluation. Parents/caregivers were asked whether their child received 22 different service types, related to a mental health problem, in the prior 6 months. Some of the services include inpatient and outpatient therapy, case management, after-school services, transportation, and more. The MSSC was completed at 6, 12, and 18 months. Although psychometrics are not yet available for the MSSC, there is evidence that caregiver report of child service use is consistent with provider records (Ascher, Farmer, Burns, & Angold, 1996; Horowitz et al., 2001).

Sample Description

Sample descriptives are presented in Table 1. The sample consisted of 320 children and adolescents ranging in age from 4 to 17.5 years at study intake. Their average age was 12, and 73% were male. The racial distribution of the sample was 90% White, 4% American Indian, and 6% other. Thirty four (11%) participants were of Hispanic ethnicity. A majority of families (57%) reported a gross household income of $25,000 or less. Seventy-one percent were eligible for Medicaid. Comparing descriptives across the three groups, it is apparent that youth who received both interventions ( wrap + MST) had higher CBCL and CAFAS scores and experienced more placements than the other two groups during the 6 months prior to baseline. This suggests that youth who received both wrap-around and MST likely did so because their mental health problems were more severe than those of the other two groups. It is probable that these youth did not respond to the first intervention attempt.

Four differences emerged while comparing descriptives across youth in the wrap-only group versus youth in the MST-only group. Youth in the wrap-only group were younger, were more likely to be male, were more often referred from education

versus corrections/court, and had higher Internalizing t scores on the CBCL. The importance of these differences for outcome comparisons is less than would have been expected based on the written protocol for treatment assignment. That is, the wraponly and MST-only groups did not differ in terms of behavioral severity at baseline (CBCL Externalizing t scores), despite the fact that youth in MST-only were more often referred from court/corrections. Referral from the courts did not function as a proxy for behavioral severity. Moreover, as shown in Table 1 , the largest percentage of youth in all groups was referred from mental health or child welfare. Despite the lack of evidence for selection bias between these two groups, baseline severity was included as a covariate in the inferential models, and interaction terms were introduced to control for differences across all groups in Total Problem scores.

RESULTS

All analyses included three groups. The wrap-only group (n = 213) consisted of youth who received only wraparound during the study period. The MST-only group (n = 54) consisted of youth who received only MST during the study period. The wrap + MST group (n = 53) consisted of all children who received both wraparound and MST at any time during the 18month study period. In this group, receipt of the two interventions overlapped chronologically in some cases but not in others. These differences were controlled for in the mixed models by including a variable coded for simultaneous receipt of wraparound and MST at each study wave. The average length of treatment was 15 months for youth in wrap-only, 5.5 months for youth in MST-only, and 10.2 months for youth in wrap + MST.

Sample Attrition

Rates of study attrition were 1 1% at 6 months, 28% at 12months, and 37% at 18 months. Attrition status at 18 months did not differ by treatment group, gender, age, minority status, family income, or baseline CBCL or CAFAS scores. Multiple imputation procedures were performed using Stata, Version 9.2. All models were run twice – once using only available, nonimputed data and once using imputed data. Results did not differ; therefore, results from the nonimputed model are presented.

Analytic Approach

Following variable diagnostics, three types of analyses were performed: service use descriptives; clinical and functional change descriptives from pre- to posttreatment; and inferential statistics from linear mixed models, performed using Stata, Version 9.2.

Service Use Descriptives

Service use was examined across study groups and coded as yes/no based on any report of service use across the 18-month study period. Rates of missing data at each time point ranged from 14% to 61%. Nonetheless, findings are reported because (a) the way the variable was coded allowed multiple chances for a youth to provide data across time points, and (b) the available data provide a meaningful comparison of the proportion of actual respondents within each group reporting service use. Results are shown in Figure 1.

Because both wraparound and MST aim to prevent restrictive placements, use of primarily nonrestrictive services was expected for all groups. This expectation was confirmed. One notable finding was a higher rate of service use across nearly the entire service spectrum for the wrap + MST group. This service use pattern is not surprising and may be more related to a higher level of need in this group than to the intervention model received.

For the other two groups, patterns of service use were in line with expectations. Youth in the MST-only group were more likely to participate in family preservation and family therapy. Given that MST is a family-based therapy model, this is a confirmatory finding. Similarly, youth in the wrap-only group were more likely to use case management services, as expected. Wraparound is seen by some as a form of intensive case management, so this also is a confirmatory finding. Youth in the wrap-only group were more likely than youth in the MST-only group to use nearly all other types of services included on the MSSC. This is probably related to the difference in duration of intervention across the two groups (15 months vs. 5.5 months).

Clinical Findings

Descriptive Results. Average outcome scores across the four study time points are shown in Figure 2. As demonstrated, both CBCL and CAFAS scores gradually declined (improved) over the study period for all three groups. Paired t tests indicated the change from baseline to 1 8 months was significant at the ?

On the CBCL, the percentage of youth who started in the clinical or borderline range at baseline and moved below the borderline range by the end of the study was 32% for wrap-only, 62% for MST-only, and 20% for wrap + MST. Scores decreased on average by 8.3 points in wrap-only, 13.7 points in MST, and 10.5 points in wrap + MST. Although 10 points represents a change of one standard deviation (which can be viewed as a meaningful change), 80% of the total wrap + MST group remained at or above the borderline range at 18 months, suggesting that despite the change in total score, most youth in this group were still in need of intensive services at the end of the study. By comparison, 64% of the wrap-only group and 30% of the MST-only group were in the borderline or clinical range at the end of the study.

On the CAFAS, the percentage of those who started in the marked or severe impairment range at baseline and moved into the minimal- to-moderate impairment range by the end of the study was 36% for wrap-only, 66% for MST-only, and 26% for wrap + MST. The total scale score decreased on average by 34.2 points in wrap-only, 47.8 points in MST-only, and 49 points in wrap + MST. Again, although one standard deviation is around 40 points on the CAFAS, nearly three- quarters of the wrap + MST group remained in the marked-to-severe range of impairment at the end of the study, indicating need for intensive services. Comparatively, 61% of wrap-only youth and 34% of MST-only youth were in the marked-to-severe range at the end of the study.

Inferential Results. Group differences in outcomes were examined at each wave using linear mixed models. These models used maximum likelihood estimation with an unstructured covariance structure, including random coefficients and random slopes. The models controlled for baseline CBCL and CAFAS scores, gender, age, minority status, family income, and number of living placements reported at each 6-month follow-up. Propensity score matching was also considered to account for baseline differences; however, the sample size was insufficient to allow for reliable estimates.

Preliminary models in which controls were included for time since treatment completion and treatment dose (total amount of wraparound or MST received over the study period) indicated these variables were not predictive of outcome on either the CBCL or the CAFAS . As such, these two variables were not included in the models presented here. Results from this model did not differ from the results from models run without these controls. Tables 2 and 3 show results from the CBCL and CAFAS models, respectively. In both models, the wrap- only group served as the comparison group. An unconditional growth model was run first, followed by a series of nested models that included a group by time interaction term to model differences in the rate of change over time across groups. Goodness of fit and change in variance of the random parameters were examined for each successive model. Results from the final, bestfitting model are presented here.

The CBCL model (see Table 2) consisted of 688 observations from 298 youth over the four waves of data collection. The rate of decline across the 18-month study period was significantly greater for the MST-only group compared to the wraponly group, demonstrated by a significant Group ? Wave interaction for MST-only. There was no significant difference in change over time between the wrap + MST group and the wrap-only group. Family income negatively predicted CBCL scores. This effect was net of group membership, suggesting that family income impacted change in clinical symptoms over and above the treatment group. An interaction between group and baseline CBCL scores was included in the model to account for group differences in severity on outcome. As shown in Table 2, baseline severity in the MST-only group (MST-only ? Baseline CBCL) did not predict outcome, suggesting that the finding of greater improvement in the MST group was not explained by any group differences in baseline clinical severity. The CAFAS model (see Table 3) included 687 observations from 298 youth over the full follow-up period. There were no significant differences in CAFAS outcome scores between the wrap-only and the MST-only groups; however, the wrap + MST group score was significantly higher (worse) compared to the wrap-only group. The absence of a Group ? Time effect suggests there was no difference in the rate of change over time. The difference was rather in overall mean score. Controlling for other variables, there were no differences in group outcomes by CAFAS baseline scores. As such, interaction terms for group by baseline CAFAS score were omitted from the model presented.

Although age, sex, race, and income did not independently predict CAFAS score, number of placements reported during the study was positively associated with CAFAS score at follow-up. Each additional placement was associated with an 8.2-point increase in score. This effect was net of group membership, suggesting that placement history impacted change in functioning over and above membership in any of the three treatment groups.

DISCUSSION

Findings from the study suggest that youth in all groups improved over time on both clinical symptoms and more generalized functioning. Findings also suggest that participants were engaged in the service system during the study period and that the system of care successfully maintained youth in community-based settings, largely avoiding the use of restrictive placements. Wraparound participants appeared to receive more services over the 18-month period, but this may simply be because they were enrolled in the system of care for a longer period of time, on average, than those who received only MST.

Youth receiving only MST demonstrated more improvement in clinical symptoms than did those who received only wraparound over the 1 8-month follow-up assessment. In addition, youth who received only MST were more likely than youth who received only wraparound to move out of the clinical range of impairment by the end of the study. A majority of youth in both wraparound groups remained in the borderline to clinical range on the CBCL at the 18-month follow-up period, indicating these youth were still experiencing a high level of mental health need. Overall, CBCL results suggest that youth who received only MST improved at a faster rate and to a higher degree than those who received wraparound. Given that the study took place at one site where community resources were effectively held constant across groups, the immediate implication of the findings was that MST was more effective than wraparound.

An alternative explanation for the positive MST finding is that youth in the MST-only group were more likely to improve because they met baseline criteria that specifically fit with the intended target population for MST. The baseline clinical characteristics of youth in MST-only versus youth in wrap-only were significantly different in some areas, but there was no indication that youth assigned to MST had more severe behavior problems than youth assigned to wraparound. Moreover, the interaction of group and severity was only significant for the wrap + MST group in the mixed model, implying that severity at baseline did not contribute significantly to the difference in outcomes for youth who received only MST versus youth who received only wraparound.

Findings from the study were in line with past literature that suggests MST is effective for emotionally and behaviorally disturbed youth, while wraparound is promising but has not yet gained the same level of empirical support as MST There are several reasons why research on wraparound may have produced mixed findings. First, wraparound is difficult to study in a controlled way because treatment plans are individualized for each youth. It is possible that some youth in wraparound have access to evidence-based treatments targeted for their specific problems while others may not because of a lack of such treatment or other barriers. The average group effect in such a scenario may provide a limited estimate of wraparound utility because results from both effective and ineffective treatments may contribute to the group outcome score.

Second, the populations studied and the outcomes measured in past studies of wraparound have varied. Randomized trials comparing wraparound to foster care (Clark et al., 1998; Evans et al., 1998) have found positive effects in both clinical and functional domains. In contrast, both the Carney and Buttell (2003) and the Pullman et al. (2006) studies focused on juvenile offenders and reported mixed functional outcomes; clinical outcomes were not measured. The only study besides the current one to focus on children in systems of care (Bickman et al., 2003) measured both functional and clinical outcomes and reported no significant findings in either domain. The setting in which wraparound is provided, including both the target population and the outcome domains studied, may influence the results of any study on the impact of the intervention. Although wraparound is broadly targeted to youth with any serious emotional disorder, it may be differentially effective for different clinical subgroups and across different service systems where quality of services available varies.

One prior study (Bickman et al., 2003) found that wraparound leads to higher use of services compared to usual treatment. A similar finding emerged in the current study. Youth in wraparound appeared to receive more services than youth in MST over the 1 8- month study period. In this sense, wraparound appears to meet its goals. However, as in the Bickman et al. study, which found no difference in outcome, the current findings similarly give no indication that use of more services in the wraparound group had any clinical benefit above and beyond that experienced by youth enrolled in MST-only. This may have implications for the cost-effectiveness of system- versus treatment-level interventions for youthful populations.

One strength of the study was that data were collected onsite and then analyzed by an independent research team. Much of the past literature on wraparound and MST has emerged from developers or champions closely affiliated with these models. The primary strength of the study is that it is the first to examine an integrated model of wraparound and MST. This task was made possible by the quality of both service coordination and data collection at the study site.

The study was also limited in several ways. First, without a no- treatment or usual-treatment control group or random assignment it is ultimately impossible to know whether there was any causal relationship between treatment and outcomes. However, because both wraparound and MST intervene in the child’s ecology and the study occurred at a single site where the resources that are available in the community would presumably have been equally available to youth in all three treatment groups, it is less likely that nontreatment- related factors contributed to differences in treatment outcome between the wrap-only and the MST-only groups. Furthermore, the type of observational research presented here is considered valid for examining correlations between variables (e.g., treatment and outcome) that are unlikely to occur by chance and for studying treatments as they occur in the real world as opposed to the research laboratory (American Psychological Association Presidential Task Force on Evidence-Based Practice, 2006; Institute of Medicine, 2006). The current study is a product of the current momentum around a practice-based evidence ideology.

Additional limitations include limited representation of minorities in the sample and lack of fidelity data for inclusion in the study. Regarding the lack of minority youth, results should not be generalized to minority populations. On a positive note, however, many past studies of family-based interventions (especially in the case of MST) have focused on minority samples, so this study is unique in that sense and offers new information on the relevance of wraparound and MST for rural, White youth and their families. Regarding the lack of fidelity data, inclusion of WFI results from the time of the study surpasses the standard of reporting of fidelity in past wraparound outcome studies. Confirmation from MST Services, Inc., that the rigorous MST adherence protocol was followed at the site fits with the findings of the study in terms of service use, length of treatment, and outcomes for the MST-only group.

The final limitation was the significant difference in baseline severity between youth who received both interventions and youth who received one or the other. This may relate to the naturalistic setting of the study – youth with the most severe problems received both services, and, as such, treatment effects may have been more difficult to achieve in this group. Because of the high level of severity, it is difficult to draw conclusions about the effectiveness of either model for these youth. In a site with both wraparound and MST, it appears that only youth who did not respond to the first intervention approach, perhaps because of severity, received both interventions.

Taken at face value, the results from the combined intervention group suggest that youth with the most severe problems may need more than what is currently available even in stateof-the-art service settings such as the one studied here. By 18 months after enrollment, a large majority of the youth in this group were still experiencing significant clinical and functional impairment despite high rates of service use. The implications for cost-effectiveness may be pessimistic for this group. Two studies have reported on MST’s effectiveness with very severe populations (Ogden & Halliday- Boykins, 2004; Schaeffer & Borduin, 2005). In the Ogden and Halliday- Boykins study, MST effects were modest compared to other MST trials. The authors argued this was likely due to the high perceived quality of services in their comparison group. In light of findings from the current study, another possibility is that MST needs enhancement to meet the challenges presented by youth with the most severe problems. Additional findings suggest that family income and placement history may predict youth outcomes regardless of the type of treatment received. Further, these demographic variables may have different influences on clinical versus functional change. Lower family income predicted worse clinical outcomes regardless of treatment group membership. Family income is an aspect of socioeconomic status that has been shown to place youth at risk for disruptive behavior disorders during adolescence (Herrenkohl, Hawkins, Chung, Hill, & Battin-Pearson, 2001). Prior studies including this variable have focused on poverty at the community level. The extent to which family income is indicative of neighborhood resources was not known in the current study. More research is needed to tease out the elements of socioeconomic status that may independently contribute to risk.

Number of out-of-home placements was highly predictive of functional change, with more placements predicting less positive change. Moving a child in and out of placements may be severely damaging to his or her functioning, perhaps due to a repeated need to readjust and a lowered sense of personal se- curity resulting from instability. Out-of-home placements are also more likely to expose youth to contagion (Dishion & Dodge, 2005), a process in which placing youth with behavior problems together is thought to exacerbate their problems. One study with foster children showed an increase in problem behaviors over time for children who experienced multiple placements (Newton, Litrownik, & Landsverk, 1999). Conversely, youth with more severe behavior problems may be more likely to experience unsuccessful placements and thus expose themselves to multiple placements over time. Some longitudinal research has shown that children with externalizing problems are at greater risk for multiple placements than children without such problems (Nugent & Glisson, 1999).

One of the criteria for a treatment to be considered evidence- based is achievement of outcomes that are equivalent to those reported from an already established evidence-based treatment in a direct group-comparison study (Lonigan, Elbert, & Johnson, 1998). In the current study, the wrap-only group did not improve clinically at a rate equivalent to that of the MST-only group. Accepting that there were no selection biases that impacted outcomes, this finding suggests that targeted, evidence-based treatment models may offer significant benefits for youth with SED beyond what can be expected from intervention at the service level alone. This is in line with much controlled research suggesting that evidence-based child mental health treatments are often brief (3-5 months) and narrowly targeted clinically, e.g., cognitive behavior therapy (CBT) for depression (Brent et al., 1997), parent-child interaction therapy for disruptive behavior disorders (Schuhman, Foote, Eyberg, Boggs, & Algina, 1998), and trauma-focused CBT for child trauma (Cohen, Mannarino, & Knudsen, 2005). Further controlled research is needed to clarify the role of system-level interventions in settings where evidence-based practices are either available or practicable.

References

Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/ 4-18 and 1991 profile. Burlington: University of Vermont, Department of Psychiatry.

American Psychiatric Association Committee on Nomenclature and Statistics. (1994). Diagnostic and statistical manual of mental disorders (2nd ed.). Washington, DC: Author.

American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. The American Psychologist, 67(4), 271-285.

Aos, S., Phipps, P., Barnoski, E., & Leib, R. (2001). The comparative costs and benefits of programs to reduce crime. Olympia: Washington State Institute for Public Policy.

Ascher, B. H., Farmer, E. M. Z,, Bums, B. J., & Angold, A. (1996). The Child and Adolescent Services Assessment (CASA): Description and psychometrics. Journal of Emotional and Behavioral Disorders^, 12-20.

Bickman, L., Smith, C. M., Lambert, E., & Andrade, A. R. (2003). Evaluation of a congressionally mandated wraparound demonstration. Journal of Child and Family Studies, 12, 135-156.

Borduin, C. M., Henggeler, S. W., Blaske, D. M., & Stein, R. J. (1990). Multisystemic treatment of adolescent sexual offenders. International Journal of Offender Therapy and Comparative Criminology, 34(2), 105-113.

Brent, D. A., Holder, D., Kolko, D., Birmaher, B., Baugher, M., Roth, C, et al. (1997). A clinical psychotherapy trial for adolescent depression comparing cognitive, family, and supportive therapy. Archives of General Psychiatry, 54(9), 877-885.

Bruns, E. J., Burchard, J. D., Suter, J. C, Leverentz-Brady, K., & Force, M. M. (2004). Assessing fidelity to a community-based treatment for youth: The Wraparound Fidelity Index. Journal of Emotional and Behavioral Disorders, 12(2), 79-89.

Bruns, E. J., Suter, J. C, Force, M. M., & Burchard, J. D. (2005). Adherence to wraparound principles and association with outcomes. Journal of Child and Family Studies, 14, 52 1-534.

Bums, B. J., & Goldman, S. K. (Eds.). (1999). Systems of care: Promis- ing practices in children’s mental health, 1998 series: Volume IV. Promising practices in wraparound for children with serious emotional disturbance and their families. Washington, DC: Center for Effective Collaboration and Practice, American Institutes for Research.

Burns, B. J., & Kutash, K. (2000). Child and adolescent measures of functional status. In Handbook of psychiatric measures and outcomes (pp. 357-392). Washington, DC: American Psychiatric Association.

Bums, B. J., Schoenwald, S. K., Burchard, J., Faw, L., & Santos, A. (2000). Comprehensive community-based interventions for youth with severe emotional disorders: Multisystemic therapy and the wraparound process. Journal of Child and Family Studies, 9, 283314.

Carney, M. M., & Buttell, F. (2003). Reducing juvenile recidivism: Evaluating the wraparound services model. Research on Social Work Practice, 13, 551-568.

Clark, H. B., Prange, M., Lee, B., Stewart, E., McDonald, B., & Boyd, L. (1998). An individualized wraparound process for children in foster care with emotional/behavioral disturbances: Follow-up findings and implications for a controlled study. In M. E. Epstein, K. Kutash, & A. Duchnowski (Eds.), Outcomes for children and youth with behavioral and emotional disorders and their families: Programs and evaluation best practices (pp. 513-542). Austin, TX: PRO-ED.

Cohen, J. A., Mannarino, A. P., & Knudsen, K. (2005). Treating sexually abused children: 1 year follow-up of a randomized controlled trial. Child Abuse ? Neglect, 29(2), 135-145.

Curtis, N. M., Ronan, K. R., & Borduin, C. M. (2004). Multisystemic treatment: A meta-analysis of outcome studies. Journal of Family Psychology, 18(3), 41 1-419.

DeKraai, M., Hoffman, S., Dillion, Y, Handley, T, Baxter, B., & Tvrdik, A. (2004, March). The impact of multisystemic therapy on children within a system of care. Paper presented at the 1 7th Annual Research Conference: A System of Care for Children’s Mental Health-Expanding the Research Base, Tampa, FL.

Dishion, T. J., & Dodge, K. A. (2005). Peer contagion in interventions for children and adolescents: Moving towards an understanding of the ecology and dynamics of change. Journal of Abnormal Child Psychology, 33(3), 395^MX).

Evans, M., Armstrong, M., Kuppinger, A., Huz, S., & Johnson, S. (1998). A randomized trial of family-centered intensive case management and family-based treatment: Outcomes of two communitybased programs for children with serious emotional disturbance. Tampa: University of South Florida, College of Nursing.

Farmer, E. M., Dorsey, S., & Mustillo, S. A. (2004). Intensive home and community interventions. Child and Adolescent Psychiatric Clinics of North America, 13, 857-884.

Faw, L. (1999). The state wraparound survey. In B. J. Bums & S. K. Goldman (Eds.), Systems of care: Promising practices in children’s mental health, 1998 series: Volume IV. Promising practices in wraparound for children with serious emotional disturbance and their families (pp. 61-66). Washington, DC: Center for Effective Collaboration and Practice, American Institutes for Research.

Henggeler, S. W., Rowland, M. D., Halliday-Boykins, C, Sheidow, A. J., Ward, D. M., Randall, J., et al. (2003). One-year follow-up of multisystemic therapy as an alternative to the hospitalization of youth in psychiatric crisis. Journal of the American Academy of Child and Adolescent Psychiatry, 42(5), 543-55 1 .

Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (1998). Multisystemic treatment of antisocial behavior in children and adolescents. New York: Guilford Press.

Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneau, E. J., & Edwards, D. L. (2002). Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child and Adolescent Psychology, 3 /,155-167.

Herrenkohl, T. I., Hawkins, D., Chung, I., Hill, K. G., & BattinPearson, S. (2001). School and community risk factors and interventions. In R. Loeber & D. P. Farrington (Eds.), Child delinquents: Development, intervention, and service needs (pp. 21 1- 246). Thousand Oaks, CA: Sage.

Hoagwood, K., Bums, B. J., Kiser, L., Ringeisen, H., & Schoenwald, S. K. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52(9), 1 179-1 189. Hodges, K. (1996). CAiW and adolescent functional assessment scale. Ypsilanti, MI: Department of Psychology, Eastern Michigan University.

Holden, E. W., Friedman, R. M., & Santiago, R. L. (2001). Overview of the national evaluation of the comprehensive community mental health services for children and their families program. Journal of Emotional and Behavioral Disorders, 9, 4-12.

Horowitz, S., Hoagwood, K., Stiffman, A. R., Summerfeld, T., Weisz, J. R., Costello, E. J., et al. (2001). Reliability of the services assessment for children and adolescents. Psychiatric Services, 52(8), 1088-1094.

Institute of Medicine. (2006). Improving the quality of health care for mental and substance-use conditions. Washington, DC: National Academy of Sciences.

Leshied, A., & Cunningham, A. (2002). Seeking effective interventions for serious young offenders: Interim results of a four- year randomized study of multisystemic therapy in Ontario, Canada. London: Centre for Children & Families in the Justice System.

Littell, J. H. (2005). Lessons from a systematic review of effects of multisystemic therapy. Children and Youth Services Review, 27(4), 445-463.

Lonigan, C. J., Elbert, J. C, & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children: An overview. Journal of Clinical Child Psychology, 27(2), 138-145.

National Advisory Mental Health Council’s Workgroup on Child and Adolescent Mental Health Intervention Development and Deployment. (2001). Blueprint for change: Research on child and adolescent mental health. Rockville, MD: National Institute of Mental Health.

New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America. Rockville, MD: Department of Health and Human Services. (DHHS Publication Number: SMA-03-3832).

Newton, R., Litrownik, A., & Landsverk, J. ( 1 999). Children and youth in foster care: Disentangling the relationships between problem behaviors and number of placements. Child Abuse & Neglect, 24, 1363-1373.

Nugent, W., & Glisson, C. (1999). Reactivity and responsiveness in children’s services systems. Journal of Social Service Research, 25, 41-60.

Ogden, T, & Halliday-Boykins, C. A. (2004). Multisystemic treatment of antisocial adolescents in Norway: Replication of clinical outcomes outside of the U.S. Child and Adolescent Mental Health, 9(2), 77-83.

Pullmann, M. D., Kerbs, J., Koroloff , N., Veach- White, E., Gaylor, R., & Sieler, D. (2006). Juvenile offenders with mental health needs: Reducing recidivism using wraparound. Crime and Delinquency, 52, 375-397.

Rowland, M. D., Halliday-Boykins, C. A., Henggeler, S. W., Cunningham, P. B., Lee, T. G., Kruesi, M. J., et al. (2005). A randomized trial of multisystemic therapy with Hawaii’s Felix Class youth. Journal of Emotional and Behavioral Disorders, 13(1), 13-23.

Schaeffer, C. M., & Borduin, C. M. (2005). Long-term follow-up to a randomized clinical trial of multisystemic therapy with serious and violent juvenile offenders. Journal of Consulting and Clinical Psychology, 75(3), 445-153.

Schoenwald, S. K., Henggeler, S. W., Brondino, M. J., & Rowland, M. D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process, 39(1), 83-103.

Schuhman, E. M., Foote, R., Eyberg, S. M., Boggs, S., & Algina, J. (1998). Parent-child interaction therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clinical Child Psychology, 27, 34-35.

VanDenBerg, J., & Grealish, M. (1996). Individualized services and supports through the wraparound process: Philosophy and procedures. Journal of Child and Family Studies, 5(1), 7-2 1 .

About the Authors

LEYLA FAW STAMBAUGH, PhD, is an NIH postdoctoral fellow in the Services Effectiveness Research Program in the Department of Psychiatry and Behavioral Sciences at the Duke University School of Medicine. Her research is focused on treatment process and outcomes for children with emotional and behavioral disorders. SARAH A. MUSTILLO, PhD, is an assistant professor in the Services Effectiveness Research Program in the Department of Psychiatry and Behavioral Sciences at the Duke University School of Medicine. Her current work is focused on applying advanced statistical methods to various areas of child mental health research. BARBARA J. BURNS, PhD, is professor of medical psychology and director of the Services Effectiveness Research Program in the Department of Psychiatry and Behavioral Sciences at the Duke University School of Medicine. Her current research is focused on strategies to enhance implementation of effective clinical interventions for youth with severe emotional disorders. ROBERT L, STEPHENS, PhD, is a technical director at Macro International, Inc. He has helped conduct the national evaluation of the Center for Mental Health Service’s Comprehensive Community Mental Health Services for Children and Their Families Program for the past 7 years. BETH BAXTER, MS, is regional administrator at Region 3 Behavioral Health Services in Kearney, NE. DAN EDWARDS, PhD, is a clinical instructor in the Department of Psychiatry at the Medical University of South Carolina. He is also vice president and manager of Network Partners at MST Services, Inc., in Charleston, SC. MARK DEKRAAI, JD, PhD, is with the University of Nebraska Public Policy Center, where he is involved in projects related to child and family behavioral health and communitybased systems of care. Address: Leyla Faw Stambaugh, Department of Psychiatry & Behavioral Sciences, Box 3454, Duke University Medical Center, Durham, NC 27710; e-mail: lstambaugh@psych .duhs.duke.edu

Authors’ Notes

1 . This work was supported through a subcontract from Macro International, Inc., to Barbara J. Bums, PhD, at Duke University.

2. We would like to thank Ann Tvrdik at Region 3 Behavioral Health Services and Ye Xu at Macro International for their assistance with transfer of data and interpreting data codes.

Comments

comments