Marianne Welmers

Alliance and Treatment Outcome 29 CHAPTER 2 Study quality was coded and assessed using a Study Quality Checklist (SQC) developed by the third and first author of this article based on the Quality Assessment Tools for Quantitative Studies (QATQS; Thomas, Ciliska, Dobbins, & Micucci, 2004), the Quality Index (QI; Downs & Black, 1998), and the Cochrane Collaboration’s tool for assessing risk of bias (Higgins et al., 2011). The SQC allows the rating of 15 criteria per study on publication status, selection bias, pretest differences, missing data, reliability and validity of process measures, reliability and validity of outcome measures, attrition, study- dropouts and report on treatment and sample size characteristics. Total SQC scores ranged from6 to 30 on a 0 (low) to 45 (high) scale. In order to assess interrater reliability of the SQC, 22 out of 28 included studies were independently coded by the first author and a master’s graduate student in Forensic Child and Youth Care Sciences. The ICC was .95. Calculation of Effect Sizes and Statistical Analyses For each study, Pearson’s r was calculated to estimate the correlation between alliance and outcome. In cases where two treatments were compared with one of them being a family-involved treatment, Pearson’s r was calculated only for the sample that received family-involved treatment. Most effect sizes were calculated based on reported standardized regression coefficients, Pearsons r correlations, and means and standard deviations for treatment completers and dropouts. All calculations were based on formulas of Borenstein, Hedges, Higgings, and Rothstein (2009), Lipsey and Wilson (2001), Rosenthal (1991), Rosenthal (1994), and Rosenthal and DiMatteo (2001). If effect sizes could not be calculated based on the information in the study report, authors were contacted to retrieve additional information. In seven studies, the study reported non- significant correlations, but did not provide sufficient data to calculate an effect size. In these cases, the value of zero was assigned ( n = 47 effect sizes), which is considered a conservative estimate of the true effect size (Rosenthal, 1995). Furthermore, effect sizes were coded as positive if correlations were in the expected direction (i.e., higher levels of alliance, alliance improvement, or lower levels of unbalanced alliance were related tomore positive therapy outcome), whereas correlations not in the expected directionwere coded as negative. In total, 361 effect sizes were computed. Effect sizes on alliance change scores and outcome ( n = 15, k = 3 studies) and on split alliance and outcome ( n = 17 from k = 5 studies) were each analyzed in separate meta-analyses because of the different nature of the alliance. To prevent extreme effect sizes or moderating variables from having a disproportionate influence on the statistical analyses, we searched effect sizes and continuousmoderators for outliers (standardized scores higher than 3.29 or below -3.29; Assink & Wibbelink, 2016). No outliers were found.