Marianne Welmers

Chapter 2 36 Next, each correlation was transformed to Fisher’s Z before combined effect sizes were calculated and moderator analyses were conducted (Assink & Wibbelink, 2016), and transformed back into Pearson r after analyses for ease of interpretation. Effect sizes were interpreted following Cohen’s (1988) guidelines: The effect is considered small if r is at least .10, medium if r is at least .30 and large if r is at least .50. Most included studies report on multiple informants of alliance, multiple times of measurement and multiple outcomes. Therefore, for most studies more than one effect size was calculated. Traditional meta-analytic approaches are based on the principle that the included subject samples are independent and thus, including multiple effect sizes based on the same sample violates this principle (Lipsey &Wilson, 2001). However, following other recent meta-analyses (e.g., Assink et al., 2015; Van der Stouwe et al., 2014), amultilevel randomeffects model was used for the calculation of combined effect sizes and for the moderator analyses in order to account for dependency of effect sizes. This approach has been shown as superior to the fixed-effects approaches employed in traditional meta-analysis for models with moderators (Van den Noortgate & Onghena, 2003). In the present study, a three-level meta-analytic model was used for analysis of the data, modeling three sources of variance: sampling variance of the observed effect sizes (Level 1), variance between effect sizes from the same study (Level 2), and variance between studies (Level 3). This model was used to calculate an overall estimate of the association between level of alliance and therapeutic outcome, the association between alliance change scores and outcome and the association between split alliances and outcome in family therapy. Furthermore, it was used to obtain estimates of effect sizes by including moderator variables in the model to determine whether the observed variation was explained by study, sample or methodological characteristics of studies. To perform the statistical analyses using a three-level model, we followed guidelines as described by Assink and Wibbelink (2016). We used the function “rma.mv” of the metafor package in the R environment (version 3.3.1; R Core Team, 2016). The R syntax and protocol was written so that during the analyses three sources of variance were modeled. We used the t- distribution for testing individual regression coefficients of the meta-analytic models and for calculating the corresponding confidence intervals. To determine whether moderator analyses should be conducted, we applied the 75% rule of Hunter and Schmidt (1990). They state that when less than 75% of the total variance can be attributed to random sampling error (level 1), heterogeneity at level 2 (within studies) and 3 (between studies) can be considered substantial, and moderator