Esther Mertens
| 119 Components of School-Based Interventions Data Extraction Studies were coded for information concerning the study (e.g., year of publication, country where study was conducted), sample (e.g., age, gender distribution), design and method (e.g., randomization, attrition analyses), intervention (e.g., intervention provider, aim of intervention), effect size data (e.g., outcome category), and intervention components (e.g., problem solving, practice, parental involvement). The intervention components were based on the reviews and meta-analyses by Boustani and colleagues (2015), Kaminski and colleagues (2008), Onrust and colleagues (2016), Peters and colleagues (2009) and Van der Put and colleagues (2018). An overview of all components and their definitions is presented in Appendix A. Sources cited in the study and other freely available materials, such as descriptions from the developer or websites, were retrieved for coding the components (Boustani et al., 2015; Kaminski et al., 2008). In cases where insufficient data were reported for calculating the effect size, the first author was contacted. When this author had not responded after a reminder, the second or last author was contacted and, if necessary, reminded. If the required data could not be obtained after this, the study was excluded from the meta-analysis (see Figure 1 for the flow diagram). Of the included studies, 28% (30 studies) was coded independently by a second coder for reliability. The inter-rater reliability was moderate to excellent (Landis & Koch, 1977) with an average intra-class-correlation of .97 ( SD = .05), ranging from .88 to 1.00, for continuous variables, and an average Cohen’s kappa of .82 ( SD = .11), ranging from .60 to 1.00, for categorical variables. Coding of the component ‘Insight building’ was not reliable with Cohen’s kappa of .52. Disagreements between the two coders were discussed and solved unanimously. Calculation and Analyses of Effect Sizes Effect sizes were represented as Cohen’s d , reflecting the standardizedmean difference between the intervention and control condition, following the procedures of Lipsey and Wilson (2001). Effect sizes were calculated at post intervention (i.e., within 6 months after the intervention) and corrected for baseline differences. Positive effect sizes indicated better results for the intervention compared to the control condition. All effect sizes were adjusted using the Hedges’ (1983) small sample correction prior to analyses. Outliers were examined and, when believed to be unrepresentative, winsorized by replacing outliers with the value of the lower or upper value of two standard deviants from the mean (Lipsey & Wilson, 2001). 6
Made with FlippingBook
RkJQdWJsaXNoZXIy ODAyMDc0