Hester van Eeren

| Chapter 5 5 | 102 The standardized bias was assessed before and after applying the PS to determine whether balance was achieved. The balance of the baseline covariates and missing indicators was assessed in the weighted sample. As a rule of thumb, it was assumed that balance was achieved when the standardized bias was less than .25 (Harder et al., 2010; Ho, Imai, King, & Stuart, 2007; West et al., 2014). The standardized bias of all covariates was carefully evaluated in addition to the balance of important, prognostic covariates (Ho et al., 2007). In addition to the standardized bias, the variance ratio and the five-number-summary of the continuous covariates were assessed to determine whether these distributions were similar in higher order moments (Austin, 2009). The distributions of the estimated variances are assumed to follow an F-distribution (Austin, 2009). The 2.5 th and 97.5 th percentiles can serve as a guide as to which variance ratios are tested to be equal between the treatment groups (Austin, 2009). The five-number summaries should also be used as a qualitative assessment because there is no method to test the similarity of these summaries between treatment groups (Austin, 2009). Analysis of treatment effect Regression analysis was used to estimate treatment effect estimates in the weighted sample. The treatment effect on the primary outcome measure — externalizing problem behavior measured with the CBCL—was estimated with an OLS regression on the outcome and the treatment indicator as an independent variable. The effect of interventions on the secondary outcome measures—living at home, being in school or having a job, new contact with the police—was analyzed with logistic regression analyses. The results were used to calculate average risk differences and risk ratios, as these measures are collapsible among subgroups. Odds ratios are not collapsible, meaning they are not comparable when they result from analyses with different sets of covariates or over different subgroups (Goossens, Redekop, & van Gils, 2015). These measures were estimated using ordinary cross tabs of the outcomes and treatment indicators in the weighted sample. For example, for the outcome ‘living at home after treatment’, the risk ratio was estimated as the probability of living at home after MST divided by the probability of living at home after FFT. The risk difference is the difference between these probabilities, estimated as the probability of living at home after MST minus the probability of living at home after FFT. For ‘engaged in school or work’ and ‘new police contacts’, the probability of being engaged in school or work and of having had police contact during treatment were looked at. The 95% confidence intervals of the final treatment effects were estimated using simple bootstrapping, as advised by Austin and Small (2014). In total, 5,000 bootstrap samples were drawn from the weighted sample and in each bootstrapped sample, treatment effects were estimated as described. A nonparametric percentile-based approach was used to define the 95% interval (Austin & Small, 2014). Regression analyses were done with and without adjustment for the covariates used to calculate the PS. Analyses with the treatment indicator as the only covariate in the study sample (N=697) were followed by analyses in the complete case sample (N=361; 132 FFT and 229 MST)—with and without all covariates—to overcome possible misspecification of this model and to assess whether results were robust (Harder et al., 2010; Rubin & Thomas, 2000). A case was determined ‘complete’ when all baseline data were available