Marianne Welmers

Chapter 3 72 The Shared Sense of Purpose dimension can only be observed when two or more family members are in a session together, the other three dimensions can also be scored with only one family member present in a session. After observing the session, based on the frequency, intensity and clinical meaningfulness of the marked behaviors, coders assign global ratings on each dimension on a 7 point Likert scale ranging from -3 ( extremely problematic ) to +3 ( extremely strong ). For the purpose of this study, the SOFTA-o was translated from English to Dutch, following guidelines as prescribed by Van Widenfelt, Treffers, De Beurs, Siebelink, and Koudijs (2005). For coding the videotaped IPT-sessions with the SOFTA-o, we used the training manual by Friedlander et al. (2005) as a guideline. The first author received training from Valentín Escudero, one of the developers of the SOFTA. After training and translation of themanual, first author coded12 videotapeswith at least two familymembers in the session toprovide a coding standard for training other coders. Coding dilemmas from these 12 videotapes were discussed with Valentín Escudero. Next, 3master students of Educational Sciences were trained, receiving 15 hours of coder training over five weeks. They were introduced to the theoretical framework of the SOFTA, coding guidelines, practice material from the developers and Dutch practice material taken from the drama-series In Therapie ( In Therapy) . Trained coders independently coded at least 10 videotapes to increase their reliability as coders compared to the golden standard codings by first author and received feedback on each coding. As advised by Friedlander et al. (2005), training continued until coders differed no more than one point in their scale scores in 90% of the cases. After their training, each coder rated a random selection of the videotapes. Coding dilemmas were discussed and difficult parts were consensus coded duringmeetings with the coding team every two weeks. In total, 90 sessions were independently coded after training. Of these sessions, 14 random selected sessions (16.6%) were double coded by the first author, coders were blind to these double coded sessions. To assess interrater reliability, we calculated intra-class coefficients for the 14 double coded sessions using the singlemeasures of a two-way mixed effect model based on absolute agreement (Koo & Li, 2016). We first assessed reliability of the subscale scores as originally developed, that is, with a rating from -3 to +3 based on observed behaviors. Because these regular subscale scores had very little variance and because of this did not achieve sufficient interrater reliability, we chose to use a score for each subscale reflecting the number of observed positive behaviors per 60 minutes. ICC’s for these scores were .637 for Engagement, .551 for Emotional Connection, .558 for Safety, and -.129 for Shared Sense of Purpose. According to Cicchetti (1994), ICC’s are fair when >.4, good > .6 and excellent >.8. The negative ICC for Shared Sense of Purpose could be explained by a low number of randomselected sessions for double coding with two or more familymembers