Els van Meijel

33 Evaluation of the STEPP | Chapter 2 Performance of the STEPP The STEPP showed moderate discriminative ability for child PTSD, with areas under the curve for diagnosed PTSD of 0.68 (95% CI 0.53-0.82) and for self-reported PTSD symptoms 0.69 (95% CI 0.56-0.81). The parent score resulted in an AUC of 0.59 (95% CI 0.43-0.75) for self-reported PTSD symptoms, which is too low to discriminate. Results of the ROC analyses are presented in Table 2.2, showing the accuracy (sensitivity and specificity) and the positive and negative predictive values for different cut-off values for the STEPP. Because a screening instrument should basically identify all cases (maximize sensitivity), the STEPP showed optimal performance in detecting children and parents with PTSD at a cut-off value of 2. High negative predictive values should screen out those who are unlikely to develop PTSD. We therefore had to accept poor specificity, which could lead to false positives. Intra-rater reliability was tested for a categorical score (‘At risk’ or ‘not at risk’) based on the cut-off score. At the original cut-off scores (4 for children and 3 for parents), intra-rater reliability showed moderate agreement for both the child and parent part (κ = 0.46 and 0.45 respectively). The differences in answering question 4 (“Does your child have any behavior problems or problems paying attention?”) and question 7 (“When you got hurt, or right afterwards, did you think you might die?) were responsible for two additional cases with positive scores at the second assessment. We found no systematic pattern of discrepancy between test and re-test assessment for either of the items. In one of the cases, even question 2 (“Were you with your child in an ambulance or helicopter on the way to the hospital?”) was answered differently. When using the adjusted cut-off scores of 2, intra-rater reliability improved to substantial for the child part (κ = 0.66) and to almost perfect for the parent part (κ = 0.83).