MR measure corresponded with the perceived contribution in each particular review case. Statistical analysis Inter‐rater reliability was measured with Cohen’s kappa statistic (kappa), weighted kappa statistic (wk), which accounts for ordered categories, percentage agreement and/or the intraclass correlation coefficient (ICC), depending on the measure. We calculated both the overall averaged kappa and an average kappa that controlled for hospital by adjusting for the hospital‐specific variances (17). We calculated the percentage agreement per category with the formula (2*a)/(2*a + b+c+d+g), where ‘a’ is the agreed number of cases for a category and ‘b’, ‘c’, ‘d’ and ‘g’ are the number of cases where only one reviewer assigned that category. In this article, ‘agreement’ refers to the initial agreement between the two reviewers, unless stated otherwise. We were interested in the ICC for absolute agreement and employed a two‐ way ICC, assuming that the raters’ effects will contribute to the variability of the ratings as random effects (18). To study the association between patient and HAI characteristics and the perceived contribution, we used the consensus value of the measure. When a consensus was not reached, the assessment of the TP was used. To diagnose contribution to death (3CAT and WHOCAT), we used a random forest classifier approach. A set of the best predictors was selected to achieve an optimal prediction accuracy. Using this set, we switched to model construction in order to assess the association between the variables and the categorical outcome by means of multinomial logistic regression. In the overall analysis of 3CAT, we could perform a multilevel analysis, allowing for clustering at the hospital level. With HAI‐specific subsets, these models usually did not converge. Variables that had a p‐value < 0.2 in the univariate analysis were included in the multivariate analysis. The final model was attained by manual backward selection, controlling the decrease in model fit with the ‐2log likelihood test. We used SAS software version 9.4 of the SAS system (SAS Institute Inc., Cary, United States) and R, version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria). Ethical statement The study protocol was submitted to the medical research ethics committee (MREC) of the University Medical Centre Utrecht. As the study was not interventional the need for further evaluation was waived. Participating hospitals also approved the study protocol. 110 Chapter 6
RkJQdWJsaXNoZXIy MTk4NDMw