225 Reproducibility study on invasiveness of pulmonary AdC with a modified classification pathologist’s feeling of (un)certainty about the decision of invasion in each case. The mean and standard deviation of the case uncertainty score was 0.5 ± 0.57 (range 0-2.0). All 25 cases with an uncertainty score >0.4 revealed discordant annotations with invasion “yes” and invasion “no” in round 1 and 2. From the 34 cases with an uncertainty score <0.2 the number of discordant annotations with invasion “yes” and invasion “no” in rounds 1, 2 and 3 were 3, 2 and 0, respectively. The correlation analysis for the average uncertainty score is shown in Table 4. Table 4. The pathologists assigned a score for the feeling of uncertainty regarding the decision of invasion. The spearman rank correlation coefficient for the average uncertainty score per case is shown including p-values for several parameters. As expected, the correlation with total tumor size is low. Note that the correlation is high (>0.5, [p<0.001) for invasive size measurements (first drawn invasive line: invasive size 1), sum of invasion scores if more than one line was drawn (total invasive size), longest invasive line if more than 1 invasive line was drawn (maximum invasive size), percentage of non-invasive pathologists and a maximum agreement score (irrespective invasion “yes” or “no”) score. round 1 round 2 Round 3 correlation p-value correlation p-value correlation p-value total tumor size -0,069 0,571 -0,118 0,332 -0,033 0,784 invasive size 1 .824** <0.001 .849** <0.001 .847** <0.001 total invasive size .831** <0.001 .852** <0.001 .835** <0.001 maximum invasive size .825** <0.001 .848** <0.001 .842** <0.001 total tumor size (only invasion=”yes”) -0,032 0,790 -0,101 0,405 0,021 0,865 invasive size 1 (only invasion=”yes”) .556** <0.001 .481** <0.001 .462** <0.001 total invasive size (only invasion=”yes”) .777** <0.001 .734** <0.001 .692** <0.001 maximum invasive size (only invasion=”yes”) .655** <0.001 .572** <0.001 .564** <0.001 percentage non-invasive pathologists .802** <0.001 .871** <0.001 .804** <0.001 maximum agreement score -.823** <0.001 -.892** <0.001 -.841** <0.001 correlation = Spearman’s rank correlation with uncertainty score per case Feedback and Round 3 After round 1 and 2 feedback consisted of i) feedback of the assessments from peers (41 pathologists), ii) case specific heatmaps with underlying histology and comments, iii) detailed comments on areas of likely biological and iatrogenic collapse, iv) a detailed search for foci of invasion, v) assessment of the presence of elastin (usually not associated with papillary or acinar, unless desmoplastic stroma is mixed with tumor cells)206, and vi) information about presence or absence of recurrence in follow-up. To examine if the additional knowledge leads to an improvement in reproducibility a third 15
RkJQdWJsaXNoZXIy MTk4NDMw