Joeky Senders

121 Automating clinical chart review be excluded. Likewise, other statistical and lexical properties might have a significant influence on model performance and deserve further investigation as well. Despite these limitations, this study provides an open-source NLP pipeline, as well as unique insights into the statistical requirements, for text mining in clinical research. Future studies In order to promote the transparency, reproducibility, and external generalizability of NLP research in healthcare, we advocate that future studies release their source code on a publicly-available repository. Given its strong correlation with model performance, we suggest that future studies report measures of interrater agreement whenever ground truth is based on a consensus rating of human annotators. Further investigating the association between model performance and statistical or lexical properties allows for careful selection of variables eligible for text mining, as well as tailoring the NLP approach to the nature of the clinical text corpus at hand. Conclusion The current study has developed an open-source NLP pipeline for automated variable extraction, which can guide the development of text mining frameworks for other patient cohorts, medical reports, and clinical characteristics as well. In the current sample, model performance was correlated with the interrater agreement of the manually provided labels rather than the frequency distribution of the variables of interest. Class imbalances up to a 9:1 ratio, as well as 50-100 observations in theminority group, should therefore not be considered as contraindications for clinical text mining. Future studies should report measures of interrater agreement whenever ground truth is based on a consensus rating of human annotators and employ open-source coding to promote the transparency, reproducibility, and external generalizability of NLP research in healthcare.