Anne van Dalen

88 I Chapter 3 learning algorithms flagged ‘near miss’ events in the dataset, and events were ‘tagged’ when they were considered to be relevant. Following, the dataset was analysed by the OR Black Box® analysis team (a specialized trained team of surgeons and human factors specialists) in full to double-check for fault-positive, negative and inappropriate placed flags of the learning algorithms in order to avoid faulty analysis. Since the software and analysing team uses English as primary language, the team was asked to speak English during the recording of the surgical cases. Study participants were told that they could always revert back to Dutch, if necessary. Yet, the debriefings were done in Dutch. As the contractor of the MDR resides in Canada, the Canadian analysis team was briefed about local standard operating procedures before start of study, by all the participating surgeons. The analysis was based on well-known, scientifically validated rating scales that can be found in literature, such as the System Engineering Initiative for Patient Safety (SEIPS) model of work system and patient safety, the Non-Technical Skills for Surgeons (NOTSS), The Scrub Practitioners’ List of Intraoperative Non-Technical Skills (SPLINTS) system and the Disruptions in Surgery Index (DISI). 23–26 This original ‘tagged performance report’ was considered to be too lengthy and granular for feasible debriefing the operating team, hence it was further translated into a graphical summarized performance report. This graphical performance report model compromised a summarized ‘video clip’ of about 10 min. Figure 2 shows an example of the OR Black Box® performance report. The video clip included the 2 overview camera’s, the anaesthesia monitor and laparoscopic camera as depicted in Figure 1 and 2. The structured feedback from the OR Black Box® analysis team (Toronto, Canada) was added to the summarized ‘video clip’ in annotations, including all relevant positive (green line) and negative events (red line) of the particular case. As shown in Figure 2, the timeline of the procedure and video clip is visualized in the lower part of the report. The green and red lines represent the positive and negative rated human factor events. The green or red squares within these lines represent a specific safety threat or resilience support event for which written feedback is provided in the right upper part of the report. These events were discussed during the team debriefing. All personally identifiable information was stripped from the performance report (faces