15250-m-cuypers

70 Chapter 4 testing were affiliated to the initiating hospital, but not involved in any other stage of DA development. Care providers ages ranged from 35 to 60 and all had a minimum of five years of experience in their current position. All participants were instructed to use the DA from the perspective of a patient diagnosed with low-risk Pca and eligible for all four treatments covered in the DA (active surveillance, surgery, brachytherapy, external beam radiotherapy). No specific further usage instructions were given in order to let participants use the DA as naturalistically as possible. Participants were asked to think aloud when navigating through the DA and to mention every remark or difficulty they encountered during DA usage. This procedure is commonly used to investigate human-computer interactions and has been applied before for DA usability testing as well 31 . The usability test was run in two simultaneous sessions in the outpatient clinic of the initiating hospital, with two observers from the development team present in each session. The observers monitored if the participants’ verbalization matched their DA usage (e.g. saying navigation was easy accompanied by clicking on the correct buttons). As the DA only consists of a limited number of steps, if any action was not verbalized by the participant, a clarifying question was asked to the participant. During DA usage participants did not receive further feedback or other instructions from the observers. Each participant was given 30 minutes to use the DA followed by a fifteen minute semi-structured interview. The goal of the interview was to reflect on DA usage in addition to the comments made while using the DA. Interviews are commonly added to think-aloud procedures to ensure that the most important aspects have been covered during the usability test and to reduce the risk of bias in the interpretation of participants’ verbalizations 32 . The interview covered five questions asked to all participants: 1. ‘What were your expectations upfront?’ 2. ‘What is your first impression of the DA?’3.‘Was the information understandable and useful?’4.‘What were positive aspects?’ 5. ‘What can be improved?’ Only patients were then asked: 1. ‘Would you recommend this to other patients?’ and 2. ‘What feeling did the DA gave you?’ Care providers were asked if they would offer this DA to patients. Participants were then thanked for their participation and received a bottle of wine as token of appreciation for participating. Measures and Analysis As a first step, all notes from all sessions and observers were combined and labeled as either general comments about the DA or related to a specific section of the DA. All comments were then further categorized to Usability, Layout, Language, Content, Amount, Values Clarification or DA Summary. Next, the accuracy and urgency of all