Carl Westin

4-3 Method 73 4-3-6 Dependent measures In relation to resolution advisories in the simulation, the following dependent mea- sures were collected: acceptance (binary, accept or reject), agreement rating (on a 1-100 scale), response time (from advisory onset to accept or reject button press), and perceived difficulty of each scenario (on a 1-100 scale). In addition, safety per- formance was measured in number of separation losses. Trust in the human and automated source was measured in two questionnaires. These were the source bias online questionnaire (SBQ) and the source bias Visual Analogue Scale (SBVAS). The SBQ was based on a commonly used survey for assessing operator trust in automation. 170 It consists of twelve statements that participants answer on a 7-point Likert scale ranging from 0 (not at all) to 7 (extremely). Participants were encour- aged to provide explanations to each of the twelve statements. All participants filled out two identical online questionnaires, one for each source encountered in the sim- ulation (human and automation). Translations of each statement were provided in participants’ native language (Swedish). VAS is a subjective questionnaire instrument with which participants indicate their agreement with a statement along a continuous line with two endpoints. In the SBVAS, endpoints were replaced with the different sources encountered in the simulator. Participants were instructed to indicate their preferred source (if any) in relation to eleven different statements, by making a mark on a 100 mm line associ- ated with each statement. A demographics questionnaire collected information on controllers’ age, gen- der, experience, and ratings held. A simulator questionnaire asked controllers if they thought advisories were disruptive, if they accepted advisories even when disagree- ing with them, and if they accepted advisories without prior inspection. Answers were collected on a 7-point Likert scale ranging from 0 (not at all) to 7 (extremely). 4-3-7 Procedure The experimental procedure followed a three-phase approach. The initial prequel simulation was conducted to record participants’ conflict solutions to the designed conflict. Following a brief welcome and introduction, the demographics question- naire and participant consent forms were completed. Participants received ten min- utes of briefing and forty minutes scripted experimenter-assisted training before playing the measurement scenarios. The measurement scenario, with the designed conflict, was repeated four times. To prevent scenario recognition, scenarios were rotated and waypoints renamed between repetitions. In addition, repetitions were intertwined with dummy scenarios. Debriefings indicated that participants did not recognize scenarios and, therefore, were unaware of the repetitions.