Carl Westin

3-3 Method 47 Each (baseline) scenario featured only one designed conflict (and this was al- ways between two aircraft). Geometry of the designed conflict was varied only between baseline scenarios. The conflict pair was initially aligned with its respec- tive exit points and, thus, required no initial participant intervention. The other aircraft in the sector were considered “noise” aircraft to distract the participant from the conflict pair. Some noise aircraft were misaligned with their exit point and dis- played in grey, whereas aligned noise aircraft were displayed in green so that the participant could immediately see which aircraft had not yet been cleared to their respective exit point. Designing the conflict scenarios took a great deal of effort, and we faced several challenges to experimental control. For example, we did not want participants to solve conflicts earlier than the advisory, or for noise aircraft to disrupt the designed conflict. Arriving at our final experimental design and scenario set required a good deal of developmental testing and iterative fine-tuning. For instance, we had to de- rive conflicts for which the solution was not immediately obvious, so as to 1) invite a variety of solutions which would be required for our conformance manipulation, and 2) make the advice of “automation” non-trivial. We also had to create scenarios in such a way that they were geometrically comparable but not recognizable as such. Finally, and as discussed later, we had to devise a framework for comparing conflict solutions, for reasons of defining conformal and nonconformal solutions. For reasons of experimental control, we made certain simplifying assumptions. First, no wind conditions were taken into account. Second, all aircraft remained on the same flight level (290) and could not be changed. Third, aircraft motion was simulated by first order, linear kinematic equations. Finally, and as is often done in ATC simulations, the system ran at faster than real time, in this case 4x. 3-3-5 Experimental design We used a 2x2x2 design, with LOA, complexity and conformance all varied within subject. LOA was ultimately collapsed, and therefore excluded in statistical anal- ysis. Figure 3-3 illustrates the experimental design. We conducted a series of two human-in-the-loop simulations. The first prequel simulation captured participants’ manual performance in maintaining safe separation between aircraft. The subse- quent experiment simulation presented participants “automated resolutions,” which were, unbeknownst to the participants, actually replays from the prequel simulation. Half of these replays were a given participant’s very own previous solution (the con- formal condition), and half were replays of a colleague’s different but workable so- lution (the nonconformal condition). Participants were free to either accept or reject a given advisory and implement an alternative solution.