Please take a moment to read this extremely informative article, “Flight Simulation Motion Systems: A Pseudoscience Imposed on Air Carriers”, published by Jean LaRoche, CQFA Director of R&D. This article is co-authored by Gavan Lintern who has a Ph.D. in Engineering Psychology (University of Illinois, 1978) with research focused on cognitive requirements for complex military platforms and aviation training systems.
LaRoche says: “Airlines are asked to spend billions of dollars on pseudoscience to train their pilots in flight simulators. Post-Pandemic rebuild will be the historic opportunity for regulators to rethink the certification requirements of commercial pilot training programs. Not only does simulation motion fail to significantly benefit pilots in transfer to the aircraft, but no-one has been able to demonstrate that the absence of simulator motion during training poses any risk to aviation safety. In half a century of research, there has never been the evidence to enable lawmakers to force air carriers to spend billions of dollars on synthetic motion.”
Learn more by visiting the article on LinkedIn, on ResearchGate, or by continuing your read below.
Airlines are asked to spend billions of dollars on questionable functions and gadgets to train their pilots in flight simulators. There is a historic opportunity for regulators to rethink the certification requirements of commercial pilot training programs.
January 27th, 2021- Despite billions spent and half a century of research, there is no proof, however small, of any benefit related to simulator motion for the safety of the travelling public. The last burden air carriers need today are useless operational costs. If it is true that life will not be the same after the pandemic, the time has come for regulators to align regulatory requirements and pilot training standards with proven scientific evidence.
The first known flight simulator was built in England in 1910 when Captain Haydyn Arnold Sanders developed his Sanders Teacher. Made from spare Sanders aircraft parts, the installation perched atop a pylon exposed to the winds. At the end of the 1920s, Edwin A. Link created the first American flight simulator; the Link Trainer, which, further developed over the next decade, was used to instruct hundreds of Allied pilots during WW II. Starting in the 1950s, advances in servo controls and analogue and then digital computers allowed the development of the modern simulators we know today.
The first synthetic motion systems rotated a few centimeters around the roll/pitch axes and used a cam rotor to simulate light turbulence. Faced with the enthusiasm of the candidates trained with these systems, simulator manufacturers developed ever more complex and costly sensory devices. Although none of the simulated movements ever really reproduced the movements of an airplane in flight, both the candidates-in-training and the instructors were convinced of their usefulness. Civil aviation authorities, seduced by the momentum of pilot enthusiasm, mandated motion devices in training standards (e.g., Federal Aviation Administration, 2021).
Concurrently, however, the accepted relevance of synthetic movements has encouraged questions about the level of fidelity required to optimize transfer of learning from the simulator to the aircraft. Gerathewohl (1969), in an FAA funded review of the training effectiveness of simulation fidelity, amalgamated civilian, military and space training issues, and compared Full Flight Simulators (with platform motion) to Part-Task Trainers (without platform motion). He concluded that motion is essential for at least some types of flight training.
The false promise
Nevertheless, some research scientists have questioned the educational merits of tricking the vestibular system. Are pilots trained in motion simulators better commanders of a fixed-wing aircraft? In chorus, pilots, manufacturers and legislators communicate their commonsense response: “Yes, of course! The plane moves, the simulator must move.” Scientists know, however, that common sense is science’s worst enemy.
Eddowes (1978) compared student pilots trained in Flight Training Devices (without motion) to those trained in Full Flight Simulators (with motion). His results converge on those of his contemporaries: training with motion produces better simulator pilots, but that advantage does not carry over to the airplane where those trained without motion perform just as well. Why is this the case? Schmidt & Wulf (1997) have noted that an increasing dependence on simulator sensory stimuli occurs at the expense of the transfer to real flight. Although there is no clear evidence that training with a motion system increases the dependence on sensory stimuli in a way that negatively impacts piloting performance, that is somewhat surprising. In flight training of recovery from unusual attitudes, students must learn to focus on their instruments and ignore the sensations from their inner ear. In principle at least, training in a simulator without motion would seem to offer some benefit. Conceivably, researchers have not found such a benefit because they have not developed experimental protocols that would test for it.
Nevertheless, the main take-home point from the research that has been done is that training with a motion system does not benefit performance in the airplane, at least for fixed wing aircraft and for pilots with the levels of experience as normally employed by airlines. An excellent demonstration of the false promise of synthetic movements is found in a rigorous transfer experiment by Koonce (1979). Three groups of pilots underwent simulator training program without knowing the real purpose of the research. One group was trained in a simulator with full motion and another with washout motion that provided more realistic roll-onset cues and avoided unrealistic side forces during sustained banked attitudes. A control group received the same training in the same simulator but with the motion switched off.
All pilots were evaluated by two pilot examiners on standardized assessment scenarios in the simulator during training and then airborne on the same assessment scenarios. Koonce writes: “The performance of subjects studied in the simulator tends to differ as a function of the type of motion cues provided by the simulator. (…) the no motion condition resulted in poorer performance than the two motion groups in the simulator. ( …) The better performance in the simulator with motion can be attributed to the aiding of the pilot through alerting onset cues provided by the motion system. The no motion pilots must acquire all of their information with respect to the vehicle’s attitude or changes in attitudes from the instruments only.” From the in-flight assessments, Koonce measured the real transfer of learning on board the aircraft. He then concludes: “There were no significant differences between the three groups for both VFR and IFR maneuvers in the aircraft.” In this experiment, there was no discernible transfer benefit for training with motion.
Arguments in favor of synthetic movements
Three main arguments are invoked in favor of motion (Vaden, 2005). First, the theory of identical elements (Thorndike, 1903) implies that, to obtain the best possible transfer, a high degree of fidelity of the learning environment is required. Based on his verbal learning research, Osgood (1949) has described a more nuanced relationship between model fidelity and transfer of learning. In flight simulation, the concept of fidelity has always remained difficult to define largely because almost all of the theoretical statements on transfer of learning are derived from studies of cognitive tasks with no thought to the fact that perception and action have a role to play. Although accepted theories support the fidelity view, they are based on a restricted type of data that does not adequately represent the range of activity required in piloting an aircraft. In addition, it is difficult to identify fidelity requirements for flight training because of the diversity of the sorts of things pilots have to do such as carrier landings and nap-of-earth flight. Some basic research on manual control task has shown better transfer from training systems that are less similar than the target task (Lintern, 1991), thereby suggesting that it is more important to focus on designing a simulator to maximize the training effect than to reach for high fidelity.
The second argument in favor of motion is based in the observation that a simulator with motion is easier to fly than is a simulator without motion. As evident in the Koonce research, motion cues help pilots fly the simulator through the provision of motion onset cues. Also evident in the Koonce research is that this performance advantage in the simulator does not transfer to the aircraft. More generally, not only does simulation motion fail to significantly benefit pilots in transfer to the aircraft, but no-one has been able to demonstrate that the absence of simulator motion during training poses any risk to aviation safety. In half a century of research, there has never been the evidence to enable lawmakers to force air carriers to spend billions of dollars on synthetic motion. Aviation psychologist Sammy Szpic from Toulouse reports: “When we confound exercises and educational objectives, attention tends to focus on the development of tools, to the detriment of what must remain the basis: the improvement of skills. The real concern of any training action is to train usefully.” There is no evidence to suggest that public safety on board airliners is increased by mandating synthetic motion for pilot training.
The third argument relates to flight instructor and flight trainee preference. Subjective preference for motion simulators in the industry remains virtually unanimous. As early as 1978, J. R. Hall reported that pilots preferred simulators with synthetic movements, even when large format visual screens were installed. In 2007, the Air Line Pilots Association (ALPA International) published a Safety Committee Statement of Position which reads: “Motion is required because pilots operate in an arena of motion and the vestibular system provides them with the most powerful and rapidly sensed cue for self-motion control.” This opinion is still widely conveyed in industry symposia. Overall, experts seem to favor the use of stimuli from synthetic movements for pilot training. However, Bürki-Cohen et al. (2001) point out that the same experts admit that there is no scientific basis for their opinion. Let us add that those same experts systematically omit talking about the costs.
A quote from an earlier paper will serve to summarize our thoughts on these three arguments. “The thinking and debate about the value of motion systems for flight-training simulators must stand as one of the significant failures of human factors. Solid empirical work continues to be ignored in the design of new systems while most current research effort is directed toward issues of marginal relevance. The analytical studies that have been done appear to be motivated more by the desire to justify procurement decisions that have already been made than to assess whether such purchases are warranted.” (Lintern, 1987).
Furthermore, the expense associated with the deployment of high-fidelity, full-motion simulators draws resources away from other important instructional concerns. Quality of instructors can have considerable impact on training outcomes, yet little effort is put into ensuring the quality of instructors. Typically, pilots become instructors because of a combination of seniority and preference. Little work is put into ensuring that those who can become skilled teachers are selected and then trained for the job. Most significantly, despite the fact that instructors have an important role to play in teaching skills such as Crew Resource Management, studies of simulator fidelity have ignored the role of the instructor.
Costly Enthusiasm
Synthetic motion systems are the result of R&D going back several decades. Beyond reducing latency by a considerable amount over the past 20 years, the design, digital algorithms, and manufacturing have been amortized over a very long time. In a captive market like that of certified Full Flight Simulators (since air carriers have a legal obligation to use them), the relationship between the purchase price and the manufacturing cost is not necessarily linear. Full Flight Simulators cost up to 10 times the price of equivalent fixed-base Flight Training Devices; they require more robust builds to endure decades of accelerative forces. The premature wear of on-board components incurs recurrent costs for the operators. Add in the requirement for oversized buildings to house the motion platforms. Further, it is futile to consider installation in a high-rise building since Full Flight Simulators have to be anchored on a considerably reinforced concrete floor. The training centers are deployed over large surfaces often on premium sites near airports. On top of paying for electromechanical maintenance of the motion actuators, operators have to manage and ensure the risk of rocking up to 14 tons of steel in a closed room, with human beings inside and around the simulators.
Hundreds of research projects have attempted to quantify, without success, the return on investment of the billions of dollars that the industry has expended for the purchase of Full Flight Simulators. When investing 10 times more in an industrial process, shouldn’t we expect a return at least 10 times higher? In fact, few topics in aviation psychology have been researched as much as synthetic movements. In a study at the Massachusetts Institute of Technology, Go et al. (2000) concludes: “(….) this study indicates that the motion (…) does not, in an operationally significant way for the tasks tested, affect First Look evaluation, Training progress, or Transfer of training.“
Bürki-Cohen et al. (2011) described the enthusiasm of pilots for synthetic motion and analyzed each of the elements retained by ALPA in 2007, which include the arguments of the simulator manufacturers. They concluded that budgets assigned to affordable training devices, especially in view of a shortage of experienced pilots, would have a greater impact on the safety of the traveling public than the use of ever-more expensive motion devices.
Davidson (2018) published a summary of research projects focusing on type rating B747-400 and involving the FAA, NASA and US Department of Transportation: “The use of motion platforms in both training and evaluation did not have a significant effect (…) The authors concluded that the use of motion platforms did not have significant effect upon the outcome. They suggest that this information would assist to finding more cost-effective solution to today’s pilot training need. “
Stanley N. Roscoe, Ph.D., considered one of the fathers of aviation psychology, as well as being the author of the book Flightdeck Performance, The Human Factor, denounced synthetic motion during his last visit to the CQFA of Saint-Honoré, Quebec: “The chief pilots have so well sold the hyper fidelity of the simulators to their pilots that the latter always ask for more. Legislators take part in the dance each time they publish national training standards imposing hyper fidelity. But none asks the question: Is it really effective? Scientists know very well that they can turn off the motion without interfering with crew training, except when a Civil Aviation Inspector is around! “
Beyond the number of pilots to be trained and checked annually, the high acquisition costs, physical constraints of installation, and operating costs of Full Flight Simulators dictate the number of simulators an airline will install. In a market-driven industry, profitability is always a concern. Training expenses must be managed carefully. Management of those expenses is challenged when equipment costs comprise an unnecessarily large component of total training expenses. This leads to a situation in which training equipment must be tightly scheduled for maximum and efficient use. In practice, airline simulators are scheduled for 20 hours a day with incoming crews required to wait on the bridge to minimize changeover time with outgoing crews.
This tightly scheduled demand for simulator time leaves little room for improvisation when additional training is required, or when a Pilot Proficiency Check is incomplete. In the 30 years that JL (first author) has been training approved check pilots and airline training captains, the lack of time remains a universal criticism. Simulator schedules are so busy that instructors and check pilots have to compress their work down to the minute at the expense of training objectives. Instructors and check pilots are unanimous that chronic lack of time impedes contemporary training of Crew Resource Management and Threat and Error Management.
Among large national carriers, acquisition of more affordable Flight Training Devices would relieve the time pressure on instructors and check pilots — much like adding new classrooms in a school. Decent training credits for using Flight Training Devices would allow specialized carriers who cannot afford prohibitively expensive Full Flight Simulators to develop in-house training programs adapted to their operational specificity at a lower cost. Further, this would allow much needed geographic and scheduling flexibility. Subsequently, budgets could be directly allocated to operational safety.
Fix the historical error
Recent studies of the airline industry have shown an association of between economic performance and safety (Fardnia, Kaspereit, Walker & Xu, 2020; Kalemba & Campa-Planas, 2019). It is unclear from those studies whether good economic performance leads to safety or vice versa, but most likely, both emerge from high-quality organizational management; what Weick and Sutcliffe (2001) would view as organizational mindfulness. The COVID-19 pandemic has financially challenged even the best managed airlines. Clear thinking in relation to motion systems for flight training simulators offers an opportunity for organizationally mindful airlines to strengthen their training programs as they trim costs, thereby offering opportunities to enhance both safety performance and financial performance with one evidence-based decision.
Based on 50 years of research, a historic opportunity presents itself to rectify training program certification. The first step would be to allow air carriers to deactivate the motion system on simulators and avoid further expenditures on maintenance of those systems. Ultimately, legislators must question the merit of each requirement, whether it be certified training tools, lesson content, or competency checking schedules and standards. The money wasted on non-essentials will never serve the traveling public.
About the authors
Jean LaRoche, CQFA Director of R&D, has been training Canadian Approved Check Pilots, Training Captains, and Inspectors for 30 years. He is a Fellow of the Royal Aeronautical Society and the AQTA 2019 Fecteau Award. He coauthored the WOMBAT Situational Awareness and Stress Tolerance Test. Founded in 1968, CQFA is the national aeronautics institute of Quebec’s Ministry for Education, Higher Education, Research and Science.
Gavan Lintern has a Ph.D. in Engineering Psychology (University of Illinois, 1978). His research has focused on cognitive requirements for complex military platforms and aviation training systems. Gavan is a Fellow of the Human Factors and Ergonomics Society and serves on the editorial board of Cognitive Engineering and Decision Making. He retired in 2009. He holds an adjunct appointment at Monash University Accident Research Centre in Melbourne, Australia, otherwise filling in as minder of the home pets and general home roustabout.
References
ALPA (2007). The Need for Motion in Flight Simulation. Safety Commitee Statement Position. Air Line Pilots Association, International. Washington, D.C.
Bürki-Cohen, J., Go, T.H., & Longridge, T. (2001, August). Flight simulator fidelity considerations for total air line pilot training and evaluation (AIAA Paper 2001-4425). AIAA Modeling and Simulation Technologies Conference, Montreal, QC, Canada
Bürki-Cohen, J., Sparko, A.L., & Bellman, M. (2011). Flight Simulator Motion Literature Pertinence to Airline-Pilot Recurrent Training and Evaluation. USDOT/TITA/Volpe National Transportation Systems Center, Cambridge, MA
Caro, P.W. (1976). Some Factors Influencing Tranfer of Simulator Training. Third Flight Simulation Symposium of the Royal Aeronautical Society, London, England
Davidson, P.J. (2018). A summary of studies conducted on the effect of motion in flight simulators pilot training. AviationFocus, Hong Kong
Eddowes, E. E. (1978). Flight simulator fidelity and flying training effectiveness. In D. E. Erwin (Ed.), Psychological fidelity in simulated work environments (pp. 45-56). Symposium conducted at the annual meeting of the American Psychological Association, Toronto, Canada.
Fardnia, P., Kaspereit, T., Walker, T. and Xu, S. (2020). Financial performance and safety in the aviation industry. International Journal of Managerial Finance, Vol. 17 No. 1, pp. 138-165. https://doi.org/10.1108/IJMF-03-2019-0095
Federal Aviation Administration (2021). PART 60—Flight Simulation Training Device Initial and Continuing Qualification and Use. Author: Electronic Code of Federal Regulations (eCFR)).
Gerathewohl, S.J. (1969). Fidelity of Simulation and Transfer of Training: A Review of the Problem. Federal Aviation Administration, Office of Aviation Medicine.
Go, T.H., Bürki-Cohen, J., & Soja, N.N. (2000). The Effect of Simulator Motion on Pilot Training and Evaluation. (AIAA-2000-4296) AIAA Modeling and Simulation Technologies Conference, Denver, CO
Hall, J.R. (1989). The Need for Platform Motion in Modern Piloted Flight Training Simulators. Technical Memorandum, FM35. Royal Aerospace Establishment, p. 12.
Kalemba, N. and Campa-Planas, F. (2019). Safety and the economic and financial performance in the airline industry: is there any relationship? Aviation. 23, 1 (Apr. 2019), 7-14. DOI:https://doi.org/10.3846/aviation.2019.9744
Koonce, J.M. (1979). Predictive validity of flight simulators as a function of simulator motion. Human Factors. Vol 21(2), 215 – 223.
Lintern, G. (1991). An informational perspective on skill transfer in human-machine systems. Human Factors, 33, 251-266.
Lintern, G. (1987). Flight Simulation Motion Systems Revisited. Human Factors Bulletin. Vol 30, No. 12 (Décembre).
Lintern., G., Roscoe, S.N., & Sivier, J.E. (1990). Display Principles, Control Dynamic, and Environmental Factors in Pilot Training and Transfer. Human Factors, 32(3), 299-317
Myers, P.L., Starr, A.W., & Mullins, K. (2018). Flight Simulator Fidelity, Training Transfer, and the Role of Instructors in Optimizing Learning. International Journal of Aviation, Aeronautics, and Aerospace. 5(1)
Osgood, C.E. (1949). The similarity paradox in human learning: A resolution. Psychological Review, 56, 132-143
Roscoe, S.N., O’Hare, D. (1990). Flightdeck Performance, The Human Factor. The Iowa State University Press, Ames, Iowa
Roscoe, S.N. (2005). Private communication. Centre québécois de formation aéronautique CQFA, Saint-Honoré, QC, Canada
Schmidt, R.A., Wulf, G. (1997). Continuous Concurrent Feedback Degrades Skill Learning: Implication for Training and Simulation. Human Factors 39(4), 509-525
Szczepanski, Leland (2000). Move or not to move? Continous question. AIAA Modeling and Simulation Technologies Conference, Denver, CO. (AIAA No. 2000-4297).
Szpic, S. (2021). Private communication. Toulouse, France
Thorndike, E.L. (1903). Educational psychology. New York: Lemcke and Buechner
Tiauw, H. Go., Bürki-Cohen, J., Soja, N.N. (2000). The Effect of Simulator Motion on Pilot Training and Evaluation. AIAA Modeling and Simulation Technologies Conference, Denver, Colorado, 14-17 August 2000.
Vaden, E.A., Hall, S. (2005). The Effect of Simulator Platform Motion on Pilot Training Transfer: A Meta-Analysis. International Journal of Aviation Psychology, 15(4), 375-393.
Weick K.E. & Sutcliffe, K.M. (2001). Managing the unexpected: Assuring high performance in an age of complexity. San Francisco, CA: John Wiley.
