The How, When, and What of Adaptive Training
Study at the U.S. Air Force Gaming Research Integration for Learning Laboratory® (GRILL®) aims to create a live adaptive simulation environment based on trainee performance and cognitive workload
This is a brief recap of the NTSA Tech Grove webinar, “The How, When and What of Adaptive Training.”
With the need to develop more specialized expertise across a variety of fields and faster time-to-readiness, organizations are beginning to pursue more adaptive and personalized learning.
Adaptive training can have a significant impact on training outcomes but can require more time and expense. Jonathan Diemunsch, computer scientist at the U.S. Air Force Gaming Research Integration for Learning Laboratory® (GRILL® ), and Dr. Summer Rebensky, Scientist and Capability Lead at Aptima, Inc., shared their insights on using commercial game engines to streamline and lower the cost of developing an adaptive training environment.
The aim of their study at the GRILL, where they leverage commercial off-the-shelf game technology and VR to solve Air Force training challenges, was to create a live adaptive simulation environment based on trainee performance and cognitive workload.
“We’ve been using game-based training for a little over a decade now and we find that it is a great way to prototype systems to show the art of the possible,” Diemunsch said. “We also find that the game-based systems can give you a very good sense of whether or not what you’re doing is headed in the right direction.”
They provided the following insights on identifying proficiencies and performance measures, developing an adaptive training logic, and employing different ways to adapt.
Why Adaptive Training?
Rebensky said the move to adaptive training reflects the reality that individuals bring different skills, experiences, and backgrounds to training. Training that works well for one may not work well for others. Adaptive training adjusts or aligns the content or scenario to the trainee’s needs.
Compared to more rote training, Rebensky said research shows that when trainees are put in the “zone of proximal development,” in which the content they receive is a bit more challenging than their skill level, training gains are higher. Alternatively, scenarios that are too difficult tend to lead to disengagement and lower performance, while scenarios that are too easy can bore trainees and reduce training effectiveness.
While traditional computer-based trainers often use simple pass/fail rules, adaptive training uses a more data-driven approach. To apply adaptive training in simulation, the system is designed to use data about the user’s performance and state to determine when and how to adapt any variable that has been parameterized, including the scenario, the environment, the weather, the state of the aircraft, or the number of adversaries.
In the adaptive study Diemunsch and Rebensky conducted at the GRILL, the goal was to leverage gaming tech to deliver the exact right experience to the individual. Individuals wearing biosensors were measured as they went through a driving simulation using a commercial-off-the-shelf game engine.
“Can we take someone’s physiological signals and live change the driving scenario in the middle, calibrating it to their stress levels and performance metrics? Can we dynamically update the environment?” Rebensky asked. She said the first step to building an effective adaptive simulator is to identify the skills that need to be trained and if they can be measured in a simulated environment.
“It’s important to have a complete picture of the proficiencies that make up readiness. Those need to be captured, trained, and assessed, otherwise the risk is sending someone in the field unprepared.”
Rapid Prototyping Using Game Tech
The advantage of game engines, Rebensky said, is that they can easily log various types of data used to determine proficiency. In the GRILL study, the goal was to capture metrics representative of good driving performance. Those included how fast subjects drove, how much they varied from the center of the road, the number of times they went off-road or when just one wheel touched off a narrow road.
Diemunsch added that another advantage of game engines is that they handle much of the heavy lifting for the physics and animation, allowing for rapid prototyping. He said the GRILL team often uses Unreal Engine for its ability to produce a more polished-looking product and its ability to work with numerous controllers without programming, making it easier to integrate with other technologies.
To keep workload high In the study, cognitive memory tasks that required the driver to respond to system alerts and radio calls were layered on top of the driving skills. Those tasks Rebensky explained, were designed to be mission-relevant, similar to IED warnings and incoming COMMs messages in a military vehicle.
In the first iteration, they collected preliminary data that included physical and cognitive performance, and physiological measures. These were then combined into a matrix using a simple rule set.
The matrix was then used to determine whether the next condition for the driver should become more or less difficult and whether assistance should be provided or not. Rebensky said the benefit of starting with a simple rule set to collect data is the ability to determine if there are other skills that should be measured and to identify areas where individuals potentially struggle.
“Using this type of scaffolding approach to design an adaptive system, you can start to see the values, where you’re getting them, and then invest in them further.” Rebensky went on to explain how a game engine can be quickly stood up as a testbed which helps to iterate and refine the design to find answers.
Adapting Scenarios, Feedback, and More
One type of live adaptation that the GRILL team explored was the roads. As road segments would spawn out and become more or less curvy, drivers had to work harder to stay within a narrow road while maintaining speed to rejoin their convoy. Rebensky explained how game engines make it easy to adapt within the scenario so that roads or other elements can change dynamically.
The ability to time-sync data from the physiological sensors with the simulation events provided a huge benefit, said Diemunsch. As physiological data was streamed into Unreal Engine and time-synced, the team could immediately see a trainee’s physiological response to a stimulus or event.
The result is, “We may end up changing the thresholds for when someone receives more or less difficult content, or when the pacing should scale or occur more or less frequently,” said Rebensky.
She added that an adaptive approach can also be used to adjust the level of feedback, so that trainees receive more tailored feedback at the beginning based on how they are doing and where they could improve.
For individuals that struggled to balance the cognitive tasks while driving, Rebensky said they also provided multi-modal cueing. “As their physio signals indicated they may be stressed, we provided auditory and visual assistance, so we were also able to explore some operational augmentation strategies as well.”
The Impact of AI
With the increasing amounts of data able to be collected, Rebensky said artificial intelligence (AI) and machine learning can play a key role in identifying what the trainee likely needs to experience next, which was the goal of the effort. The data can also be used to train AI models to deliver a more individualized approach down the line.
Diemunsch said sometimes using a game engine in the long run isn’t the way you want to go, but that, “Usually for the first pass to get an idea of what is technically possible, it’s a major cost saver.” Exploring within a lower fidelity game-based simulation also allows lessons learned to be transferred to higher fidelity adaptive systems or live training.
For more information on how the team built out their adaptive framework, collected measures, and ran participants through a rule set, contact aptima_info@aptima.com for a copy of the paper presented at I/ITSEC 2022.
“There’s a lot of decisions and there’s pros and cons to each that are covered within the paper,” Rebensky added.
UPDATE: At I/ITSEC 2023, Rebensky will present an extension of this work about how VR data can be leveraged within the training and simulation community: “Virtual Reality Provides Real Data: How Data in VR Transforms the Concept of Readiness.”
Aptima will also be showcasing its Adaptive Training solutions at I/ITSEC 2023 in Booth #1101. Learn more at https://www.aptima.com/iitsec/
To watch the webinar, visit: https://www.youtube.com/watch?v=RR8ZL-S68DA
