Accelerating and Enhancing ISD Analyses for Faster Speed to Fleet

1st Place Winner of the 2023 Navy Ready, Relevant Learning Challenge

Training development is a meticulous, months-long process that requires Instructional System Designers and Developers (ISDs) to produce detailed training analyses before new content can be brought to life. These analyses are currently labor-intensive and time-consuming, particularly for large-scale military training programs that need to meet rigorous standards.

When new threats, technologies, or operational changes emerge, the military cannot rapidly update existing or create new training curricula due to the lengthy ISD process. For the U.S. military, this causes training programs to frequently lag behind operational requirements.

To address that challenge, and a call to industry for solutions, Aptima has harnessed the language proficiency of large language models (LLMs) such as OpenAI’s ChatGPT to develop NAUTICAL™.

A web-based tool designed to mirror the ISD’s workflow, NAUTICAL performs alongside ISDs as an ‘expert resource,’ automating many of the training analyses and outputs, while preserving oversight for transparency, explainability, and trust.

Generative AI-based Platform for Instructional System Design Analysis and Training Insights

Saves Time and Reduces Costs

NAUTICAL enhances current ISD workflows through intuitive interfaces and templates. This enables ISDs to focus on strategies for answering questions and deriving results, while automating many of their repeatable tasks. Once the system is set up, prompt templates and product items are created, and the data sources identified, ISDs can batch process and review results quickly and efficiently.

Dovetails with Established ISD Workflows

NAUTICAL is designed to dovetail with the established workflows of ISDs, to enhance adoption rates while minimizing the learning curve. By anticipating the needs of ISDs, Aptima has crafted a suite of interfaces that not only align with current user practices but also elevates the entire workflow experience. The result is a user-centric platform that simplifies the transition to advanced AI support, ensuring that the focus remains on the ISD’a critical analytical work, not on navigating the toolset.

Explains Results for Accuracy and Trust

NAUTICAL’s prompting strategy utilizes requests to the LLM to provide both an answer and a rationale for that answer. This strategy increases the accuracy and interpretability of the LLM responses: asking the LLM to explain its reasoning when providing an answer prods the LLM to more precisely consider how its answers are generated. This process provides the ISD a clear explanation of the answer and increases their confidence in results, especially in cases where they may disagree with the answer, or the answers provided are not immediately intuitive.

Extensible to Future Models and Other Applications

NAUTICAL is designed to accommodate the integration of multiple LLMs and other applications with minimal effort. This flexibility means that as new and more advanced LLMs emerge, NAUTICAL can incorporate these tools seamlessly, without the need for a system overhaul or re-engineering of the underlying methodology. Such versatility ensures that NAUTICAL remains at the cutting edge, ready to harness future advancements in AI and machine learning to continually enhance the ISD process.

Can Generative AI and LLMs speed up training development? YES!

NAUTICAL provides an augmented workflow for ISDs that utilizes AI technology as a collaborative tool for efficiently developing training solutions.

• Better Quality Results: NAUTICAL’s tailored prompt engineering methodology is at the forefront of delivering superior quality results. By focusing on structured data fields rather than relying solely on direct language input, NAUTICAL provides a more granular level of control and precision in prompt creation. This data-driven strategy allows for the iterative refinement of prompts, ensuring that each iteration is informed by the last, thereby progressively enhancing the accuracy and relevance of the output.
• Enhanced Support of ISDs: For the 2023 Navy Ready, Relevant Learning Challenge, Aptima enhanced existing Navy Excel workbooks used by ISDs by integrating them with the rationale provided by the LLM responses. This fusion allows for ISDs to work with known product formats (such as the Excel Training Analysis Model) but still reap the benefits of interacting with the LLMs responses and reasoning. ISDs can also export products into intermediary Excel files that contain both the answers to each product item as well as the rationale provided by the LLM for those answers.
• Improved Human-AI Teaming: Extensive user-centered testing with ISD shaped NAUTICAL’s ability to complement existing skills while expanding analytical potential. NAUTICAL demonstrates ‘the art of possible’ when thoughtfully leveraging AI to augment human expertise and unlock operational advances.

For more information, please contact nautical@aptima.com.