Testing AI Social Intelligence
Aptima Awarded DARPA Contract to Test the Social Intelligence of AI
Program to evolve AI from tool to teammate for future of expert human/AI teams
Arlington, VA – January 13, 2020 – Artificial intelligence (AI) can plan a fast route to victims in a collapsing building. It cannot understand why the rescue team diverts from that route, or predict the team’s actions. So, it cannot coordinate or communicate well with the team. Current AI is a good tool; it’s a poor teammate because it does not understand the most important part of its environment—its human team.
To address this challenge, the Defense Advanced Research Projects Agency (DARPA) is funding research to imbue AI with social intelligence, a basic building capability necessary to create expert human/AI teams in the future. To support this initiative, Aptima announced today a contract award from DARPA to assess the social intelligence of AI. Aptima will serve as the test and evaluation lead for the DARPA program called ASIST: “Artificial Social Intelligence for Successful Teams.”
Through ASIST, DARPA envisions computer-based agents that, akin to humans, will observe the human team and its environment, infer teammates’ goals and plans, and, as conditions change, advise the team in ways that fit the human perspective. Aptima will apply its expertise in how humans and AI systems think, learn, and perform to lead the evaluation of this new form of AI, exploring human-machine teaming and machine social intelligence. Aptima’s partners in this effort are the Center for Human, Artificial Intelligence, and Robot Teaming at Arizona State University, and the Center for Open Science. The team’s work entails coordination of research and engineering performers, development of a distributed testbed for AI-human collaboration, and design and execution of experiments that test the ability of ASIST AI to support human teammates in increasingly complex environments.
Dr. Jared Freeman, Aptima’s Chief Scientist and Principal Investigator for the ASIST T&E team.
“Teams that incorporate AI and humans effectively will have an advantage on the battlefield and business,” said Dr. Jared Freeman, Aptima’s Chief Scientist and Principal Investigator for the ASIST T&E team. “Today’s AI do not partner well. AI does not share a mental model of the mission with human team members, nor does it have a theory of the human mind with which to predict human action or understand it. As a result, current AI can give humans advice that is spectacularly wrong or irrelevant. Our team will help DARPA create and assess socially competent AI by applying our expertise in AI, human behavior, experimentation, and measurement.”
The AI Challenge and what makes humans good teammates
DARPA laid out the challenge clearly in its solicitation to academia and industry1: “Humans intuitively combine pre-existing knowledge with observations and contextual clues to construct rich mental models of the world around them and use these models to evaluate goals, perform thought experiments, make predictions, and update their situational understanding. When the environment contains other people, humans use a skill called theory of mind (ToM) to infer their mental states from observed actions and context, and predict future actions from those inferred states. When humans form teams, these models can become extremely complex. High-performing teams naturally align key aspects of their models to create shared mental models of their environment, equipment, team, and strategies. ToM and the ability to create shared mental models are key elements of human social intelligence. Together, these two skills form the basis for human collaboration at all scales, whether the setting is a playing field or a military mission.”
ASIST’s goal is to create agents with social skills that can demonstrate in concrete scenarios the ability to comprehend team processes, trust, and other aspects of social cognition. Social and computer scientists on ASIST will develop and test theory and AI, respectively. Aptima, ASU, and COS will develop the Social Intelligence Manipulation & Measurement Laboratory (SIM2 Lab) as the virtual testbed on which it will perform these evaluations. Minecraft, a customizable open-world 3-D sandbox game environment, will be used to run the scenarios.
For example, in a possible search and rescue scenario, an inept AI tool lacking social intelligence might overwhelm First Responders, pushing too much information at them. In contrast, an AI with social intelligence and a shared mental model would understand the team’s mission and, with a theory of mind, predict their decisions and actions. By monitoring the team’s communications for information they don’t have, it could then selectively deliver guidance about the state of the building, the victims, and the aid that’s needed, much as human teammate might.
“Aptima has a long history of measuring, modeling, and improving team performance. We build innovative technologies to measure teamwork and guide it,” added Freeman. “We are excited to have the opportunity to advance a new form of AI that can anticipate and adapt to the information needs of its human teammates.”
1 DARPA Broad Agency Announcement Artificial Social Intelligence for Successful Teams (ASIST), March 2019
The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.
About Aptima:
Since 1995, Aptima has specialized in optimizing human performance, applying expertise in how humans think, learn, and perform to solve training and readiness challenges throughout the Department of Defense. With 850+ contracts, our unique ability to identify, build, and field solutions rooted in 1) groundbreaking research at AFRL, ARI, ONR, DARPA, and other Government labs, 2) engineering human-centered training products, and 3) transitioning and fielding military-grade solutions to the most challenging problems. Visit www.Aptima.com
Aptima Media Contact:
Joel Greenberg
DCPR
Joel@dcpr.com
202-363-1065 | 202-669-3639 C