Kairos Research recently embarked on an ambitious R&D effort as part of the National Geospatial-Intelligence Agency’s (NGA) GEOGCOG program. According to NGA’s website, “the objective of GEOCOG is to leverage advances in behavioral and cognitive neuroscience to develop low-cost, unobtrusive methods to measure, estimate and model analysts’ behaviors and cognitive states during human–machine teaming.” Kairos is part of a multi-institution research team led by HRL Laboratories. The team’s project, called SENSE (Sensemaking Effectiveness using Neurocognitive Signatures of Efficiency), seeks to develop an infrastructure for testing the impact of new software tools on human-in-the-loop geospatial intelligence (GEOINT) analysis.
The GEOCOG program defines GEOINT as “the exploitation and analysis of imagery and geospatial information to describe, assess, and visually depict physical features and geographically referenced activities on the Earth.” Furthermore, “GEOINT includes tasks involving collection, sense-making, and analysis of data and imagery to survey physical features, assess human activity, and make spatial decisions.” In recent years, government organizations such as NGA have begun to incorporate AI-enabled software tools into GEOINT workflows with the aim of improving both the speed and accuracy of intelligence analysis. However, human analysts remain a critical component of the GEOINT analytic pipeline. Thus, to have maximum benefit, any new tools must interact optimally with the human element of the GEOINT enterprise. A key insight of NGA’s GEOCOG program is that effective integration must be measured at multiple levels of analysis — ranging from the cognitive level (how does a new tool affect analysts’ attention and mental workload?), to the behavioral level (how does the tool impact analysts’ task execution strategy and their interactions with the data?), to the performance level (how does the overall human-in-the-loop system perform with the addition of the new tool?). And while it is possible to draw some inferences about the effectiveness of new software tools based on performance testing alone, such tests rarely reveal cognitive and behavioral impacts. Assessing such impacts may be critically important in predicting whether a new tool will ultimately be embraced by analysts, and whether it will improve analytic efficiency in the long run.
To help achieve the goals of GEOCOG, Kairos is developing an interactive dashboard that enables tool evaluators to visualize the cognitive, behavioral, and performance-level impacts of a software tool in real-time as GEOINT analysts interact with it. According to Kairos Senior Software Engineer, Srikanth Nadella, the dashboard simultaneously displays rolling information relating to eye movements, mouse clicks, key strokes, and other instrumented behaviors and physiological signatures, as well as real-time estimates of an analyst’s cognitive state (e.g., whether the analyst is attentive, distracted, etc.). Mr. Nadella added, “Our dashboard includes a ‘forensic mode,’ which will allow evaluators to view, aggregate, and explore the data after data collection is complete, further facilitating discovery and insight.” In addition to dashboard development, Kairos is leading the development of SENSE’s cognitive state assessment models. These models infer an analyst’s cognitive state through the mathematical integration of diverse physiological (including neurophysiological) and behavioral data.
This material is based upon work supported by the National Geospatial-Intelligence Agency under Contract No. HM0476-22-C-0065. Any opinions, findings and conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Geospatial-Intelligence Agency.
