Technical Report
Sensemaking Research Roadmap
Report Number: SERC-2018-TR-117
Publication Date: 2018-11-01
Project:
Principal Investigators:
Co-Principal Investigators:
On 2-3 May 2018, the Systems Engineering Research Center (SERC) convened a workshop to examine current research trends, challenges, and open science questions in artificial intelligence (AI)-enabled sensemaking technologies. The Office of the Director for National Intelligence (ODNI), with support from the Office of the Deputy Assistant Secretary of Defense for Systems Engineering (DASD(SE)) funded this effort. Nearly 50 experts from both academia and the intelligence community (IC) gathered together to contribute. The finding of these experts formed the starting point of this research roadmap.
The consensus from the workshop was that the two most pertinent research thrusts that are needed for developing a higher form of sensemaking are (1) multi-modal analysis for sensemaking and (2) hybrid systems for sensemaking. Within these thrust areas exist several research tracks worthy of pursuit. We discuss these in Sections 3 and 4 of this report, respectively.
Currently, four elements influence multi-modal sensemaking systems: (1) the volume of information generated by both conventional and social media sensors; (2) the intentional corruption of information in these modalities; (3) the different speeds at which information propagates over the internet, (4) the often contradictory information about the same topic from different sources. Although there has been significant research done to mitigate these elements, these tend to be more on intra-modal or cross-modal modalities, using a few subsets of data. Significant research is required to progress from these analytics to true multi-modal sensemaking systems.
Therefore, we propose the following research tracks: interoperability issues, reliability, and trustworthiness, next-generation AI and sensemaking algorithms, and assessing and closing the loop for multi-modal sensemaking systems. Some of this research has already begun, some will take a decade or more to develop. In Section 5, we provide a projected timeline for the research. We believe these tracks are crucial to developing the science of holistic sensemaking.
Hybrid sensemaking faces its own set of challenges. IC-related issues often rely on human analysts, creating a bottleneck that cannot be solved with additional computing power. Hence, the goal of HS2 is to explore the possibilities for human-machine hybrid systems. We seek to eventually develop a networked ecosystem of human analysts and machines. The HS2 research tracks are: HS2 taxonomy & performance measures, interactive & continuous sensemaking, HS2 autonomy & trust, HS2 as networks: organizational sciences perspective, and HS2 interfaces. Although we have made some strides along the HS2 research tracks (e.g., developing a taxonomy, domain-specific HS2, and HS2 modeling) much of the hybrid sensemaking work is still to come. As you will see in Section 5 of this report, we project much of the HS2 research to be 5-10 years out or more.