Workshop Report
Research Workshop: AI4SE & SE4AI Report
Start Date: 2020-10-28End Date: 2020-10-29
Event: Research Workshop: AI4SE & SE4AI
Event: Virtual
Lead Authors:
Dr. Dinesh Verma
Co-Authors:
Ms. Kara Pepe
Dr. Peter Beling
Mr. Thomas McDermott
Objective: Systems Engineering (SE) is undergoing a digital transformation that will lead to further transformational advances in the use of Artificial Intelligence (AI) and Machine Learning (ML) technology to automate many routine engineering tasks. At the same time, applying AI, ML, and autonomy to complex and critical systems encourages new systems engineering methods, processes, and tools.
To address this current and evolving reality, The US Army Combat Capabilities Development Command Armaments Center (CCDC AC) Systems Engineering Directorate (SED) and the Systems Engineering Research Center (SERC), a University Affiliated Research Center (UARC) for the Department of Defense (DoD), jointly sponsored the inaugural Artificial Intelligence for Systems Engineering/Systems Engineering for Artificial Intelligence (AI4SE/SE4AI) workshop. This two-day virtual event gathered members of the Government, Academic and Industry communities to learn from leaders already using AI in this space and share ideas focused on the workshop’s main objective: how to define relevant SE and AI challenges, areas of exploration and methodologies to use, and ways in which to collaborate and research in the upcoming years.
The Workshops: The total 15 presentations focused on relevant topics within the areas of Machine Learning/Artificial Intelligence (ML/AI), Artificial Intelligence for Systems Engineering (AI4SE), Systems Engineering for Artificial Intelligence (SE4AI), and Digital Engineering (DE). A unifying theme was the increasing need for organizations and systems to be agile to keep up with the dynamic nature of AI in order to achieve the ultimate goal of delivering the most relevant and effective tools to the soldier in the field. Essential areas for focus emerged, notably: the importance of data—its acquisition, analysis and maintenance; the need to update business processes, including workforce development and retention; and the importance of collaboration.
Outcomes: Current systems engineering practices do not support the long-term outcome of “Human-Machine Co-Learning”, which implies an upcoming evolutionary phase in the Systems Engineering community consisting of three “waves”: the first includes technologies and approaches that increase the transparency of decisions produced by AI systems; the second will produce systems that learn and are robust and predictable in the type of key applications normal to SE; and the third wave involves systems that adapt and learn dynamically from their environments, which will develop trust in machine-to-machine and human-to-machine (and maybe machine-to-human) interactions. A notional roadmap of this evolution spans five categories:
- AI/ML Technology Evolution: The technological implementation of AI systems need to evolve in directions relevant to SE.
- Automation & Human-Machine Teaming: The purpose of AI in systems is generally automation of human tasks and decisions.
- Augmented Engineering: AI technologies will increasingly be used to augment the work of engineering.
- Digital Engineering: The current digital engineering transformation will enable augmented engineering.
- Workforce and Culture: Significant transformation is needed in the SE workforce, with greater integration of software and human behavioral sciences at the forefront.
Future Work: The SERC and CCDC AC intend to organize a follow-on workshop in 2021 to explore specific outcomes from this initial 2020 meeting. The intended goal will be to curate panel discussions and facilitate group breakout sessions that produce actionable applications of AI4SE and SE4AI as well as relevant research ideas.
Feedback gathered from participants of this inaugural 2020 event (in response to the question "Based on your experience in this workshop, what do you think are the top 2-3 ideas or initiatives worthy of further discussion?") will guide future exploration, including:
- Establish an AI test bed that contains data sets that can be created and manipulated to affect outcome.
- Explore a path forward for AI and XAI for SE and mission engineering.
- Integrate digital twins in the operational phase.
- Embed ethics considerations of AI into the engineering process.
- Test and Evaluation (T&E) with AI/ML components.
- Consider collaborative functional design.
- Extend the current SE toolset to AI/ML components.
- Develop resilience concepts, modeling constructs, and vulnerability assessment frameworks for AI-heavy systems in the presence of adversarial threats [adversarial AI].
- Develop theoretical foundations with verifiable robustness.
- Consider new generations of attack and defense methods for comparison.
- Consider Workforce Development/Human Capital Development efforts, particularly in light of workforce shortages faced by both SE and AI and the competition for talent between technology companies and government agencies.