- This event has passed.
AI4SE & SE4AI Workshop 2023
Wednesday, September 27, 2023 - Thursday, September 28, 2023
Sponsored & organized by U.S. Army DEVCOM Armaments Center (AC) Systems Engineering Directorate and the Systems Engineering Research Center (SERC) in collaboration with the George Washington University
Balancing Opportunity and Risk: The Systems Engineer’s Role in the Rapid Advancement of AI-Based Systems
The 2023 AI4SE/SE4AI Workshop was held in-person on September 27-28 at the George Washington University in Washington, DC. Abstracts were sought from government, industry and academia for presentations and panels to explore the exciting advancements and challenges in the field of artificial intelligence (AI) and systems engineering (SE). All selected abstracts and presentations must be approved for public release.
The rapid development of AI presents both tremendous opportunities and inherent risks. As AI continues to revolutionize various industries, it becomes crucial to strike a delicate balance between seizing the potential benefits and addressing the potential risks. The conference theme, “Balancing Opportunity and Risk: The Systems Engineer’s Role in the Rapid Advancement of AI-Based Systems,” aims to foster discussions and insights on the responsible deployment of AI in systems engineering and how systems engineering can support the development of robust and ethical AI systems.
TOPICS OF INTEREST:
- Application of AI on SE projects
- Application of SE on AI projects
- V&V and T&E of Learning Based Systems
- Systems Approaches to Reliability and Safety of Learning Based Systems
- Workforce Development for AI4SE & SE4AI
- AI/ML Technology Evolution
- Augmented Intelligence, Cognitive Assistants and Decision Aiding Tools
- Automation & Human-Machine Teaming in Systems and Missions
- Digital Engineering Infrastructure and Methods in Support of AI
- Advances in Explainability, Interpretability, and Transparency
- Advances to Address Cognitive Bias in AI
- Testbeds for Learning-Based Systems
- SE and Data Collection/Curation for AI in Systems
- Standards of Trust and Evaluation of Trust in SE (Technical, Organizational, Societal)
- Impact of Large Language Models/Foundation Models on SE
The conference theme, “Balancing Opportunity and Risk: The Systems Engineer’s Role in the Rapid Advancement of AI-Based Systems,” explores the intricate relationship between artificial intelligence (AI) and systems engineering, encompasses five distinct tracks. The SE4AI track focuses on leveraging systems engineering principles and methodologies to develop robust and efficient AI systems. The AI4SE track delves into the application of AI in systems engineering processes, enabling enhanced decision-making, optimization, validation, and verification. The Human-AI Teaming track examines the collaboration between humans and AI, exploring how to maximize the synergistic potential while addressing ethical and social implications. The Trustworthy AI track highlights the critical aspects of safety, reliability, and ethical considerations in developing and deploying AI systems. Lastly, the Digital Engineering & SE Workforce Development track explores the evolving role of digital engineering and its impact on the systems engineering workforce, fostering skill development and adaptation in an AI-driven landscape. This comprehensive framework ensures a holistic exploration of the opportunities and risks associated with AI in systems engineering, paving the way for responsible and sustainable development.
KEYNOTE SPEAKERS
Ms. Jennifer Swanson is the Deputy Assistant Secretary of the Army for Data, Engineering & Software (DASA(DES)), Office of the Assistant Secretary of the Army for Acquisition, Logistics and Technology (ASA(ALT)). In this position, Ms. Swanson leads implementation of modern software practices including agile software development and DevSecOps, data centricity, and digital engineering across ASA(ALT). Since standing up DASA(DES), she established a Unified Data Reference Architecture (UDRA) which will enable digital transformation by giving Program Managers and industry partners guidance on how to produce, share and consume data. At the direction of the VCSA, she co-led a Capability Portfolio Review securing senior leader endorsement to drive agile development and DevSecOps throughout the Army. And under her leadership, the Request for Proposal process is changing to require industry to deliver using modern software practices.
Prior to joining ASA(ALT), Ms. Swanson served as the Director of the U.S. Army Communications-Electronics Command (CECOM) Software Engineering Center (SEC) headquartered in Aberdeen Proving Ground, MD. As Director, she oversaw a global mission, running an organization of approximately 3,000 military, civilian and industry employees, with an annual budget in excess of $700 million. She provided state of the art software engineering products and services throughout the Army.
Dr. Kimberly Sablon received her PhD in applied physics with a focus on nanophotonics from the University of Arkansas in 2009. She has published more than 60 peer-reviewed papers and contributed to ten scientific and technical books. She performed a set of critical reviews to identify technological challenges and research opportunities in the areas of reconfigurable multimodal sensing, communications, cognitive autonomous systems, AI-controlled networks and generative AI that could offer the greatest benefit to the Department of Defense while assessing the risks to national security.
In her current role as the Principal Director for Trusted Artificial Intelligence and Autonomy at the Office of the Assistance Secretary of Defense for Critical Technologies, Dr. Kimberly Sablon leads and coordinates scientific and technological efforts to ensure DOD superiority in future cognitive autonomous systems and hierarchical networks placing much emphasis on testing, evaluation, verification, and validation of dynamic AI systems. To accelerate development of AI-enabled systems and critical enablers, Dr. Sablon has set a strategic vision that is centered around AI systems engineering taking into consideration complexities of the real-world environment. Her strategic vision reflects the need for an advanced intellectual and research base in critical areas, which includes multimodal and interactive trusted perception, Warfighter-in-the-loop design, development and training in live, virtual and constructive environments, autonomous cognition and prediction, distributed, hetero-hierarchical AI architectures to enable edge intelligence, autonomous networks of autonomous systems, and continuous adversarial testing and red-teaming to enhance resiliency of AI systems against adversarial manipulation and deception along with development of approaches for recognizing machine-generated deception.
Mike “Rabbi” Harasimowicz, is the Director for Artificial Intelligence Innovations at Lockheed Martin’s AI Center leading Rapid Prototyping for Cognitive Application Development and Ethical Use of AI. Prior to his role at Lockheed, Mike served as an executive consultant for the Department of Defense’s Joint Artificial Intelligence Center in AI Product Development and as initial cadre of the JAIC’s Responsible AI Champions. Additionally, Mike was the Managing Director of Data Analytics and AI Development in the Intelligent Solutions Group at J.P. Morgan Chase. Mike retired from the U.S. Air Force in 2015 as Wing Commander of the 688th Cyberspace Wing after 25 years of innovating new cybersecurity, intelligence and warfighting tactics and technologies with specialized training as a civilian research engineer, Air Force Weapons Instructor and SAASS Strategy Graduate.
KEYNOTE
Ms. Jennifer Swanson, Deputy Assistant Secretary of the Army for Data, Engineering & Software, Office of the Assistant Secretary of the Army (Acquisition, Logistics and Technology)
KEYNOTE
Dr. Kimberly Sablon, Principal Director, Trusted Artificial Intelligence and Autonomy, Office of the Under Secretary of Defense for Research and Engineering
DEVCOM ARMAMENTS CENTER PERSPECTIVES
Dr. Myron Hohil, SSTM for Artificial Intelligence, Sensors & Collaborative Munitions, US Army DEVCOM Armaments Center
PANEL: CERTIFICATION OF LEARNING-BASED SYSTEMS
Moderator: Dr. Zoe Szajnfarber, The George Washington University
Dr. Jonathan Barkand, Defense Acquisition University
Dr. Tyler Cody, Virginia Tech National Security Institute
MAJ Christine Krueger, The George Washington University
Dr. Jason Summers, ARiA
DOWNLOAD PRESENTATION (KRUEGER)
PANEL: APPLICATION OF LLMs to SYSTEMS ENGINEERING
Moderator: Dr. Myron Hohil, US Army DEVCOM Armaments Center
Mr. Amir Abrari, SPEC Innovations
Mr. John Crissman, CNA Corporation
Dr. Carlo Lipizzi, Stevens Institute of Technology
DOWNLOAD PRESENTATION (ABRARI)
Establishing a Measurement Science for Joint Work with Models
Dr. Peter Denno, National Institute of Standards and Technology
Application of Systems Theoretic Process Analysis (STPA) to the Mission Assurance of AI-Enabled
Systems
Mr. Robert Crombie, Mr. Daniel Byrne, The Aerospace Corporation
Application of the Traditional Materiel Release Process to the DoD Ethical Principles of Artificial
Intelligence, and Roadmap
Mr. Benjamin Werner, Mr. Benjamin Schumeg, US Army DEVCOM Armaments Center
Using Artificial Intelligence to Accelerate Deployment of Learning-Based Systems
Dr. Jason Gregory, US Army DEVCOM Research Laboratory; Dr. Satyandra K. Gupta, University of Southern California
Conformal Prediction for Testing and Evaluation of Intelligent/ML Systems
Mr. Paul Horton, Georgia Tech Research Institute
Evaluating the Explainability and Interpretability of AI Systems
Ms. Alaysha Shearn, Dr. David Broniatowski, Dr. Armin Vosoughi, The George Washington University
Dr. Valerie Reyna, Cornell University
AI-Assisted Generation of SysML Models to Accelerate Agile Digital Engineering
Mr. Matthew Jones, Ms. Erin Crabb, Leidos
Requirement Discovery Using Embedded Knowledge Graph with ChatGPT
Dr. Nipa Phojanamongkolkij, Mr. Braxton VanGundy, Mr. Ramana Polavarapu, Mr. Joshua Bonner, NASA Langley Research Center; Dr. Barclay Brown, Collins Aerospace
LLM Based SysML Virtual Assistant
Mr. Alexander Demagall, Mr. Gabe Apanza, Dr. Daniel Selva, Texas A&M University
Melding Minds and Machines: Harnessing AI Capabilities to Further Model-Based Guided
Engineering
Ms. Melinda Ong, Mr. Andrew Gaskins, Booz Allen Hamilton
A Constructed System of Analysis to Enable Automation and Reasoning in Multi-Model Analysis
Mr. Daniel Dunbar, Dr. Mark Blackburn, Dr. Tom Hagedorn, Stevens Institute of Technology
Large Language Model Enabled Generation of Systems Engineering Artifacts
Dr. Paul Wach, Dr. Taylan Topcu, Mr. Mohammed Husain, Virginia Tech
KEYNOTE
Mr. Michael “Rabbi” Harasimowicz, Director, AI Innovations, Lockheed Martin
SERC PERSPECTIVES
Dr. Peter Beling, Director, Intelligent Systems Division, Virginia Tech National Security Institute and SERC Research Council Member; Dr. Ali Raz, Assistant Professor, George Mason University
PANEL: COMPARING ETHICAL AND SAFETY FRAMEWORKS FOR PROMOTING TRUST IN AI SYSTEMS
Moderator: Mr. Andy Lacher, NASA Langley Research Center
Dr. David Broniatowski, The George Washington University
Mr. Chuck Howell, MITRE
Dr. Kim Wasson, Joby Aviation
Is the Machine a Partner or a Tool? A Major Issue of Human-AI Teaming
Dr. Guy Boy, Paris Saclay University (CentraleSupélec) & ESTIA Institute of Technology
What is Human-in-the-Loop, Really?
Mr. Aditya Singh, Dr. Zoe Szajnfarber, The George Washington University
Managing the Continuity of Human Biases in AI SE Applications
Dr. Steven Conrad, Colorado State University; Ms. Heidi Perry, MIT Lincoln Laboratory
Opportunities and Risks of Incorporating LLMs in the Systems Engineering and Design Workflow: A Case Study of Robotic System Design Process
Dr. Zoe Szajnfarber*, Mr. Oladele (Richard) Adeyeye, Dr. Robert Pless, The George Washington University
The Need to Update Design and Certification Performance Standards for Operator Intervention of
Autonomous Systems
Dr. Lance Sherry, Dr. George Donohue, Ms. Jomana Bashata, George Mason University
Mission Engineering in Healthcare and What We Can Apply to the Military
Mr. Richard Threlkeld, Dr. Cihan Dagli, Missouri University of Science and Technology
Hiring Trained Animals: Generative AI Patterns and Practices for Systems Engineering
Dr. Barclay Brown, Collins Aerospace
Systems’ Perspectives for AI Acceleration in Large Enterprises and Government Agencies
Dr. Jyotirmay Gadewadikar, Mr. Jim Lockett, Mr. Jeremy Marshall, MITRE
Test and Evaluation (T&E) of Complex AI-Enabled Systems (AIES)
Dr. Flo Reeder*, Ms. Elena Charnetzki, MITRE
Systems Engineering Processes to Test AI Right (SEPTAR)
Ms. Carol Pomales, Dr. Flo Reeder, Dr. Carlos Balhana, Mr. Jim Lockett, Ms. Ivy Chen, MITRE
A Systems Engineering Perspective on Safety of AI-based Systems
Mr. Reginald Holmes, Dr. Hanumanthrao “Rao” Kannan, University of Alabama in Huntsville
Simulating the Tradespace with Synthetic Data
Dr. Cameron Turner, Clemson University; Dr. Matthew Castanier, Dr. David Gorsich, Dr. Greg Hartman,
US Army Ground Vehicle Systems Center