Engaging on AI Advancements and Collaborative Systems Engineering
Experts in systems engineering (SE) and artificial intelligence (AI) gathered for the third annual AI4SE & SE4AI Workshop on September 21-22 at Stevens Institute of Technology in Hoboken, New Jersey.
The SERC co-hosted the workshop with the U.S. Army DEVCOM Armaments Center (DEVCOM AC) Systems Engineering Directorate. Associate Director Al Stanbury helped organize the panels and presentations. “This event has been one where we can focus on the two main themes of AI for SE and SE for AI products,” Stanbury said. “There are lots of events, but we need more of this type where we can calibrate between government, academia and industry to share knowledge, things we are doing, and things we need to be doing to generate greater research interest.”
Retired Army Colonel R. Chris DeLuca gave the keynote address as the Director of Specialty Engineering with the Office of the Under Secretary of Defense for Research and Engineering. He stressed the need for the Department of Defense (DoD) to integrate data and AI capabilities.
“Data sets for AI will quickly become the DoD’s most valuable asset,” DeLuca said. “We must get data right before moving on with AI in the department. Data must be visible, accessible, understandable, linked, trustworthy, interoperable, and secure. That’s a systems engineering challenge.”
“The body of content has been growing from year to year,” said SERC CTO Tom McDermott. “We’re seeing a diverse set of people and the integration of computer science, engineering, data analytics and other fields.”
Dr. Eric Kelly of data firm Quansight attended in person after logging in virtually for the 2021 workshop. “I really wanted to hear the perspective from the government about what we need in terms of AI,” Kelly said. “I heard great information about issues like safety concerns and being able to evaluate how safe AI models are.”
The workshop featured two discussion panels. In the first, Dr. Jason Cook Senior Scientific Technology Manager, Systems Engineering Research at DEVCOM AC moderated the panel on Workforce Development with panelists Dr. Rosa Heckle of MITRE, Dr. Zoe Szajnfarber of George Washington University, Jaime Hernandez-Cordero of the DoD, and Rhonda Maus of Defense Acquisition University (DAU). The panel spoke about immersive workforce training, certification programs, and the need to guide decision makers who lack technical understanding.
On the second panel, Dr. Ralph Tillinghast, Senior Scientific Technical Manager: Future Intelligence, Army Futures Command, moderated a discussion on Government Visions and Needs with Carol Pomales of MITRE, Dr. Raj Iyengar of the U.S. Nuclear Regulatory Commission, and R. Chris DeLuca of OUSD(R&E). The panel discussed advanced modeling, digital twins that represent physical spaces, and the trustworthiness of AI.
“You can poll user trust in AI with standardized metrics,” Pomales said. “Trustworthiness of AI is more complex. It goes back to how you built that AI. Did you lock down your infrastructure to prevent data poisoning? Did you anticipate less likely conditions where the AI needs to perform? The goal is to establish a pedigree on how that AI was built, to establish a trustworthiness that reflects the quality of the AI process that was followed.”
Both days featured parallel tracks of presentations. The first day included AI4SE and SE4AI, and the second day included AI4SE & DEV and SE4AI & V&V (verification and validation).
During the AI4SE track on the first day, Kelly of Quansight presented on the company’s collaboration with DEVCOM AC. The project incorporated AI elements into the OneSAF (One Semi-automated Forces) combat simulator to establish reinforcement learning, which Kelly said creates a more realistic experience for users.
“Reinforcement learning is an exploratory algorithm that is trying to find every possible solution when you give it a goal,” Kelly said. “A lot of times, when we’re talking about AI and the problems we’re experiencing, that starts with having to get the data and figuring out what to do with it. With reinforcement learning, you’re generating data on the fly.”
In closing remarks, McDermott looked ahead by explaining how the AI4SE & SE4AI Workshop grew out of discussions during an INCOSE (International Council on Systems Engineering) conference in 2019. “In the SERC, we created a roadmap that looked at how AI and systems engineering might co-evolve,” McDermott said. “Like everything else we’re seeing when we look at the digitalization of engineering, they seem to be evolving tremendously fast and tremendously slow at the same time. There will be fairly large cultural and technical shifts for us all, driven around data.”
The slides for each presentation are available on the SERC website. A research report is forthcoming.
Follow the Systems Engineering Research Center (SERC) on LinkedIn for updates on research, events and other news.