Applying Systems Engineering Today to Shape Tomorrow
Building on past research, Dr. Bryan Mesmer believes insights he is developing on architecting can serve stakeholders across future generations.
Mesmer, a faculty member at The University of Alabama in Huntsville, recently presented an updated perspective of a SERC project he is conducting to improve the architecture process in the U.S. Army. Progressing through that project motivated Mesmer to contemplate a systems engineering approach to architecting.
“The real issue here is the need for a justification of our decision-making,” Mesmer said at the 2022 Annual Research Review. “We know in architecting we have to justify our decisions to the current stakeholders, but also to the stakeholders 40, 50, 60 years down the road. Right now, there’s no good way in which we rigorously make those decisions to provide that justification.”
Mesmer outlined five steps for developing a decision-making process for architectural decisions. The first is to determine the decision framework. The next three derive from that framework, involving understanding inputs of preferences, beliefs, and alternatives. The fifth step involves understanding the distinction between designing and architecting.
Mesmer said that decision-making in architecting is often based on heuristics, which he finds lacking. He and his research colleagues studied sources of knowledge in the architecting community that propose using heuristics. They found citations for those heuristics with little evidence or that relied on case studies unrelated to architecting.
“Any architecting handbook that you see, any textbook or article, is typically focused on the heuristics to good decision-making in architecting,” Mesmer said. “Heuristics are fine as long as there’s clear supportive evidence, and we did not see that in the sources we examined.”
For architecting decisions, Mesmer advocates applying normative techniques that are mathematically rigorous to devise justifiable and repeatable decisions.
“Normative decision theory is a mathematical way people should make decisions,” Mesmer said. “There are three main inputs to that theory: understanding the alternatives, understanding the [stakeholder] beliefs on the outcomes of the alternatives, and understanding the [stakeholder] preferences on the outcomes of the alternatives.”
While explaining the difference between designing and architecting, Mesmer noted an absence of measurements in the architecting process.
“In design engineering, we can measure things.” Mesmer said. “We don’t have thrust, speed, or weight for an architecture. We need to be able to assess those things we can actually look at [such as expected cost or expected performance]. One of the reasons why we can’t do that right now is we don’t have the tools to analyze the architecture during the decision-making process.”
Another difference might involve standards, policies, or other circumstances within an architecture that impact how stakeholders make decisions, such as long time horizons with high uncertainties.
“There are many stakeholders along the time horizon, not only who are using and implementing the architecture, but who could kill the architecture,” Mesmer said. “They could say we’re done with this one, let’s restart, let’s do something new. While we can see that in design engineering because the timeframe is a lot tighter, it’s not as often as we would see it in architecting.”
Mesmer proposed several steps for further research, including surveying the architecting and stakeholder communities to understand preferences, determining sources of uncertainty in outcomes and ways to measure them, and formulating a methodology for generating new decision alternatives.
“The normative decision approach we’re proposing to use in architecting is really unchanged from design engineering,” Mesmer said. “There’s nothing new in the actual process or in the actual framework. The challenge is solely in forming and representing the three inputs of alternatives, preferences, and beliefs, and that’s where I believe the focus of architecting really needs to be if we want to be able to justify our decision-making to future stakeholders.”
Learn more about Dr. Mesmer’s discussion by downloading his presentation slides. Follow SERC on LinkedIn for updates on systems engineering research.