Co-Principal Investigator: Mr. Tom McDermott, SERC Deputy Director and CTO, Stevens Institute of Technology
Co-Principal Investigator: Dr. Alejandro Salado, Associate Professor of Systems and Industrial Engineering, University of Arizona
The systems engineering community see the potential for significant benefit from digital engineering (DE) and digital transformation. The DoD Digital Engineering Strategy[1] and DoD Digital Modernization Strategy[2] are two examples of the approaches the Department is taking to address these issues. Despite the promised value of DE, specific benefits of digital and model-based systems engineering (MBSE) have not yet been translated to organizational value drivers and success metrics.
Organizations are searching for guidance on measuring the value and benefits of DE and MBSE. For digital transformation to be successful, the systems engineering community needs a set of common guidance on what data to collect and what top-level measures can be used consistently across a variety of programs. SERC Researchers Tom McDermott and Alejandro Salado and PhD candidate Kaitlin Henderson are leading research focused on addressing that need.
Previous SERC research on success metrics developed two frameworks that categorize key DE and MBSE benefits and identify critical enablers and obstacles, as illustrated below.[3]
These two frameworks highlight the most important enterprise performance metrics for digital engineering transformation. The research team also found that the community perceives these as beneficial but has not yet standardized nor developed best practices linking digital engineering practices to quantitative performance measures.
Mr. McDermott explained, “We understood to build a rigorous measurement model we had to first build a causal model that identified the flow of relationships through those benefits. The project joined up with the Digital Engineering Measurement Framework development effort sponsored by the Aerospace Industries Association (AIA), the International Council on Systems Engineering (INCOSE), the National Defense Industrial Association (NDIA), and the DoD Practical Software and System Measures (PSM) initiative.” This group of subject matter experts generated an initial metrics table using a metrics specification process developed by PSM.[4] Mr. McDermott continued, “What we were able to do with the causal model was work with subject matter expert inputs and understand the causal relationships between their view of what you might measure and the benefits we identified to come to an agreed-upon initial list of metrics.” The integration of the project into the broader community is a testament to the value of the work.
The initial Digital Engineering Measurement Framework will be published at the end of 2021. To ensure a successful transition, the research team is looking for organizations or projects that would be interested in piloting the metrics. For further information, contact Mr. Tom McDermott.
[1] Available at: https://ac.cto.mil/wp-content/uploads/2019/06/2018-Digital-Engineering-Strategy_Approved_PrintVersion.pdf
[2] Available at: https://media.defense.gov/2019/Jul/12/2002156622/-1/-1/1/DOD-DIGITAL-MODERNIZATION-STRATEGY-2019.PDF
[3] Available at: https://sercuarc.org/results-of-the-serc-incose-ndia-mbse-maturity-survey-are-in/
[4] Available at: www.psmsc.com