Summary Report: WRT-1001 Digital Engineering Metrics
Enterprises and System of Systems
Systems Engineering and Systems Management Transformation
Report Number: SERC-2020-SR-003
Publication Date: 2020-06-08
Project: Digital Engineering Measures
Thomas McDermott Jr.
Dr. Eileen Van Aken
The DoD Digital Engineering (DE) strategy1 outlines five strategic goals for transformation, targeted to “promote the use of digital representations of systems and components and the use of digital artifacts as a technical means of communication across a diverse set of stakeholders, address a range of disciplines involved in the acquisition and procurement of national defense systems, and encourage innovation in the way we build, test, field, and sustain our national defense systems and how we train and shape the workforce to use these practices.”
DE is defined as ‘‘an integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support lifecycle activities from concept through disposal. A DE ecosystem is an interconnected infrastructure, environment, and methodology that enables the exchange of digital artifacts from an authoritative source of truth.”2 Model-Based Systems Engineering (MBSE) is a subset of DE, defined as “the formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases.”3 MBSE has been a popular topic in the SE community for over a decade, but the level of movement toward broad implementation has not always been clear. With the release of the DoD DE Strategy, a clear set of high-level goals are defined for the DoD acquisition community and its industry base. The terms DE and MBSE are used interchangeably throughout this report.
Digital transformation is a change process heavily rooted in “how we train and shape the workforce to use those processes”, as noted by Goal 5 of the DoD Digital Engineering Strategy. Each of the DoD’s goals implies that an enterprise, organizational unit, or multi-organizational program has a means to define the outcomes of a DE strategy, performance metrics, measurement approaches, and leading indicators of change in the transformation process. This research sought to define a comprehensive framework for DE benefits and expected value linked to the ongoing development of DE enterprise capabilities and experienced transformation “pain points,” enablers, obstacles, and change strategies.
A key result of this research is the development and definition of two frameworks that categorize DE benefits and adoption strategies that can be universally applied to a formal enterprise change strategy and associated performance measurement activities. The first framework is linked to the benefits of DE and categorizes 48 benefit areas linked to four digital transformation outcome areas: quality, velocity/agility, user experience, and knowledge transfer. This framework identifies a number of candidate success metrics. A test application to an ongoing DoD pilot project was completed and is documented in this report. The second framework addresses enterprise adoption of DE and provides a categorization of 37 success factors linked to organizational management subsystems encompassing leadership, communication, strategy and vision, resources, workforce, change strategy and processes, customers, measurement and data, workforce, organization DE processes relate to DE, and the organizational and external environments.