Publication

MOTIVATION AND CONTEXT

One of the key elements of the SERC’s research strategy is transforming the practice of systems engineering – “SE Transformation.” The Grand Challenge goal for SE Transformation is to transform the DoD community’s current systems engineering and management methods, processes, and tools (MPTs) and practices away from sequential, single stovepipe system, hardware-first, outside-in, document-driven, point-solution, acquisition-oriented approaches; and toward concurrent, portfolio and enterprise-oriented, hardware-software-human engineered, balanced outside-in and inside-out, model-driven, set-based, full life cycle approaches. 

These will enable much more rapid, concurrent, flexible, scalable definition and analysis of the increasingly complex, dynamic, multi-stakeholder, cyber-physical-human DoD systems of the future. Four elements of the research strategy for SE Transformation are the following:

1. Make Smart Trades Quickly: Develop MPTs to enable stakeholders to be able to understand and visualize the tradespace and make smart decisions quickly that take into account how the many characteristics and functions of systems impact each other
2. Rapidly Conceive of Systems: Develop MPTs that allow multi-discipline stakeholders to quickly develop alternative system concepts and evaluate them for their effectiveness and practicality
3. Balance Agility, Assurance, and Affordability: Develop SE MPTs that work with high assurance in the face of high uncertainty and rapid change in mission, requirements, technology, and other factors to allow systems to be rapidly and cost-effectively acquired and responsive to both anticipated and unanticipated changes in the field
4. Align with Engineered Resilient Systems (ERS): Align research to leverage DoD’s ERS strategic research initiative and contribute to it; e.g., ERS efforts to define new approaches to tradespace analysis.

For strategy 3, “Systems” covers the full range of DoD systems of interest from components such as sensors and effectors to full systems that are part of net-centric systems of systems and enterprises. “Effectiveness” covers the full range of needed system quality attributes or ilities, such as reliability, availability, maintainability, safety, security, performance, usability, scalability, interoperability, speed, versatility, flexibility, and adaptability, along with composite attributes such as resilience, sustainability, and suitability or mission effectiveness. “Cost” covers the full range of needed resources, including present and future dollars, calendar time, critical skills, and critical material resources.

RT-46, Tradespace and Affordability, is a major SERC initiative within SE Transformation. It particularly focuses on the tradespace among a system’s ilities, or non-functional requirements. Its project name is ilities Tradespace and Affordability Project (iTAP). The ilities differ from functional requirements in that they are systemwide properties that specify how well the system should perform, as compared to functions that specify what the system should perform. Adding a functional requirement to a system’s specification tends to have an incremental, additive effect on the system’s cost and schedule. Adding an ility requirement to a system’s specification tends to have a systemwide, multiplicative effect on the system’s cost and schedule. Also, ilities are harder to specify and evaluate, as their values vary with variations in the system’s environment and operational scenarios.

Further, the satisfaction of their specifications is much harder to verify than placing an X in a functional traceability matrix, as the verification requires considerable effort in analysis across a range of environments and operational scenarios. As a result, it is not surprising that problems in satisfying ility requirements are the source of many DoD acquisition program cost and schedule overruns. Also, with some exceptions such as pure physical systems and pure software systems, there is little technology in the form of scalable methods, processes, and tools (MPTs) for evaluating the satisfaction of multiple-ility requirements and their associated tradespaces for complex cyber-physical-human systems.

The increasingly critical DoD need for such capabilities has been identified in several recent studies and initiatives such as the National Research Council’s “Critical Code” Report (NRC, 2010), the SERC “Systems 2020” Report (SERC, 2010), the “Manual for the Operation of the Joint Capabilities Integration and Development System” (JROC, 2011), and the DoD “Engineered Resilient Systems (ERS) Roadmap” (Holland, 2012). The particular need for Affordability has been emphasized in several USD(AT&L) and DepSecDef “Better Buying Power” memoranda (Carter et al., 2010-2013) and research-need studies such as the AFRL “Technology Horizons” report (Dahm, 2010).

The major objectives of the initial 5-month Phase 1 activity were to lay strong foundations for ITAP Phase 2, including knowledge of Department of Defense (DoD) ility priorities; foundations and frameworks for ITAP analysis; extension and tailoring of existing ITAP methods, processes, and tools (MPTs); and exploration of candidate Phase 2 pilot organizations for ITAP MPTs.

Four activities were pursued in achieving these objectives:

1.  Ility Definitions and Relationships. Phase 1 included a discovery activity to identify and analyze DoD and other ility definitions and relationships, and to propose a draft set of DoD-oriented working definitions and relationships for the project. 
2. iTAP Foundations and Frameworks. This effort helped to build iTAP foundations by elaborating key frameworks (process-based, architecture-based, means-ends based, value-based), anticipating further subsequent elaboration via community efforts.
3. Ility-Oriented tool demos and extension plans. This effort created initial demonstration capabilities from strong existing ITA analysis toolsets and explored piloting by user organizations in the DoD Services.
4. Program management and community building. This effort included coordinating efforts with complementary initiatives in the DoD ERS, and counterpart working groups in the International Council for Systems Engineering (INCOSE), the Military Operations Research Society (MORS), and the National Defense industry Association (NDIA).

The Phase 1 results for activities 1 and 2 included initial top-level sets of views relevant to ilities tradespace and affordability analysis that provided an initial common framework for reasoning about ilities, similar in intent to the various views provided by SysML for product architectures and DoDAF for operational and architectural views. The views included definitions, stakeholder value-based and change-oriented views, views of ility synergies and conflicts resulting from ility achievement strategies, and a representation scheme and support system for view construction and analysis.

Phase 1 also determined that strong tradespace capabilities were being developed for the tradespace analysis of physical systems. However, based on sources such as the JCIDS survey of combat commanders’ tradespace needs, it found that major gaps existed between commanders’ ility tradespace needs and available capabilities for current and future cyberphysical-human systems. The SERC also characterized the benefits and limitations of using existing tools to address ility tradespace issues, via collaboration with other leading organizations in the DoD ERS tradespace area, such as the Army Engineer Research and Development Center (ERDC) and TARDEC organizations, NAVSEA, the USAF Space and Missile Systems Command; DoD FFRDCs such as Aerospace, Mitre, and the Software Engineering Institute; and Air Force and Navy participants via the SERC Service academies AFIT and NPS. 

As a result, the focus of Phase 2 has been to strengthen the conceptual frameworks underlying ilities tradespace and affordability analysis, and to apply the methods and tools identified and extended in Phase 1 on problems relevant to DoD, using the information available from development of a large weapon systems and large automated information systems. The SERC worked with system developers directly and via participation and leadership in Government and industry working groups in such organizations as INCOSE, NDIA, and the Army-led Practical Systems and Software Measurement organization, to gain a deeper shared understanding of the strengths and limitations of the tradespace tools and methods developed under Phase 1 and elsewhere. 

Phase 2 activities also expanded the set of ilities represented in the tradespace, organized them into a more orthogonal value-based, means-ends hierarchy, obtained initial results in identifying and quantifying the synergies and conflicts resulting from strategies to optimize individual ilities, and developed prototype tools for representing and applying the results. 

The Ility-oriented tool demos performed in Phase 1 also led to Phase 2 interactions with DoD organizations, particularly TARDEC and NAVSEA, interested in their applicability in enhancing their systems engineering capabilities. These interactions led to refinements of existing methods and tools to address set-based vs. point design of ground vehicles and ships, and on extensions from physical systems to cyber-physical-human systems and to affordability analysis. Further interactions leading to piloting engagements include AFIT’s use of the CEVLCC life cycle cost model and related T-X Training System Tradespace Analyses. The pilot program involves advanced pilot training aircraft, simulators and course instructional elements. Its pilot organizations are the Air Force Life Cycle Management Center and the Air Education and Training Command.

A third area of engagement starting from exploratory discussions in Phase 1 is a new task to develop Next-Generation, Full-Coverage Cost Estimation Model Ensembles, initially for the space domain, based on discussions and initial support from the USAF Space and Missile Systems Center (SMC). Phase 2 work on this topic involved several meetings with SMC and the Aerospace Corp. with USC and NPS to set context and initial priorities. These included addressal of future cost estimation challenges identified in the SERC RT-6 Software Cost Estimation Metrics Manual developed for the Air Force Cost Analysis Agency, and a scoping of fullcoverage of space system flight, ground, and launch systems; hardware, software and labor costs; and system definition, development, operations, and support costs. Initial identification of primary model ensemble elements and their cost drivers for a resulting COSATMO model resulted from a USC workshop involving Air Force, Navy, Aerospace Corp, CMU-SEI, and space systems industry representatives, along with guidance for structuring COSATMO in ways that would expedite usage of the framework for similar models in the ground, sea, and air domains. 

Phase 3 will have a new RT number, RT-113, reflecting its continuation into the new SERC 5- year contact. Its plans are oriented around the three Phase 2 tasks above, as follows: Task 1, iTAP Foundations, will complete the formalization and hierarchical organization of the key DoD ilities; fully populate the synergy and conflict relationships among the ilities; expand the quantification of the synergies and conflicts; and refine the prototype tools for representing and applying the results. It will also develop complementary views for addressing DoD highpriority ilities-related issues dealing with uncertainties such as sources of change and early costeffectiveness analysis.

Task 2, iTAP Methods and Tools Piloting and Refinement, will follow up on the engagements with DoD organizations started in Phase 2 and others, to pilot the application of SERC methods and tools to DoD- system ility tradespace and affordability issues, particularly in the cyberphysical-human systems and economic analysis areas. The methods and tools will then be refined, based on the results of the pilot applications.

Task 3, Next-Generation, Full-Coverage Cost Estimation Model Ensembles. Beginning with work in the space domain with USAF/SMC and the Aerospace Corp., this task will research and develop an ensemble of cost estimation models covering the systems engineering, development, production, operations, sustainment, and retirement. It will cover the full range of system artifacts and activities: for space systems, these would include flight systems, ground systems, and launch systems. The models will be developed to facilitate tailoring to domains other than space, using for example the domain-oriented work breakdown structures in MILSTD-881C.