Publication

PART I: RESEARCH TASK OVERVIEW

Part I of this report provides and overview of this research task, including the surrogate pilot experiments and sets the context for the needed research as defined and evolved by our sponsor, as well as the objectives, scope and organization of this report. This part also provides a summary of the current set of research use cases, our Phase 1 & 2 efforts, status, events, demonstrations, deliverables, models, prototype tools and recommendations based on our increased understanding of the research objectives.

INTRODUCTION:

In 2013, the Naval Air Systems Command (NAVAIR) at the Naval Air Station, Patuxent River, Maryland initiated research into a Vision held by NAVAIR’s leadership to assess the technical feasibility of a radical transformation through a more holistic model-centric system engineering (MCSE) approach. The expected capability of such an approach would enable mission-based analysis and engineering that reduces the typical time by at least 25 percent from what was achieved at that time for large-scale air vehicle systems using a traditional document-centric approach. The research need included the evaluation of emerging system design through computer (i.e., digital) models.

Through Systems Engineering Research Center (SERC) research tasks (RT-48, 118, 141, 157, 170) starting in August 2013 there was considerable emphasis on understanding the state-of-the-art through discussions with industry, government and academia [25] [32] [39]. The team, comprised of both NAVAIR and SERC researchers, conducted over 30 discussions, including 21 on site, as well as several follow-up discussions on some of the identified challenge areas and approaches for a new operational paradigm between government and industry.

In 2015, the NAVAIR leadership concluded that they must move quickly to keep pace with the other organizations that have adopted MCE as the pace of evolution is accelerating enabled by rapidly evolving technologies. NAVAIR made the decision to press forward with a Systems Engineering Transformation (SET). In March of 2016, there was a Change of Command at AIR 4.0 (Research and Engineering) and NAVAIR leadership decided to accelerate the SET. Our research sponsor, Mr. David Cohen proposed a new operational paradigm referred to as the SE Transformation Framework that has evolved into the concept depicted by Figure 3 in the report. The research efforts starting in 2017 under RT-170 started developing a surrogate pilot concept to assess and refine the execution of the SET Framework through a series of experiments conducted as evolving pilot projects. The emphasis was on a new operational paradigm to mission and systems engineering, analysis and model-based acquisition, which would be led by NAVAIR with collaborative design efforts led by industry. We participated with our sponsors in more industry meetings to assist in communicating and clarifying these concepts for a new type of collaboration, and to assess the impacts on the NAVAIR enterprise, from both a technical and socio-technical perspective. Many objectives for assessment and refinement of the SET Framework are characterized as objectives and captured as part of a Surrogate Pilot Project plan and model that is being traced to experiment models, demonstrations, results and lessons learned.

Briefly, as articulated by our sponsor, the concept of the new SET Framework for transforming from a document-centric process with monolithic reviews to an event-driven model-centric approach involves, but is not limited to:

• A concept for collaborative involvement between Government and Industry to assess mission and System of Systems (SoS) capability analyses, where NAVAIR has the lead to:
• Involve industry in SoS capabilities assessments during mission-level analysis (to the degree possible)
• Iteratively perform trade space analyses of the mission capabilities using approaches such as Multidisciplinary Design, Analysis and Optimization (MDAO) as means to develop and verify a model-based specification
• Synthesize an engineering concept system model characterized as a model-centric specification and associated contractual mechanism based on models or associated formalism
• At the contractual boundaries, industry will lead a process to satisfy the conceptual model addressing the Key Performance Parameters (KPPs), with particular focus on Performance, Availability, Affordability, and Airworthiness to create an Initial Balanced Design
• Industry too applies MDAO at the system and subsystem level
• There is a potential need to iterate back to re-balance the needs if the trade space analyses of the solution/system for the program of record (POR) cannot achieve mission-level objectives
• All requirements are tradeable if they don’t add value to the mission-level KPPs
• These are asynchronous activities in creating an Initial Balanced Design
• Government and Industry must work together to assess “digital evidence” and “production feasibility”

Another objective for this new operational paradigm is to replace large-scale document-centric reviews such as Systems Requirements Review (SRR), System Functional Review (SFR), Preliminary Design Review (PDR), etc. with continual event-driven reviews using objective or subjective evaluation based on model-centric information. Some initial surrogate pilot demonstrations illustrated a potential approach to replace large-scale document-centric reviews with continual event-driven reviews directly within the model using objective and subjective evaluation based on model-centric information and digital signoffs, where the digital signoff is linked to the model evidence satisfying some criteria typically required at a formal review or as defined in a CDRL. A collaborative AST is being used in the surrogate pilot and is playing a key role with the continuous asynchronous reviews. NAVAIR needs some type of decision framework to assess evolving design maturity with considerations of value to the KPPs, risk and uncertainty. This surrogate pilot experiments factor in these and other types of objectives.

Early in 2017, the SET team developed the plan for rolling-out SET to NAVAIR, which defined six major Functional Areas which includes:

• SET Research (conducted by the SERC, and discussed in this report)
• Workforce & Culture
• Integrated Modeling Environment
• Process & Methods
• Policy, Contracts and Legal
• SET Enterprise Deployment (and Surrogate Pilot Experiments, also discussed in this report)

These Functional Areas have other sub functions as part of the overall effort, as shown in Figure 1 of the report. The Surrogate Experiments are being conducted using multi-phase Surrogate Pilot use cases are part of the SET Enterprise Deployment. The SET Research is being performed in the context of the surrogate experiments. The broader impacts of this research to the other sub functions of SET is also reflected by the dash boxes.

The SET Surrogate Experiments are elaborating mission and system analyses and requirements using a hypothetical system called Skyzer. Skyzer has a Concept of Operations (CONOPs) for an UAV that provides humanitarian maritime support use cases (e.g., search and rescue) as reflected in Figure 4 of the report. Phase 1 of the Surrogate Pilot officially kicked-off on December 7, 2017. The timeline of events for the Surrogate Pilot planning and execution are shown in Figure 5. Phase 1 had a very narrow scope in order to focus on the execution through the SET Framework Elements (1-4) as quickly as possible. The scope of the UAV design as requested by our sponsor included multi-physics design considerations that are based on Computational Fluid Dynamics (CFD), topology optimization, structural analysis, weight and vehicle packaging. The surrogate pilot team officially released the RFP concluding the Phase 1 Element 1 & 2 efforts. Performance constraints such as speed of 170 knots forced the design to be something other than a traditional helicopter and ultimately a design similar to the Bell Eagle Eye was proposed in the surrogate contractor RFP response models, which was evaluated in a surrogate source selection by the government team. The efforts moving forward are to align efforts with the SET priorities for the Phase 2 use cases.

NAVAIR has been reaching out to industry to engage in discussions about this new operational paradigm to acquisition since 2015. Industry has responded favorably about this change of direction. For example, industry initiated a new CONOPS of operations with organizations involved in the Aerospace Industry Association (AIA) working group [3]. The National Defense Industry Association (NDIA) Modeling and Simulation group which is looking at approaches for using digital engineering for competitive down select. In response, VADM Grosklags provided an overview of the SE Transformation at the NDIA Systems Engineering Conference in October 2017 [86]. On March 8, 2018, NAVAIR officially announced the SET as part of a larger Industry Request for Information (RFI), where industry was invited to six (6) hours of briefing material on the details of the SET [123], including details about the surrogate pilot experiments. In August 2018 NAVAIR conducted an industry review of the Acquisition System Reference Model (ASRM) to provide industry with the opportunity to make constructive comments on representation and content that will likely be provided as “System Model(s)” as GFI as part of future solicitations such as RFI or RFPs. We presented at NDIA Systems Engineering Conference in October 2018 and were approached by industry who wants to participate in the surrogate pilot; the use cases are being planned for Phase 2. At the two-day Model-based Ecosystem breakout session at INCOSE in January 2019, we briefed details about our surrogate experiments and use of OpenMBEE [138] as a foundational element of our AST and found out that Boeing has 40 programs and over 200 users using OpenMBEE, and Lockheed Martin also has many programs but plans to be part of the open-source community to advance OpenMBEE by developing the next version of the Model Management System (MMS) component of OpenMBEE.

It was announced during the presentation at the SET RFI Industry Day that the Surrogate Pilot experiments, models, generated specifications, results, and lessons learned would be shared with industry and government on the All Partners Network (APAN.org). APAN was setup and is managed by Defense Information Services Agency (DISA). DoD organizations can request their own groups, and NAVAIR has several groups for the SET. Some are internal for NAVAIR people and their contractors, but the Surrogate Pilot Group (https://community.apan.org/wg/navair-set/set-surrogate-pilot/) is open to the public with the proper registration in APAN. The Surrogate Pilot group captures weekly progress for the SET Surrogate Pilot in the Discussion threads, often with videos. We are sharing this with Industry and Government to solicit feedback and recommendations on the way we are proceeding in this pilot. Many of the lessons learned from this surrogate pilot are reflected in this report.