Publication

This document is a summary of the work that was completed in the first increment year of the SERC Research Topic DO1/TTO2/0016 “Developing Systems Engineering Experience Accelerator (SEEA) Prototype and Roadmap” supported by the Defense Acquisition University (DAU). The purpose of the research project is to test the feasibility of a simulated approach for accelerating systems engineering competency development in the learner. The SEEA research project hypothesis is: 

By using technology we can create a simulation that will put the learner in an experiential, emotional state and effectively compress time and greatly accelerate the learning of a systems engineer faster than would occur naturally on the job. 

1. Project Management - Inability to support known and evolving customer/user feedback with current staff, budget and timeframe.
2. Technology Development - Inability to tradeoff long-term architecture and technology objectives (leading to successful open source support) vs. short-term prototype goals.
3. Experience Development - Inability to produce a prototype that provides a compelling experience, supports the desired learning and is seen to be authentic.
4. Integration - Inability to successfully integrate our many ideas, approaches, requirements and developed technology and design.
5. Evaluation - Inconclusive results due to threats to validity of Experimental design (inability to generalize results), limited availability of suitable subjects and insufficient literature to support development of evaluation instruments.

A set of lessons learned was compiled and categorized as noted below:

1. Competencies, Learning and Content
2. Complexity/Effort vs. Authenticity/Learning
3. Technology
4. R&D Processes

Follow-on work has been defined for Increment 2 that is focused on the following:

• Prototype Evaluation (cont.)
• Pilot System Development
• Pilot System Evaluation
• Open Source Preparation and Deployment
• External Developer Engagement

A final review of Increment 2 was held on September 6, 2012 with the following sponsor representatives: Jim Anthony, Tony Costanza, Darren Dusza, Steven Jones, Scott Lucero, Dave Pearson, and John Snoderly. The following are the highlights of the meeting results:

As the Experience Accelerator program is moving into Increment 2, there need to be increasing efforts made in the areas of pilot and tools support to both validate its effectiveness and improve the efficiency with which new experiences can be created, the latter being essential for open source community development. It was discussed that for piloting, there needs to be pre- and post-experience facilitation. Proper support needs to be put in place to enable this facilitation. It was also determined that DAU students should be used in this validation effort which will require government Institutional Review Board (IRB) support. In this case, we should be able to use an IRB exception in that this is normal educational development. The question arose on how the hypothesized outcome of “effectively compress time and greatly accelerate the learning of a systems engineer faster than would occur naturally on the job” would be measured. While the possibility of a large longitudinal study was discussed, the preferred approach was to compare the results of game traces of SEEA subjects to game traces of systems engineers who are deemed to be experts in selected areas, perhaps with the use of control groups with and/or without alternative equivalent educational instruction, to assess the subject‟s learning. The development of the capability to capture and analyze the subject‟s behavior in the Experience will be critical to this effort. Finally, a tools assessment plan needs to be completed to ensure that the Experience Accelerator development environment is adequate to support a thriving open source community.