Publication

This report documents the results of the SERC research task RT-110: Enterprise Systems Analysis. This task is a part of a larger SERC focus area: Enterprise and Systems of Systems (ESOS). The overarching goal of the ESOS focus area is to understand how, if at all, systems engineering methods can be adapted to address the challenges imposed by both enterprise systems and systems of systems. 

As one aspect of addressing the challenges imposed by enterprise systems, an enterprise modeling methodology was developed as part of SERC research task RT-44a. The chief objective of this task (RT-110) was to evaluate that methodology critically and to identify areas for improvement. To that end, the method was applied to a case study of counterfeit parts in the DoD supply chain as test case. In parallel, the methodology was considered from a theoretical viewpoint based on the work conducted in adjacent domains such as complex systems, modeling and simulation, human factors, and economics. This two-pronged approach challenged the methodology both practically and technically. This report documents the findings of that evaluation. 

In short, the methodology was found to be a useful construct, but some challenges and areas for improvement remain. In particular, some of the steps were found to be vague and require additional guidance to actually implement. The methodology makes extensive use of visualization, but the efficacy of visualization applied to enterprise systems remains unclear. Finally, the methodology relies heavily on the composition of computational representations of different aspects of an enterprise system. Such compositions impose substantial technical challenges when attempted for enterprise systems. 

To address these issues, a number of recommendations were made. First, potential improvements and refinements to the methodology steps were developed. These culminated in a set of high level requirements for an enterprise modeling archive that would aid enterprise modelers in the execution of the methodology by capturing guidance and best practices. 

Second, past work in visualization was examined to identify guidance for implementation. The output of this was a conceptual design for an interactive visualization based on this analysis. However, a number of unresolved difficulties were identified and would require further experimentation to address. 

Third, a set of strategies was developed to mitigate the difficulties associated with model composition. These strategies explicitly recognize that model composition will not always be possible and compensates appropriately. Visualization was also considered as a potential approach to support the development of strategies when complete composition is not possible.

Beyond the modeling methodology itself, the findings of this research task have implications for the larger challenge of applying systems engineering approaches to enterprise systems. In particular, traditional systems approaches can only ever be applied to a subset of an enterprise problem. When one persists in applying only traditional systems approaches to an enterprise, one necessarily excludes all aspects of the enterprise system that are not “well behaved.” Unfortunately, this has the consequence of implicitly assuming that everything outside of the modeled subset is either fixed or has no impact on that subset. While this implicit assumption can be enforced to a degree via organizational constructs in traditional systems engineering, it is almost certainly invalid when attempting to engineer enterprise systems. Since enterprises are often highly interconnected (both internally and externally) and intrinsically adaptive, optimizing over a subset of the problem will often result in a fragile solution that can trigger a number of unintended consequences via second or higher order effects. 

To address this larger challenge, the aforementioned strategy set is critical. In essence, it explicitly recognizes the modeling limitations and then allows one to consider how to adapt to or hedge sudden changes. Here multiple models become a potential asset as they may allow a decision maker to map out potential future scenarios and hedge them. Finally, application of these strategies can be facilitated through the identification of leading indicators of a major, impending change in an enterprise system. While there is some theoretical basis for these indicators, more research is required to assess their practicality. 

While a great deal of progress was made over the course of RT-110, work remains. First and foremost, the model developed to analyze the counterfeit parts case study should be applied to determine if it actually yields useful insights. This is necessary to establish the ultimate utility of the enterprise modeling approach. Second, a single example is not sufficient to validate a methodology. Consequently, another case study should be initiated to continue to challenge the enterprise modeling methodology. Third, the efficacy of visualization to support enterprise decision making needs to be established. This means that one or more experiments should be conducted to determine the conditions under which visualizations actually aid enterprise decision makers and enhance decision making. Finally, additional work is necessary to implement the recommendations made in this report. This includes explicitly mapping the chain of phenomenon to representation to model, investigations into how to operationalize the strategy framework, as well as additional guidance for model composition and when it is appropriate to do so.