Publication

The overarching goal of the research in enterprise systems analysis has been to support decision makers and policy makers through the development of new modeling and decision support approaches. Enterprise systems are defined as those where there are multiple interacting organizations, but there is not central locus of control. As a result, change must often be the result of influence and incentives as opposed to command and control. To understand the potential impact of a policy option, one needs to capture the spread of potential future scenarios that may result. In particular, the long-term goals are to allow decision makers and policy makers to

• Identify the key drivers of system behavior and resulting outcomes
• Perform “what if” analyses
• Evaluate the efficacy of policy options to alter system behavior and outcomes
• “Test drive” the future
• Allow key stakeholders experience the behavior of the “to be” system

In support of these long-term goals, the objective of RT-138 was to evaluate and evolve the enterprise modeling methodology developed during RT-44 and RT-110. The primary mechanism to do so was the counterfeit parts case study. The focus of the case study was to understand the impact of hypothetical policy options to combat the intrusion of counterfeit parts into the defense supply chain. It is ideal as a testing ground for the modeling methodology because it involved all of the key features required of an enterprise systems: multiple interacting organizations (multiple government agencies, private corporations, and counterfeiters), no locus of control (the government can promulgate policy but suppliers don’t have to supply, and counterfeiters can try to bypass), and the system is in the midst of a major shift in state (counterfeiting has rising dramatically in recent years).

Beyond the counterfeit parts case study, there were also a number of other subtasks that explored various issues related to the application of the methodology. Figure 1 depicts the subtasks of RT-138 and relationships among them.

In short, the challenge of modeling enterprise systems is that the intrinsic complexity of the underlying social systems fundamentally limits the ability to make precise predictions using models and simulations. The analysis of historical data and extrapolation from that data may be viable during periods of relative stability. However, social systems are prone to abrupt shifts behavior (sometimes referred to as bifurcations in the dynamical systems literature). This circumstance requires a different approach to employing models for decision making than that traditionally applied in engineering, which is essentially trend extrapolation. To that end, each of the subtasks were intended to explore how to address a different problem brought about by the complexity of the enterprise system. To summarize:

• The classical decision models used in engineering modeling treat humans as perfectly rationale decision makers. However, it is well known that this not an accurate representation of human behavior. Given, the central role of humans in enterprise systems, the behavior economics case study was intended to the apply research on modeling actual human behavior to an enterprise problem. The question was whether or not this would enable the detection of behavioral changes or if the effect would get lost in the “noise.” Ultimately, the case study was conducted by examining the real world case of dynamically tolled roads for congestion management.
• Modeling enterprise systems necessarily requires the simultaneous consideration of the system from multiple perspectives. Given the nature of enterprise systems this often requires models from different scientific disciplines that were not intended to be integrated. Previous tasks (RT-44, RT-110) considered some of the challenges of composing such models. It is has been done successfully in some instances, but often times it proves difficult. Thus, the question is what allows one to reuse and compose models from different disciplines successfully. In the Aligning Phenomena with Canonical Models task, we considered how the nature of the phenomena in question affected such efforts.
• Assuming that one can build a model of an enterprise system that can be used to find abrupt shifts in the behavior of an enterprise system, the question remains how to use that information to support decision and policy making. During RT- 110, we proposed a notional strategy framework to manage the resulting risks. In the Refine Strategy Framework task we examined the notional framework and linked the strategies to the source of the uncertainty.
• No model is capable of forecasting all possible scenarios. Consequently, some changes in enterprise behavior will be a surprise. Some researchers have investigated the possibilities of early warning signals of such “surprises” in biological and financial systems. In the Methods for Mitigating Complexity task, we critically review the literature on this topic and consider whether or not these techniques can be used to mitigate the impact of a surprise change in enterprise behavior.
• Once one builds a model of an enterprise system, there remains the concern of how the insights derived can be internalized by analysts, decision makers, and policy makers. Many have employed interactive visualizations in such situations, however, it is unclear how effective they are. Given the complexity of enterprise systems, there is a very real possibility that the decision makers will perceive spurious correlations as causal. To study this concern in more depth, the Visualization Experiment task involved a human subjects research experiment where subjects were asked to use an interactive interface to diagnose the contributing factors that led to an enterprise failure.
• Finally, a single data point is never sufficient to validate an approach. Consequently, while the counterfeit parts case study provided a single test case for the enterprise modeling methodology, it is not enough to validate it. It is entirely possible that the results were artifacts of unique feature of the case. Consequently, the Initiate Follow-on Case Study task was intended to investigate another enterprise system that could serve as a second test case for the enterprise modeling methodology in an entirely different context.

The remainder of this report discusses the results of these research tasks. First are the two case studies. Section 2 describes the counterfeit parts case study, and Section 3 describes the behavioral economics case study. Section 4 considers how models of such cases are affected by model composition and reuse issues. Section 5 critically reviews the early warning signals literature for applicability to enterprise systems. Section 6 draws together the implication of the previous sections and considers how that affects enterprise modeling and the strategy framework. Section 7 explains the visualization experiment. Section 8 revisits the enterprise modeling methodology based on everything that has been learned through RT-138. Section 9 presents the preliminary description of the follow on to the counterfeit parts case study. Finally, Section 10 concludes the report and discusses future work.