Identifying and Measuring Modularity Violations on Cyber-Physical Systems
Dr. Lu Xiao
Dr. Michael Pennock
In recent years, cyber-physical systems (CPS) have achieved widespread application in diverse areas including: civil infrastructure, energy, healthcare, transportation, automotive, smart appliances, and others. Usually cyber-physical systems are composed of diverse subsystems consisting of both physical and software components developed by different vendors. These components are deeply intertwined at various levels of abstraction (e.g. data flow, physical connections, and logical connections etc.) under changing contexts to achieve the desired functionalities and the respective quality attributes, such as performance, security, scalability, maintainability, etc. With the advance of technology, the recognition of new consumer needs, and the detection of deficiencies in current systems, components need to be upgraded, replaced, and fixed---frequently, and in many domains. The key question is: can the upgrade, replacement, or problem fix happen quickly and without disrupting the rest of the system? In the traditional software engineering field, it is often not the case. Modules without any explicit structural dependencies are seen frequently changing together when new features or bug fixes were implemented. This phenomenon is called a modularity violation (MV).
To address this issue, stakeholders, such as the U.S. Department of Defense, have increasingly emphasized modular and open approaches to system development. The 5 goals of the Modular Open Systems Approach (MOSA) are to improve interoperability, facilitate technical refresh through system evolution and technology insertion, enable cost savings and cost avoidance, and foster innovation and competition. It is difficult, however, to assess whether the resulting architectures and systems are truly modular, making it difficult to gauge progress toward achieving these goals. Ultimately, it would be good to have sufficient confidence that the modules in a CPS can actually be upgraded, replaced, and fixed in a plug-and-play manner without affecting one another to maximally leverage the benefits of modularization. In software systems, the consequences of modularity violations could be higher bug-rates [Mo et. al. 2015] and increased maintenance costs. It is possible that that modularity violations could be even more harmful in cyber-physical systems. Modularity violations, derived from the latent relationships among software and hardware components, prevent the long-term evolution, maintenance, and success of cyber-physical systems. In particular, due to the inflexibility and high cost of evolving hardware modules, modularity violations in cyber-physical systems could more easily lead to vendor lock-in compared to software developed to run on commercial-off-the-shelf hardware. The stakeholders lack techniques, metrics, and models that would allow them to detect, measure, and understand modularity violations in developed and acquired cyber-physical systems.
In the incubation phase of RT-180, researchers from the Stevens Institute of Technology organized modularity violations in cyber-physical systems into three different types: 1) software vs. software, 2) software vs. hardware, and 3) hardware vs. hardware. The first type of modularity violation has been extensively investigated in prior work [Wong et. al. 2009; Schwanke et. al. 2013; Xiao et. al. 2014; Ran et. al. 2015; Xiao et. al. 2016]. There, the approach to identify and measure modularity violations relies on the analysis of source code and operational data such as maintenance activity records. These are automatically and comprehensively tracked in version control systems. The challenge to extending this type of analysis to identify modularity violations involving hardware is the lack of systematic records of operations involving hardware components. Unlike software projects that use standard version control systems to keep track of who made what changes to which parts at what time and for what reason, changes to hardware components are not always tracked in similar detail and are not as easily accessible.
To address this challenge, Stevens researchers evaluated the feasibility of inferring a subset of hardware modularity violations from an analysis of records of software changes. More specifically, some software changes may be a consequence of hardware-related modularity violations. If that were the case, analysis of version control systems could be used to detect some hardware-related modularity violations outright and flag other potential violations for further investigation. The issue is whether or not there is actually enough information in a version control system to infer a hardware modularity violation. To evaluate the feasibility of this idea, the incubator phase of this project involved the analysis of two open source cyber-physical systems: OpenWrt and MD-PnP, which aim to develop a common software infrastructure for the Internet of Things (IoT) and medical systems respectively. To analyze these systems, Stevens developed a keyword-based heuristic to help identify software-hardware modularity violations, where the software artifacts changed due to hardware related issues. They identified 70 software source files that are potentially involved in software-hardware level modularity violations because the changes made to these files involved hardware related concepts. The implication is that hardware-related concepts that propagate changes to software artifacts imply a potential modularity violation at the software-hardware level. Consequently, the incubator project demonstrated the feasibility of identifying potential modularity violations in a cyber-physical system by analyzing a software version control system.