Technical Report
WRT-1028: Validation Framework for Assuring Adaptive and Learning-Enabled Systems
-
Trusted Systems
Report Number: SERC-2021-TR-021
Publication Date: 2022-04-04
Project:
Validation Framework for Assuring Adaptive and Learning-Enabled Systems
Principal Investigators:
Dr. Bryan Mesmer
Co-Principal Investigators:
Validation in Systems Engineering is the process of determining that an artifact will perform its intended task in its deployment environment. Unlike verification (which compares measured properties against written specifications) validation concerns abstract and prospective claims about systems that are not yet built, as well as stakeholder preferences for their performance. As engineers commonly pursue validation by stating, challenging, and refining claims about system properties, we employ Toulmin argumentation models to represent this discourse in terms of the premises, claims, and warrants (rationale) that support or refute them.
We focus on the practical benefits of this view and contribute four ideas that enable construction of a tool for managing validation arguments.
(1) We define a vocabulary of primitive argument types, organized into a type-hierarchy by inheritance of critical questions (constraints on the warrant’s premises and conclusions). We claim that a handful of basic warrant types span the arguments found in systems validation and note that validation arguments commonly cite forms of trust (e.g., relying on expert opinion, delegating to a reputable firm, employing a vetted process).
(2) Given that warrants embed knowledge about the structure and content of validation arguments, we show that a warrant type-hierarchy supports a template-based editing model. We identify two main operations, for (a) identifying warrants that can support or refute specific aspects of the model, and (b) instantiating the elements of a warrant (its premises and conclusion) in a way that guarantees completeness and consistency of the result.
(3) We provide a means of evaluating validation arguments into a joint distribution of beliefs. We employ a Bayesian interpretation that sums over the conditional probability of premises, conclusions, and the probability that each warrant is apt (meaning it is a relevant model for drawing conclusions in the current context). The probabilistic interpretation of warrants is new, as is the capability to resolve argument models into a distribution over beliefs (vs a single belief vector with associated confidence values).
(4) We extend our representation of validation arguments to include alternatives and uncertainties. This makes it possible to employ the full machinery of decision analysis to evaluate organizational choices in service of systems validation, in the presence of conflicting arguments about claims and their underlying rationale.
We illustrate this capability with a worked example that computes the value of conducting a physical test to resolve a conflict between a simulation result, and an expert who believes the simulation should not be trusted.
BACKGROUND:
As stated in the most recent National Defense Strategy, success doesn’t go to the countries that develop new technologies first. Rather, it goes to those that most rapidly integrate those technologies into their warfighting systems and change their way of fighting to take advantage of the new capabilities.
Modern systems are increasingly valuing adaptation and rapid capability deployment over other success attributes. With a continuous increases in software-enabled capabilities, the Department of Defense (DoD) is requiring the ability to provide updates to systems in the field, and is emphasizing continuous development and deployment practices for all DoD programs. This requirement responds to a continuously adapting threat in today’s software-intensive systems, and the race to develop systems that use artificial intelligence and machine learning to rapidly adapt in the field. There is a pressing need for new methods and tools to continuously validate developed and deployed systems, and an emerging need to create methods and tools for validating autonomous systems, including those that learn from their experiences.
Previous research proposed a method for validating the spectrum of continuously adaptive and learning enabled systems through the use of a formal, logical argument method known as the Toulmin Argument Method. The concept of argument analysis has generally only been used in systems engineering by specialty engineering disciplines, such as safety and security. Use in those domains has more focused on the design of systems, and not validation. Traditional deterministic analyses and decomposition approaches often fail when addressing uncertainty and mathematically intractable state spaces. Argument analysis is well-suited to support decisions when faced with these conditions. The Toulmin Method could augment widely used SE methods focused on confirmation of static behaviors, by establishing a structured, logical argument that an evolving capability will succeed in the field. The Toulmin Method could be the basis for a formal argument process which could be codified in a user-guided toolset specifically designed to support validation of continually adaptive and learning enabled systems.
The Toulmin method establishes a language that provides a logical basis for validating behavior when evidence supports warrants that are key to supporting overall claims. The formal “soft” proofs of the Toulmin method show promise for validating the behavior of adaptive and learning systems in a rigorous, logical and consistent manner.
Initial research defined a methodology and process using Toulmin’s work that promises to be more formal, hierarchical, and repeatable than current validation methods, based in the fundamental concept of validation as collection of knowledge to build confidence that a system enables a capability or creates a desired effect in the world. Confidence comes from an argument that the system will enable the capability and create the effect. This argument starts at the very beginning of concept development, e.g., why do we believe this design is a good idea, and grows throughout system definition and realization, becoming more compelling in each step. The structure of the argument is created as a part of initial feasibility studies to enable reasoning about system validation and provide grounds to justify or refute heuristics. Throughout development, the validation process collects, adds and limits rebuttals, and builds a warrant that the system meets its defined capabilities, using evidence collected across the development process.
Validation in Systems Engineering is the process of determining that an artifact will perform its intended task in the world. It is a difficult assessment because it concerns abstract and prospective claims about artifacts, the interests of multiple stakeholders, and the performance of artifacts in environments that are often partially modeled and understood. We adopt the perspective that argumentation, vs proof, is the appropriate model for assessing system validity, and we introduce four ideas based on Toulmin argumentation theory that enable development of a software tool for managing validation arguments: a vocabulary of primitive argument types, a template-instantiation method for composing validation arguments, a mechanism for evaluating argument models into a joint probability over claims/beliefs, and a means of weighing decision options in the presence of the arguments and counterarguments commonly encountered in engineering activities. We conclude with a worked example clarifying the value of conducting a physical test in the context of a conflict between a simulation result, and an expert who asserts that the simulation is not valid. This example illustrates a more general integration of argument models with decision theory.