Research Highlight: Framework for AI Resilience through Evaluation of Systems and Technology (FAIREST)

Mar 9, 2026

The SERC portfolio of funded research tasks furthers its guiding vision of being the networked national resource to further systems research and its impact on issues of national and global significance.

Research Highlight

Research Task: Framework for Artificial Intelligence (AI) Resilience through Evaluation of Systems and Technology (FAIREST)
Sponsor: Office of the Under Secretary of War for Research and Engineering (OUSW(R&E)), Developmental Test, Evaluation, and Assessments (DTE&A)
Principal Investigator (PI): Ms. Emma Meno (Virginia Tech)
Co-PI: Dr. Tyler Cody (Virginia Tech)
Original PI: Dr. Peter Beling (formerly Virginia Tech)

Read the final technical report

What happens between a drop in performance of sustained artificial intelligence enabled systems (AIES) and the re-engineering of a solution?

While this is the focus of resilience mechanisms for such systems, the question remains largely unaddressed in academic literature and defense guidance. Operational resilience framing to address threats to AIES performance may require encompassing the various systems, technologies, and processes that interact with AIES. Therefore, resilience mechanisms must be interoperable, compatible, and contribute to the AIES mission symbiotically, and should be considered from system inception and undergo concurrent evaluation with the statistical performance of AI within AIES during Test & Evaluation (T&E).

“AIES operational resilience represents the ability of systems to resist, absorb, recover and/or adapt in the face of adversity that can compromise mission-critical functions. Rather than focusing solely on robustness to minimize the impact of performance degradation, operational resilience encompasses how the system changes under evolving or unknown conditions. It is key to engineering functionally and testing thoroughly, particularly in considering risks from cyber threats and systems when testing is viewed as a continuum.”
– Emma Meno, Principal Investigator

Purpose

The Framework for Artificial Intelligence Resilience through Evaluation of Systems and Technology (FAIREST) enables planning for testing of AIES for operational resilience and cyber survivability. The goal is to provide justified confidence earlier in program lifecycles, and to mitigate risks that can arise from lack of test data and unknown cyber threats. The research is expected to support program offices and contractors in test planning and design.

Results

Outcomes from the research team’s analyses and comprehensive literature reviews include:

Laying the groundwork for defining testable resilient requirements for AIES. Utilizing resilience engineering, FAIREST testable requirements were derived as degradation detection, root cause attribution, response, evaluation, operator confidence, operator readiness, execution assurance, and retesting, providing lifecycle-informed test planning, model-traced requirements, and standardized metrics. A set of evaluation strategies and scoring approaches was presented to inform resilience testing metrics.
Reviewing additional guidance and best practices for cyber policy guidance.
Demonstrating methodology using a simulated military environment for safe minefield navigation in which FAIREST-enabled strategy encountered less mines compared to degraded model.
Developing initial taxonomy for defining non-adversarial and adversarial catalysts and associated attributes.
Defining conceptual framework for applying mission engineering and integration considerations for AIES.

The final technical report includes an overview of outcomes and a summary of whitepapers from the different research tasks across this program. (The research team plans to submit whitepapers for publication at various venues, not yet finalized at the time of this highlight.)

“The research established a solid foundation for AIES operational resilience and cyber survivability. The approach framed and scoped AI resilience concepts, modelled the functional role of AI in resilience context, drafted a taxonomy for AIES disturbances, defined a methodology for AIES resilience testing, and demonstrated the method in simulation. Looking ahead, FAIREST offers impact in mission engineering and integration considerations for AIES, contributions to cyber policy guidance, and future research in testing and evaluation (T&E) of AIES operational resilience.”

– Ms. Meno

Strategic Implications

Increased understanding of mission-aware AIES resilience can guide future research that explores how AI model performance has potential implications within the mission profile.

“With the rise of frontier and agentic AI capabilities, defining and addressing resilience concepts and mechanisms (especially with an SE perspective) will become even more crucial for critical applications. The inherent stochasticity and opaqueness of models employed in AIES introduce unique traits for testing. Appropriately defining and scoping operational resilience, resilience mechanisms, and accompanying T&E methodologies in context of the AIES mission stack and lifecycle will help identify emerging implications from novel models and techniques.”
– Ms. Meno

Follow SERC on LinkedIn for regular updates on systems engineering research.