Archives

SERC TALKS: “How is T&E Transforming to Adequately Assess DOD Systems in Complex Operating Environments?”

This was the final Talk of the three-part series on “Test and Evaluation” curated and moderated by: Dr. Laura Freeman, SERC Research Council Member, Director of the Hume Center’s Intelligent Systems Lab and Assistant Dean for Research in the College of Science, and Research Associate Professor in the Department of Statistics at Virginia Tech. You can view the recording of the Talk below. Prior Talks in the series are as follows:

Speaker: Dr. Sandra Hobson, Deputy Director for Strategic Initiatives, Policy and Emerging Technologies, Office of the Director, Operational Test and Evaluation, Office of the Secretary of Defense | CONTACT

Abstract
The Director, Operational Test & Evaluation (DOT&E) is senior advisor to the Secretary of Defense on operational test and evaluation (OT&E) and live fire test and evaluation (LFT&E) in the Department of Defense (DoD). Ultimately, the office of DOT&E is responsible for ensuring testing is adequate to confirm operational effectiveness, suitability, survivability, and lethality of defense systems in combat use. The technologies we are testing today and will in the future require that we rethink classical approaches to T&E. As the National Defense Strategy notes, “We cannot expect success fighting tomorrow’s conflicts with yesterday’s weapons or equipment.” Similarly, we need to modernizing T&E assessment tools, infrastructure, and expertise. In this talk, I will discuss the critical role DOT&E plays in ensuring we provide systems that work to our warfighters. I will discussion how the DOT&E Science and Technology Strategic Plan is focusing on making advancements in current T&E CONOPS in the technology areas of Software, Cybersecurity, and Next-Generation capabilities. I will also discuss the need for an integrated T&E lifecycle and how digital transformation plays a key role in changing how we do business. Finally, transformation will require us to engage the T&E workforce in education and training on new methods. New partnerships with communities pioneering new methods (to including the SERC network in academia) will assist in our transformation.
Bio
Dr. Hobson was appointed to the Senior Executive Service in 2014 as a Deputy Director in the Office of the Director, Operational Test and Evaluation, within the Office of the Secretary of Defense. She is currently the Deputy Director for Strategic Initiatives, Policy and Emerging Technologies focused on advancing operational and live fire test and evaluation tools, methods and capabilities to meet the test and evaluation demands of the future.
Prior to this appointment, Dr. Hobson supported the Director, Operational Test and Evaluation, in two other capacities: first as a Research Staff Member at the Institute for Defense Analyses and then as an Aircraft Systems and Weapons Staff Specialist in the Office of the Director, Operational Test and Evaluation. During these tenures, she provided guidance and direction on the survivability and lethality evaluation of major Department of Defense aircraft and weapons acquisition programs to include the F-35 Joint Strike Fighter, the Air Force KC-46A aerial tanker, and the Navy P-8A multi-mission maritime aircraft.
Earlier in her career, Dr. Hobson worked as a Research Associate resident at the National Institute of Aerospace/NASA Langley Research Center. She defined and executed an interdisciplinary research that included the application of smart materials to actively control boundary layer flow demonstrating the ability to advance aerospace vehicle concepts and technologies. She authored articles, scientific publications and studies to explore the application of this research towards building more fuel efficient aircraft. Other duties included Aerospace Engineering Department lecturer and Airbus North America consultant.
Dr. Hobson earned her Bachelor of Science degree in Aerospace Engineering from the United States Naval Academy and her Doctor of Philosophy degree in Aerospace Engineering from A. James Clark School of Engineering at University of Maryland. Dr. Hobson is a recipient of the National Fellowship for Exceptional Researcher awarded by UNESCO (the United Nations Educational, Scientific and Cultural Organization), and two Secretary of Defense Medals for Meritorious Civilian Service.
DOWNLOAD SLIDES
Moderator: Dr. Laura Freeman, SERC Research Council Member, Director of the Hume Center’s Intelligent Systems Lab and Assistant Dean for Research in the College of Science, and Research Associate Professor in the Department of Statistics at Virginia Tech  |  CONTACT

More information on all SERC Talks can be found here. Thank you!

SERC TALKS: “Progress in Test and Evaluation of AI-enabled Systems in the DoD”

DOWNLOAD SLIDES

Speaker: Dr. Yevgeniya “Jane” Pinelis, Chief, Test, Evaluation, and Assessment, Department of Defense Joint Artificial Intelligence Center (JAIC)  | CONTACT

Abstract
Though the objectives of independent government test and evaluation (T&E) are similar between traditional and AI-enabled systems (AIES), the science, practice, skills, and infrastructure necessary to evaluate these data-dependent systems do require an update. Over the last two years, the DoD Joint AI Center (JAIC) has led the DoD test community in identifying and filling the gaps in T&E of AI. We have made great progress, in partnership with other DoD test stakeholders, academia, and industry. Together, we have put forward T&E frameworks, tools, and contracting approaches. This talk will discuss our existing challenges, current and aspirational solutions, and the way ahead for the DoD T&E of AIES.
Bio
Dr. Jane Pinelis is the Chief of the Test, Evaluation, and Assessment branch at the Department of Defense Joint Artificial Intelligence Center (JAIC). She leads a diverse team of testers and analysts in rigorous test and evaluation (T&E) for JAIC capabilities, as well as development of T&E-specific products and standards that will support testing of AI-enabled systems across the DoD.
Prior to joining the JAIC, Dr. Pinelis served as the Director of Test and Evaluation for USDI’s Algorithmic Warfare Cross-Functional Team, better known as Project Maven. She directed the developmental testing for the AI models, including computer vision, machine translation, facial recognition and natural language processing. Her team developed metrics at various levels of testing for AI capabilities and provided leadership empirically-based recommendations for model fielding. Additionally, she oversaw operational and human-machine teaming testing, and conducted research and outreach to establish standards in T&E of systems using artificial intelligence.
Dr. Pinelis has spent over 10 years working predominantly in the area of defense and national security. She has largely focused on operational test and evaluation, both in support of the service operational testing commands and also at the OSD level. In her previous job as the Test Science Lead at the Institute of Defense Analyses, she managed an interdisciplinary team of scientists supporting the Director and the Chief Scientist of the Department of Operational Test and Evaluation on integration of statistical test design and analysis and data-driven assessments into test and evaluation practice. Before, that, in her assignment at the Marine Corps Operational Test and Evaluation Activity, Dr. Pinelis led the design and analysis of the widely publicized study on the effects of integrating women into combat roles in the Marine Corps. Based on this experience, she co-authored a book, titled “The Experiment of a Lifetime: Doing Science in the Wild for the United States Marine Corps.”
In addition to T&E, Dr. Pinelis has several years of experience leading analyses for the DoD in the areas of wargaming, precision medicine, warfighter mental health, nuclear non-proliferation, and military recruiting and manpower planning.
Her areas of statistical expertise include design and analysis of experiments, quasi-experiments, and observational studies, causal inference, and propensity score methods.
Dr. Pinelis holds a BS in Statistics, Economics, and Mathematics, an MA in Statistics, and a PhD in Statistics, all from the University of Michigan, Ann Arbor.
Moderator: Dr. Laura Freeman, SERC Research Council Member, Director of the Hume Center’s Intelligent Systems Lab and Assistant Dean for Research in the College of Science, and Research Associate Professor in the Department of Statistics at Virginia Tech  |  CONTACT

More information on all SERC Talks can be found here. Thank you!

SERC TALKS: “What Does Test & Evaluation Mean in a Digital Engineering Enabled World?”

DOWNLOAD SLIDES

Speaker: Dr. Darryl K. Ahner, P.E., Director, Scientific Test and Analysis Techniques Center of Excellence (STAT COE), Air Force Institute of Technology (AFIT) | CONTACT

Abstract
Testing has often been looked at as a (un)necessary evil. In order for testing to be value added, it must not take too much time, provide quality information, and provide timely information to appropriate affect systems engineering and acquisition decisions. In this talk we will discuss the demand for testing events, the Scientific Test and Analysis Techniques process (since STAT is mandated in DoD policy anyway), and what test planning may look like in a digital engineering environment. Examples of efficient and effective test planning will also be discussed, both those that were conducted during development and those developed post deployment.
Bio
Dr. Darryl K. Ahner, P.E. is the Director, Scientific Test and Analysis Techniques Center of Excellence (STAT COE) and was instrumental in shaping its graduate-level test engineering operational concept leading to its establishment by the Office of the Secretary of Defense in 2012. He is also a Professor of Operations Research, Program and Curriculum Chair of the Data Analytics and Data Science programs, and Director of the Center for Operational Analysis at the Air Force Institute of Technology. As Director of the STAT COE, he is responsible for executing the Center's mission to provide independent advice and assistance to designated major acquisition programs in the application of scientific test and analysis techniques for test planning and the resulting data analytics. The Center also conducts externally funded applied STAT research, provides STAT education, and develops best practices and case studies in the employment of STAT. Dr. Ahner is a licensed Professional Engineer. He has written over 50 technical articles, made over 70 technical presentations, and has led several technical working groups. He has been a faculty member at the Air Force Institute of Technology (AFIT) since 2010, where his research emphasizes optimization of test & evaluation, autonomous systems, big data analytics, reliability, stochastic models, simulation, and military operations research applications. Prior to AFIT, he taught for five years at the United States Military Academy at West Point and served for three years as the research director for an Army analysis center at the Naval Postgraduate School. In addition to being a member of the Institute for Operations Research and the Management Sciences (INFORMS), and International Test and Evaluation Association (ITEA), Dr. Ahner has been an active member of the Military Operations Research Society (MORS), most recently serving as Vice President for Professional Development and Vice President (Secretary of the Society) on the Executive Committee. A 1990 graduate of the United States Military Academy at West Point, Dr. Ahner retired from the US Army in 2012. During this time he was awarded the Legion of Merit, an award usually reserved for general officers and a Special Act Award for leading modeling, simulation, and analytic research efforts. His academic studies include Rensselaer Polytechnic Institute, where he earned a M.S. in applied mathematics and a M.S. in operations research and statistics, and Boston University where he earned his doctoral degree in systems engineering while a Charles Stark Draper Laboratory Fellow. Dr. Ahner’s awards include the 2020 Colonel Charles A. Stone Award in recognition of an individual who has made outstanding contributions to furthering the AFIT mission, member of the 2019 U.S. Air Force Analysis Team of the Year Honorable Mention, 2018 Journal of Defense Analytics and Logistics Best Paper for “Finding the Fuel of the Arab Spring Fire: A Historical Data Analysis,“ the 2016 Richard H. Barchi Prize for the best paper given at the Military Operations Research Symposium for “Suitability Analysis of Continuous-Use Reliability Growth Projection Models”, the 2017 Affiliate Society Council of Dayton Outstanding Engineers & Scientists Award, the 2014 E. Grosvenor Plowman Award for Best Paper at Council of Supply Chain Management Professionals Global Conference for "Exploring the Use of the Upper-Semi Variance as a Robust Estimator For Calculating Safety Stocks," the 2012 Colonel Charles A. Stone Award, a 2008 Special Act Award including a $5000 Cash Award presented by a 4-star general in recognition of meritorious scientific achievement, and the 2007 Wilbur B. Payne Award in recognition of the best Army analytical technical study.

More information on all SERC Talks can be found here. Thank you!

NOTE TO PARTICIPANTS: SERC Talks broadcasts is now using the Zoom for Government Webinar platform. Prior to the SERC Talk, we encourage all participants to install and test the platform to exercise full virtual capabilities. Installation is NOT required to join the session, as Zoom is accessible via web browser.

Download Zoom for Government client | Zoom for Government FAQ | Zoom for Government: How to join from your browser

More information on all SERC Talks can be found here. Thank you!

SERC TALKS: “How are Digitalization and AI Driving Improvements in Healthcare Delivery?”

DOWNLOAD SLIDES

Speaker: David Zion, Head, Systems Engineering Center of Excellence, Strategic Program Leader, Philips Healthcare | CONTACT

Abstract
This presentation examines the progression of health systems from individual diagnostic and treatment devices to connected systems and solutions. The transformation of health care delivery has changed the needs and key performance indicators that clinicians and hospital leaders use to measure success. To succeed with healthcare innovation today, new technologies must come with new business models and outcome-based incentives, aligned across all stakeholders; with collaboration based on shared objectives and risks. Hospitals want medical device providers to be integrators and brokers of technology, helping healthcare providers to embed third-party apps and devices into their workflow via a common underlying deployment platform. Patients now play a more central role in their own care decisions. Whereas the Electronic Medical Record (EMR) has created a system of record to keep track of clinical decisions and to support billing, it does not yet fulfill the need of optimally connecting staff and patients, or enabling dynamic workflows and seamless user experiences. A system of engagement, which connects and interprets patient data from various sources to support caregivers with relevant insights at the point of care will help both care givers and patients alike with better delivery of healthcare.
Bio
David Zion is currently a Strategic Program Leader at Philips Healthcare, a health technology company dedicated to improving people’s health and well-being through meaningful innovation, aiming to improve 2.5 billion lives per year by 2030, including 400 million in underserved communities. David has spent over twenty years in Ultrasound System Research and Development, leading teams in transducer design, software and hardware engineering, and program management. While leader of the global Ultrasound R&D team, David drove the significant expansion of the product offerings in Cardiology and Radiology, as well as new Point-of-Care applications, utilizing highly integrated systems design to deliver first-to-world high performance portable Ultrasound solutions. In 2018, he founded and now leads the Systems Engineering Center of Excellence at Philips, a team dedicated to deploying best practices in all aspects of the architecture, modelling, design, integration, and life-cycle management of a diverse set of products and solutions delivered to customers across the globe. Prior to working in the Philips Ultrasound business, he has worked in the Automated Test, Defense Electronics, and Telecommunications industries, delivering multiple products into production with specific focus on design-for-test and design-for-service methodologies. He holds a Bachelor’s Degree in Electrical Engineering and Computer Science from MIT and a Master’s Degree in Engineering Economic Systems from Stanford University.

More information on all SERC Talks can be found here. Thank you!

SERC TALKS: “How Can a Systems Approach Help Critical Civil Infrastructure Become Smarter, More Sustainable and Resilient?”

DOWNLOAD SLIDES

Speaker: Michael Salvato, Vice President, Infrastructure Advisory Practices, Mott MacDonald | CONTACT

Abstract
Climate change, NetZero energy, and the Fourth Industrial Revolution are all game changers for infrastructure providers. Inadequate and ill-prepared infrastructure will increase the consequences of rapid urbanization, extreme weather events and digital disruption, driving up the costs to individuals, businesses, and society, reducing economic productivity and undermining the quality of life for people and plant. To build smarter, more sustainable, and resilient infrastructure, cities will need to reimage infrastructure services they provide, and arrange deeply interconnected technological, social and environmental systems to do so. Infrastructure 4.0 is comprised, not just of physical assets and digital twins, but an interconnected web of social, institutional, and ecological systems. New, complex forms of socio-technological systems are emerging that require a synthesis across traditional disciplines of engineering, information technology, environmental science, and policy. Leaders in smart, sustainable cities are embracing information and communication technologies and other means to meet the needs of populations without compromising future generations, envisioning new possibilities, and developing transformational roadmaps for a smarter, more sustainable and resilient future.
Bio
Michael A. Salvato serves as Vice President of Infrastructure Advisory Services at Mott MacDonald - a global engineering, management and development consultancy guiding clients through many of the planet’s most intricate challenges. His current responsibilities include managing the Aviation Digital Transformation Program for the Port Authority of New York and New Jersey. In 2018, he retired from the NY MTA as the Director and Program Executive for Enterprise Information and Asset Management with over 35 years of experience in infrastructure planning, engineering, construction, asset management, program management, economics, finance, and information systems. Michael is currently doing research on smart infrastructure with New York Academy of Sciences and am a member of the Board for Smart City New York. He was an advisor for Blueprint 2025, a National Infrastructure Plan, also, a research professor of urban infrastructure systems at the NYU Tandon School of Engineering where he helped set up the Center for Urban Science Progress program on Urban Systems and Informatics, the founding Chair of the Institute of Asset Management in the US and a member of the US delegation that produced the ISO 55000 standard for asset management.

More information on all SERC Talks can be found here. Thank you!

SERC TALKS: “What Does Digital Transformation Look Like from the C-Suite?”

Speaker: Dr. William (Willy) Donaldson, Assistant Professor and Director, Biotechnology and Management Program, and Director of the Luter Business Institute at Christopher Newport University | CONTACT

Abstract
Digital transformation is hard enough without systematic resistance from the very enterprise you are trying to transform, but transformation means change and change is hard. This talk addresses some of the fundamental processes and perspectives embraced in the C-Suite (CEO, COO, CinC, Exec. Director, etc. level) and how these fundamental processes, while systematic, may not be systemic. Digital transformation requires addressing and changing some of these strongly held perspectives and addressing an enterprise culture developed over time that seeks to sustain these non-systemic dynamics.
Bio
Dr. William (Willy) Donaldson is Assistant Professor in Management at Christopher Newport University. Dr. Donaldson is the Director of the CNU Luter Business Institute and Director of the Biotechnology and Management Program. Willy has over 35 years of experience as a board member and President, has been CEO of 8 companies, and helped start dozens of companies. He has over 30 years of experience in higher education. Willy is the Founder and President of Strategic Venture Planning, a management consulting firm that assists boards, investors, families, and senior management teams to maximize results.

Dr. Donaldson’s areas of research interest include Enterprise Management Systems; Corporate Universities and their impact on performance; Family business issues, dynamics, and transitions; Entrepreneurship and Innovation; Systems Thinking; and Corporate Governance and Board Performance.

Dr. Donaldson authored Simple_Complexity: A Management Book for the Rest of Us: A Guide to Systems Thinking.
DOWNLOAD SLIDES

This Talk is free to the public and all are welcomed to join and participate in this open discussion. Slides will be up at least two hours prior to the Talk if not sooner. If you have any questions regarding joining this session, please contact the SERC Talks webinar coordinator, Ms. Mimi Marcus.

More information on future SERC Talks can be found here. Thank you!