CIAO (Complexity Intelligence Autonomy Openness) – 4 horsemen of Untrustworthy AI and ways to address them

Abstract: Emerging cooperative intelligent systems such as cooperative robots and autonomous vehicles pose unprecedent challenges for safety. These systems are often loosely connected enabling them to form and dissolve configurations dynamically. Assessing the effects of failures of all possible configurations is an intractable problem. Intentional or inadvertent interactions between systems, as well as learned new behaviour and environmental factors, lead to unpredictable or emergent behaviour. Systems exhibit many sources of uncertainty including partial observability, unreliable sensors and data collection components, stochastic effects and imperfect inferences drawn by AI components. There is typically no hierarchy within a cooperative system, which is often a collective of systems that do not impose authority, and hence no hierarchy of goals can easily be assumed in the assessment. These characteristics suggest that full assurance of safety at design time using traditional model-based techniques is impossible. And, therefore, both new techniques are needed, and part of the assurance must be moved during the operational phase. Many scientists forecast huge technological, economic, and even existential risks as AI advances this century, a potential apocalypse for humanity. In the Book of Revelation by St John, the four horsemen of the apocalypse appear with the opening of the first four of the seven seals that bring forth the cataclysmic events. Using this metaphor, I discuss four corresponding difficult challenges for trustworthy AI – Complexity, Intelligence, Autonomy and Openness (CIAO), and outline relevant research on executable and adaptive models that attempts to address these challenges