Safe and robust reinforcement learning (Prof. Dominik Baumann, Aalto Universität, Finnland)

Reinforcement learning (RL) has achieved remarkable success in diverse applications, yet applying it to learning control policies for physical systems remains challenging. Learning on physical systems demands sample efficiency, as data collection requires time-consuming experiments that induce wear and tear on the hardware. Furthermore, we require safety guarantees, as otherwise, the environment or expensive hardware may be damaged. Unfortunately, popular (deep) reinforcement learning methods fail to meet these requirements, as they require lots of training, and their theoretical analysis is challenging. In this talk, I will present alternative approaches based on Bayesian optimization that are sample-efficient and for which we can provide safety guarantees. In particular, I will propose an algorithm that learns globally optimal policies while providing safety guarantees, discuss the assumptions required in exchange for those guarantees, and propose a strategy to reduce the computational footprint of safe learning algorithms. Finally, I will take a step back and discuss the general optimization objective in reinforcement learning, how it may inherently favor non-robust policies, and how we can mitigate this risk.

Biographical information

Dominik Baumann is an Assistant Professor and leads the Cyber‑physical Systems Group at Aalto University, Finland. He received his PhD in 2020 through a collaboration between the Max Planck Institute for Intelligent Systems (Stuttgart/Tübingen) and KTH Royal Institute of Technology (Stockholm). After his PhD, he was a postdoctoral researcher at RWTH Aachen University and Uppsala University. Dominik chairs the Finland Section joint Chapter of the IEEE Control Systems, Robotics and Automation, and Systems, Man, and Cybernetics Societies. He is a member of the European Laboratory for Learning and Intelligent Systems (ELLIS) and a PI in the Finnish Center for Artificial Intelligence (FCAI). His research focuses on learning and control for networked multi‑agent systems. More info at https://baumanndominik.github.io/.