Single timescale actor critic: a small-gain analysis (Prof. Bahman Gharesifard, Department of Mathematics and Statistics, Queen`s University
Systems & Control Seminar
Abstract
We consider the used-in-practice setting of actor-critic where proportional step-sizes are used for both the actor and the critic, with only one critic update with a single sample from the stationary distribution per actor step. Using a small-gain analysis, we prove convergence to a stationary point, with a sample complexity that improves the state of the art. I will introduce all the required preliminaries; prior familiarity with reinforcement learning is not required. Time permitting, I will briefly discuss some new results related to model-free control from a reinforcement learning lens.
Biographical information
Bahman Gharesifard is a Professor with the Department of Mathematics and Statistics at Queen's University. He was a Professor with the Electrical and Computer Engineering Department at the University of California, Los Angeles from 2021 to 2024, where he was the Area Director for Signals and Systems 2023-2024. He was an Alexander von Humboldt research fellow with the Institute for Systems Theory and Automatic Control at the University of Stuttgart in 2019-2020. He held postdoctoral positions with the Department of Mechanical and Aerospace Engineering at University of California, San Diego 2009-2012 and with the Coordinated Science Laboratory at the University of Illinois at Urbana-Champaign from 2012-2013. He received the 2019 CAIMS-PIMS Early Career Award, jointly awarded by the Canadian Applied and Industrial Math Society and the Pacific Institute for the Mathematical Sciences, an Alexander von Humboldt Foundation research fellowship for experienced researchers in 2019, the SIAG/CST Best SICON Paper Prize in 2021, and the Canadian Society for Information Theory (CSIT) Best Paper Award in 2022. His research interests include systems and controls, distributed control and optimization, reinforcement learning, neural networks, social and economic networks, game theory, geometric control theory and mechanics, and applied Riemannian geometry.
Termin
16. Jul. 202516:00 - 17:00
