קולוקוויום בית הספר למדעי המחשב

"Reinforcement Learning in the presence of irrelevant information"

17 בדצמבר 2022, 11:00

Speaker: Yonathan Efroni (Meta, NYC) Room 420 \| Checkpoint Building
Abstract Reinforcement Learning (RL) is a field concerned with designing general purpose learning algorithms that solve sequential-decision tasks. In recent years, by using deep neural networks, RL algorithms were applied on high-dimensional and challenging domains, witnessing unprecedented success. Yet, despite recent advancements, the theoretical foundations of high-dimensional RL are not fully understood. A recurring theme in high-dimensional RL is the presence of irrelevant information in the observations. E.g., in a visual navigation task the observation might capture the movement of clouds, which is irrelevant for reaching the goal location. This calls for natural questions: Can such tasks be learned efficiently, depending only on the complexity of the relevant information? Can RL algorithms be robust to noise in observations? Surprisingly, contemporary RL algorithms may provably fail in the presence of irrelevant information. In this talk, I will elaborate on these failure cases and present our new provable approaches for high-dimensional RL with irrelevant information. Shared to these are techniques to filter the irrelevant information while guaranteeing near-optimal behavior. I will conclude with experimental results showcasing challenges and solutions in practice. Bio: Yonathan is a research scientist at Meta. Prior to that he completed his post-doctorate in Microsoft Research, New York. He obtained his PhD from the Technion, advised by Prof. Shie Mannor, and his Master from the Weizmann institute in Physics. His work won the outstanding paper award in AAAI19 and a best paper award in the OptRL workshop in NeurIPS19.

Speaker: Yonathan Efroni (Meta, NYC)

Room 420 | Checkpoint Building

Abstract

Reinforcement Learning (RL) is a field concerned with designing general purpose learning algorithms that solve sequential-decision tasks. In recent years, by using deep neural networks, RL algorithms were applied on high-dimensional and challenging domains, witnessing unprecedented success. Yet, despite recent advancements, the theoretical foundations of high-dimensional RL are not fully understood.
A recurring theme in high-dimensional RL is the presence of irrelevant information in the observations. E.g., in a visual navigation task the observation might capture the movement of clouds, which is irrelevant for reaching the goal location. This calls for natural questions: Can such tasks be learned efficiently, depending only on the complexity of the relevant information? Can RL algorithms be robust to noise in observations? Surprisingly, contemporary RL algorithms may provably fail in the presence of irrelevant information.
In this talk, I will elaborate on these failure cases and present our new provable approaches for high-dimensional RL with irrelevant information. Shared to these are techniques to filter the irrelevant information while guaranteeing near-optimal behavior. I will conclude with experimental results showcasing challenges and solutions in practice.

Bio:
Yonathan is a research scientist at Meta. Prior to that he completed his post-doctorate in Microsoft Research, New York. He obtained his PhD from the Technion, advised by Prof. Shie Mannor, and his Master from the Weizmann institute in Physics. His work won the outstanding paper award in AAAI19 and a best paper award in the OptRL workshop in NeurIPS19.