Research

Our research is generously funded by:

Israel Science Foundation (ISF) 2017-2021: grants 1627/17, 1629/17
ABC Robotics Initiative, BGU: 2018-2021

Imitation learning for motor task transfer (jointly with Jan Peters)

A versatile method to transfer a motor task from one agent (the expert) to another (the learner) is by imitation. Most of the imitation learning algorithms are based on behavioral cloning (BC) or inverse reinforcement learning (IRL). We are particularly interested in the correspondence problem in imitation learning, which results in the question of how can one agent (the learner) produce a similar behavior it perceives in another agent (the expert) given that the agents have different embodiments (body morphology, degrees of freedom, constraints, joints and actuators, torque limits). We are investigating imitation learning algorithms that explicitly address the correspondence problem.

Reinforcement learning for real-world applications 

Reinforcement learning has led to impressive results for sequential-decision making tasks in simulated environments (Atari, MuJoCo, AlphaGo). The successful application of reinforcement learning to real world application is still challenging due to sample complexity, the exploration-exploitation dilemma as well as generalization and reliability demands. We are investigating the requirements for real RL deployment by studying different real systems beyond the classical robotic setting.

Mental rehearsals and simulation-to-reality gap (jointly with Aharon Bar-Hillel)

Robot skill learning is time-consuming and costly. Methods to make robots learn more efficiently have huge implications, in particular, for robotic industrial applications. We are investigating methods to train the robot exclusively in simulation, which can be done much faster and – in principle – with an unlimited amount of data (mental rehearsals). To transfer the learned task to the real robot, differences in perception and control of the physical environment versus the simulated one need to be bridged. Towards this goal we are studying various deep learning methods to overcome this simulation-to-reality gap.

Reward shaping vs sparse rewards in deep reinforcement learning

Reinforcement learning is based on the reward hypothesis, stating that all goals can be described by the maximization of expected cumulative reward. To engineer a suitable reward function for a given robotic task is challenging and often requires a lot of hand-tuning and adjustments of weights and hyperparameters. Binary reward signals, which indicate successful task completion, are easy to implement, but lead to slower learning rates or often fail to converge. Recent developments, such as Hindsight Experience Replay (HER), provide a promising avenue to use sparse rewards for complex motor tasks. We are exploring various generalizations to this approach.

Human-like and human-inspired control of motor tasks using model-based and model-free reinforcement learning (jointly with Avinoam Borowsky and Tamar Flash)

Robots that show human-like behavior are important for many environments in which humans and robots interact. For example, consider a traffic situation of the future, in which a mixture of self-driving and human-controlled vehicles are sharing roads. To enable smooth traffic flow, autonomous vehicles must apply human-like driving policies and negotiation skills when overtaking, giving way, merging, or taking left and right turns. We are investigating deep reinforcement learning methods to develop agents that show human-like behavior using model-based and model-free approaches. In human-inspired control we are studying whole-body motor task of humanoid robots by using insights from human motor control and model-free reinforcement learning.

Ultrasensitive quantum sensor development for application to intelligent systems (jointly with Fedor Jelezko)

Quantum technology and artificial intelligence are at the focus of 21st century science. We are exploring quantum sensing technologies for applications in intelligent systems. In particular, we are interested in nitrogen-vacancy (NV) centers in diamond, which are inhomogeneities in the cyrstal structure of the diamond that define unique ‘atom-like’ systems with quantum properties including sharp optical and microwave transitions, Zeeman sublevels and the aibility to undergo optical pumping. The spin states of the NV center, which can be read out optically using electron spin resonance, can interact with magnetic, electric, strain and temperature fields, thus, providing the underlying principle for an NV center based imaging device with unprecedented spatiotemporal resolution.