Roei Herzig

Hi there! I am Roei, a CS Ph.D. student at Tel Aviv University and a Berkeley AI Research Lab member working with Prof. Amir Globerson and Prof. Trevor Darrell. I'm also affiliated as a research scientist at IBM Research AI.

Previously, I graduated magna cum laude from Tel Aviv University with MSc (CS), BSc (CS), and BSc (Physics), and worked as a Machine Learning & Deep Learning researcher at Nexar and Trax Image Recognition for 5 years.

I'm looking for strong Master's and senior undergrads to collaborate and publish in top-tier conferences on Video Understanding and Compositionality.

Email  /  Twitter  /  Github  /  LinkedIn  /  CV  /  Google Scholar

My research goal is to develop compositional models that capture semantic understanding for multiple domains, such as vision, language, and robotics. I believe our understanding of the world is naturally hierarchical and structured, and intelligent machines would need to develop a compositional understanding that is robust and generalizable. However, many existing vision architectures are not compositional, and thus my research goal is to design compositional models that leverage inductive biases into our architectures to generalize well across various tasks.

Research Interest:

• Machine Learning & Deep Learning: Generative Models, Graph Neural Networks, Self-Supervised Learning.
• Vision & Language: Video Understanding, Scene Understanding, Visual Reasoning.
• Vision & Robotics: Semantic Understanding, Object-Centric Representation, Structured Representation.

I'm a proud father of Adam and Liam and happily married to Esti, my amazing wife and my partner for life. When I'm not working, I'm also a history buff and love learning about science, politics, music, and the two World Wars.

Object-Region Video Transformers

We present ORViT, an object-centric approach that extends video transformer layers with a block that directly incorporates object representations.

Compositional Video Synthesis with Action Graphs

We introduce the formalism of Action Graphs, a natural and convenient structure representing the dynamics of actions between objects over time. We show we can synthesize goal-oriented videos on the CATER and Something Something datasets and generate novel compositions of unseen actions.

We present a novel model that can inherently learn canonical graph representations and show better robustness to graph size, adversarial attacks, and semantic equivalent, thus generating superior images of complex visual scenes.

We propose a novel compositional action recognition task where the training combinations of verbs and nouns do not overlap with the test set. We show the effectiveness of our approach on the proposed compositional task and a few-shot compositional setting which requires the model to generalize across both object appearance and action category.

We propose an intermediate “graph-like” representation (DSGs) that can be learned in an end-to-end manner from the supervision for a downstream visual reasoning task, which achieves a new state-of-the-art results on Referring Relationships task.

We propose a latent inter-object graph representation for activity recognition that explores the visual interaction between the objects in a self-supervised manner.

We propose a hybrid coarse-to-fine approach that leverages visual and GPS location cues with on a new large-scale driving dataset based on video and GPS data.

We collect a new SKU-110K dataset which takes detection challenges to unexplored territories, and propose a novel mechanism to learn deep overlap rates for each detection.

We propose a novel invariant graph network for mapping images to scene graphs using the permutation invariant property, which achieves a new state-of-the-art results on Visual Genome dataset.

Template: Jon Barron, Last update: 10/2021.