Kuo-Hao Zeng 曾國豪
Research Scientist, Allen Institute for AI (Ai2)
khzeng at allenai.org
I am a research scientist at the Allen Institute for AI (Ai2). My current research interests are in large-scale policy training for embodied agents, leveraging the capabilities of foundational vision models and multimodal language models.
I received my Ph.D. in the Computer Science & Engineering from the University of Washington, advised by Ali Farhadi and Roozbeh Mottaghi in RAIVN Lab.
My CV [PDF], last updated Oct 2024.
Selected Publications
* equal contribution; † equal advising
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
Matt Deitke*, Christopher Clark*, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Borchardt, Dirk Groeneveld, Jen Dumas, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, Aniruddha Kembhavi
arXiv 2024
FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuning
Jiaheng Hu, Rose Hendrix, Ali Farhadi, Aniruddha Kembhavi, Roberto Martín-Martín, Peter Stone, Kuo-Hao Zeng†, Kiana Ehsani†
arXiv 2024
PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators
Kuo-Hao Zeng, Zichen Zhang, Kiana Ehsani, Rose Hendrix, Jordi Salvador, Alvaro Herrasti, Ross Girshick, Aniruddha Kembhavi, Luca Weihs
CoRL 2024 Oral Presentation
Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World
Kiana Ehsani*, Tanmay Gupta*, Rose Hendrix*, Jordi Salvador*, Luca Weihs*, Kuo-Hao Zeng*, Kunal Pratap Singh, Yejin Kim, Winson Han, Alvaro Herrasti, Ranjay Krishna, Dustin Schwenk, Eli VanderBilt, Aniruddha Kembhavi
CVPR 2024
Seeing the Unseen: Visual Common Sense for Semantic Placement
Ram Ramrakhya, Aniruddha Kembhavi, Dhruv Batra, Zsolt Kira, Kuo-Hao Zeng†, Luca Weihs†
CVPR 2024
Selective Visual Representations Improve Convergence and Generalization for Embodied AI
Ainaz Eftekhar*, Kuo-Hao Zeng*, Jiafei Duan, Ali Farhadi, Ani Kembhavi, Ranjay Krishna
ICLR 2024 Spotlight
Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics
Kuo-Hao Zeng, Luca Weihs, Roozbeh Mottaghi, Ali Farhadi
ICLR 2023 Oral Presentation
Pushing it out of the Way: Interactive Visual Navigation
Kuo-Hao Zeng, Luca Weihs, Ali Farhadi, Roozbeh Mottaghi
CVPR 2021
AllenAct: A Framework for Embodied AI Research
Luca Weihs, Jordi Salvador, Klemen Kotar, Unnat Jain, Kuo-Hao Zeng, Roozbeh Mottaghi, Aniruddha Kembhavi
arXiv 2020
Visual Reaction: Learning to Play Catch with Your Drone
Kuo-Hao Zeng, Roozbeh Mottaghi, Luca Weihs, Ali Farhadi
CVPR 2020
Style Example-Guided Text Generation using Generative Adversarial Transformers
Kuo-Hao Zeng, Mohammad Shoeybi, Ming-Yu Liu
arXiv 2020
Agent-Centric Risk Assessment: Accident Anticipation and Risky Region Localization
Kuo-Hao Zeng, Shih-Han Chou, Fu-Hsiang Chan, Juan Carlos Niebles, Min Sun
CVPR 2017 Spotlight
Leveraging Video Descriptions to Learn Video Question Answering
Kuo-Hao Zeng, Tseng-Hung Chen, Ching-Yao Chuang, Yuan-Hong Liao, Juan Carlos Niebles, Min Sun
AAAI 2017
