Compositional Physical Reasoning of Objects and Events from Videos

Zhenfang Chen*,†, Shilong Dong*, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B. Tenenbaum, Chuang Gan

TPAMI2025

Visual and Domain Knowledge for Professional-level Graph-of-Thought Medical Reasoning

Rina Bao, Shilong Dong, Zhenfang Chen†, Sheng He, Ellen Grant, Yangming Ou†

ICML2025 (Spotlight)

Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search

Maohao Shen*, Guangtao Zeng*, Zhenting Qi*, Zhang-Wei Hong, Zhenfang Chen, Wei Lu, Gregory Wornell, Subhro Das, David Cox, Chuang Gan

ICML2025

Scene-agnostic Pose Regression for Visual Localization

Junwei Zheng, Ruiping Liu, Yufan Chen, Zhenfang Chen, Kailun Yang, Rainer Stiefelhagen

CVPR2025

Autonomous Agents from Automatic Reward Modeling and Planning

Zhenfang Chen*, Delin Chen*, Rui Sun*, Wenjun Liu*, Chuang Gan

ICLR2025

FlexAttention for Efficient High-Resolution Vision-Language Models

Junyan Li, Delin Chen, Tianle Cai, Peihao Chen, Yining Hong, Zhenfang Chen , Yikang Shen, Chuang Gan

ECCV2024

ContPhy: Continuum Physical Concept Learning and Reasoning from Videos

Zhicheng Zheng*, Xin Yan*, Zhenfang Chen*^†, Jingzhou Wang, Qin Zhi Eddie Lim, Joshua B. Tenenbaum, Chuang Gan

ICML2024

SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge

Andong Wang*, Bo Wu*, Sunli Chen, Zhenfang Chen, Haotian Guan, Wei-Ning Lee, Li Erran Li, Chuang Gan

CVPR2024

GENOME: Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules

Zhenfang Chen*, Rui Sun*, Wenjun Liu*, Yining Hong, Chuang Gan

ICLR2024

CoVLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding

Junyan Li, Delin Chen, Yining Hong, Zhenfang Chen, Peihao Chen, Yikang Shen, Chuang Gan

ICLR2024

SALMON: Self-Alignment with Principle-Following Reward Models

Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, Chuang Gan

ICLR2024

Visual Chain-of-Thought Prompting for Knowledge-based Visual Reasoning

Zhenfang Chen*, Qinhong Zhou*, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund, Chuang Gan

AAAI2024

Sparse Universal Transformer

Shawn Tan*, Yikang Shen*, Zhenfang Chen, Aaron Courville, Chuang Gan

EMNLP2023

Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties

Hsiao-Yu Tung*, Mingyu Ding*, Zhenfang Chen, Daniel M. Bear, Chuang Gan, Joshua B. Tenenbaum, Daniel L. K. Yamins, Judith Fan, Kevin A. Smith

NeurIPS2023 (Datasets and Benchmarks Track)

Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan

NeurIPS2023 (Spotlight)

3D-LLM: Injecting the 3D World into Large Language Models

Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan

NeurIPS2023 (Spotlight)

TextPSG: Panoptic Scene Graph Generation from Textual Descriptions

Chengyang Zhao, Yikang Shen, Zhenfang Chen, Mingyu Ding, Chuang Gan

ICCV2023

Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners

Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik Learned-Miller, Chuang Gan

CVPR2023

Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention

Mingyu Ding, Yikang Shen, Lijie Fan, Zhenfang Chen, Zitian Chen, Ping Luo, Joshua B. Tenenbaum, Chuang Gan

CVPR2023

3D Concept Learning and Reasoning from Multi-View Images

Yining Hong, Chunru Lin, Yilun Du, Zhenfang Chen, Joshua B. Tenenbaum, Chuang Gan

CVPR2023

Planning with Large Language Models for Code Generation

Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum, and Chuang Gan

ICLR2023

Deep Face Video Inpainting via UV Mapping

Wenqi Yang, Zhenfang Chen, Chaofeng Chen, Guanying Chen, Kwan-Yee K. Wong

IEEE Transactions on Image Processing (TIP), 2023

S³-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

Wenqi Yang, Guanying Chen, Chaofeng Chen, Zhenfang Chen, Kwan-Yee K. Wong

NeurIPS2022

Embodied Concept Learner: Self-supervised Learning of Concepts and Mapping through Instruction Following

Mingyu Ding, Yan Xu, Zhenfang Chen, David Daniel Cox, Ping Luo, Joshua B. Tenenbaum, Chuang Gan

CoRL2022

PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo

Wenqi Yang, Guanying Chen, Chaofeng Chen, Zhenfang Chen, Kwan-Yee K. Wong

ECCV2022

A Unified Framework for Masked and Mask-Free Face Recognition via Feature Rectification

Shaozhe Hao, Chaofeng Chen, Zhenfang Chen, Kwan-Yee K. Wong

ICIP2022

ComPhy: Compositional Physical Reasoning of Objects and Events from Videos

Zhenfang Chen, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B. Tenenbaum, Chuang Gan

ICLR2022

Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language

Mingyu Ding, Zhenfang Chen, Tao Du, Ping Luo, Joshua B. Tenenbaum, Chuang Gan

NeurIPS2021

STAR: A Benchmark for Situated Reasoning in Real-World Videos

Bo Wu, Shoubin Yu, Zhenfang Chen , Joshua B. Tenenbaum, Chuang Gan

NeurIPS2021 (Datasets and Benchmarks Track)

The Blessings of Unlabeled Background in Untrimmed Videos

Yuan Liu, Jingyuan Chen, Zhenfang Chen, Bing Deng, Jianqiang Huang, Hanwang Zhang

CVPR2021

Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning

Zhenfang Chen, Jiayuan Mao, Jiajun Wu, Kwan-Yee K. Wong, Joshua B. Tenenbaum, Chuang Gan

ICLR2021

Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehension

Zhenfang Chen, Peng Wang, Lin Ma, Kwan-Yee K. Wong, Qi Wu

CVPR2020

Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Video

Zhenfang Chen, Lin Ma, Wenhan Luo, Kwan-Yee K. Wong

ACL2019 (Oral Presentation)

Learning Local Similarity with Spatial Relations for Object Retrieval

Zhenfang Chen, Zhanghui Kuang, Wayne Zhang, Kwan-Yee K. Wong

MM2019

Boosting up scene text detectors with guided CNN.

Xiaoyu Yue, Zhanghui Kuang, Zhaoyang Zhang, Zhenfang Chen, Pan He, Yu Qiao and Wayne Zhang

BMVC2018 (Oral Presentation)

Aggregated deep feature from activation clusters for particular object retrieval.

Zhenfang Chen, Zhanghui Kuang, Kwan-Yee K. Wong, Wayne Zhang

MM17 (Thematic Workshop)

Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble

Shun Zhang, Zhenfang Chen, Sunli Chen, Yikang Shen, Zhiqing Sun, and Chuang Gan

Arxiv

ModuleFormer: Learning Modular Large Language Models From Uncurated Data

Yikang Shen, Zheyu Zhang, Tianyou Cao, Shawn Tan, Zhenfang Chen, Chuang Gan

Arxiv

Look Closer to Ground Better: Weakly-Supervised Temporal Grounding of Sentence in Video

Zhenfang Chen, Lin Ma, Wenhan Luo, Peng Tang, Kwan-Yee K. Wong

Arxiv.

Deep learning for visual retrieval, visual grounding and visual reasoning

Zhenfang Chen

Dept. of Computer Science, The University of Hong Kong, 2021