Research

I am interested in the trio of action, perception, and cognition. More particularly, I have been hoping to build machines that perceives to recognize the real world, understands the (physical) outcomes of their actions, explores novel complex behaviors, and learns/plans to achieve goals. I am fascinated by the animal beings that generalize and explore so successfully (see Moravec's Paradox).

Prospective students

I am looking for Ph.D/master/undergraduate students interested in robot learning, computer vision and machine learning. If you are a undergraduate at NTU, interested in research positions, please fill in this form. If you are interested in Mater/Doctoral research positions, please fill in this form.

Teaching

CSIE5117: Robot Perception and Learning Fall 2024, 2025
CSIE5421: Embodied Vision Spring 2025

Publications

See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation

CoRL 2025

Chih Yao Hu*, Yang-Sen Lin*, Yuna Lee, Chih-Hai Su, Jie-Ying Lee, Shr-Ruei Tsai, Chin-Yang Lin, Kuan-Wen Chen, Tsung-Wei Ke, Yu-Lun Liu

[Arxiv](TBA) [Code](TBA) [Project]

3D Diffuser Actor: Policy Diffusion with 3D Scene Representations

CoRL 2024

Tsung-Wei Ke*, Nikolaos Gkanatsios*, and Katerina Fragkiadaki

[Arxiv] [Code] [Project]

Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving

CVPR 2024

Brian Yang, Huangyuan Su, Nikolaos Gkanatsios, Tsung-Wei Ke, Ayush Jain, Jeff Schneider, and Katerina Fragkiadaki

[Arxiv] [Code] [Project]

Learning Hierarchical Image Segmentation For Recognition and By Recognition<

ICLR 2024 (Spotlight)

Tsung-Wei Ke*, Sangwoo Mo*, and Stella Yu

[Arxiv] [Code] [Project]

Test time Adaptation with Diffusion Models

NeuRIPS 2023

Mihir Prabhudesai*, Tsung-Wei Ke*, Alexander Cong Li, Deepak Pathak and Katerina Fragkiadaki

[Arxiv] [Code] [Project]

Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers

CVPR 2022 (Oral)

Tsung-Wei Ke, Jyh-Jing Hwang, Yunhui Guo, Xudong Wang and Stella X. Yu

[Arxiv] [Code] [Project]

Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning

ICLR 2021

Tsung-Wei Ke, Jyh-Jing Hwang, and Stella X. Yu

[Arxiv] [Code] [Project]

Adversarial Structure Matching for Structured Prediction Tasks

CVPR 2019

Jyh-Jing Hwang, Tsung-Wei Ke, Jianbo Shi and Stella X. Yu

[Arxiv] [Code] [Project]

Adaptive Affinity Field for Semantic Segmentation

ECCV 2018

Tsung-Wei Ke*, Jyh-Jing Hwang*, Ziwei Liu and Stella X. Yu (*equal contribution)

[Arxiv] [Code] [Project]

Neural MultiGrid

CVPR 2017

Tsung-Wei Ke, Michael Maire, and Stella X. Yu

[Arxiv] [Code]