Yapeng Tian

Assistant Professor, University of Texas at Dallas

yapeng.tian@utdallas.edu

ECSS 4.211

Bio

I am an assistant professor in the Computer Science Department of UT Dallas. I am interested in solving core computer vision and computer audition problems and applying the developed learning approaches to broad AI applications. My recent work has focused on studying audio-visual scene understanding [ECCV'18, CVPRW'19, ECCV'20, CVPR'21a, CVPR'21b, CVPR'22a] and mitigating video motions [CVPR'20a, CVPR'20b].

Before coming to UTD, I finished my PhD at University of Rochester, advised by Chenliang Xu, my master degree at Tsinghua University working with Wenming Yang, and B.E degree at Xidian University. I was a visiting student at SIAT advised by Yu Qiao. I did internships at Adobe Research with Dingzeyu Li and Meta with Alexander Richard.

Prospective students:
 My group has multiple opening Ph.D. positions in Spring and Fall 2023. Please email me your CV if you are interested. For application, please apply to our CS Ph.D. program and mention my name in your research statement. Current UTD students interested in doing research with me are welcome to email me or stop by my office during my office hours.

News

  • 07/2022: I will serve as a Senior Program Committee (SPC) Member for AAAI 2023.
  • 07/2022: One paper accepted at ECCV 2022.
  • 06/2022: One paper accepted at MICCAI 2022.
  • 06/2022: Successfully defended my dissertation! Thanks to everyone who supported me and helped me along the way.
  • 04/2022: I will attend CVPR'22 Doctoral Consortium.
  • 03/2022: Two works: audio-visual question answering and MRI SR are accepted by CVPR 2022.
  • 12/2021: Two papers are accepted by AAAI 2022.
  • 10/2021: One paper on sounding object localization is accepted by BMVC 2021!
  • 07/2021: One paper on video matting is accepted by ICCV 2021!
  • 03/2021: Our two works: co-learn sounding object visual grounding and sound separation and audio-visual robustness are accepted by CVPR 2021!
  • 02/2021: We will co-organize a CVPR 2021 Tutorial on Audio-visual Scene Understanding!
  • 01/2021: Co-organized the WACV 2021 Tutorial on Audio-visual Scene Understanding. More details can be found in our website.
  • 10/2020: I was in the top 10% of high-scoring reviewers for NeurIPS 2020!
  • 07/2020: Our audio-visual video parsing work got accepted by ECCV 2020 as a Spotlight.
  • 05/2020: Our three papers will be presented in the CVPR 2020 Sight and Sound workshop.
  • 02/2020: Two papers on video restoration got accepted by CVPR 2020! Congratulations to all co-authors!
  • 01/2020: RDN is accepted by IEEE TPAMI! Congratulations to Yulun!
  • 12/2019: Please check our deep audio prior paper.
  • 08/2019: One paper is accepted by IEEE TIP. Congratulations to Xuechen!
  • 07/2019: One paper is accepted by ICCV 2019. Congratulations to Wei!
  • 05/2019: Our two works: audio-visual event localization and audio-visual video captioning will be presented in the CVPR 2019 Sight and Sound workshop.
  • 02/2019: I will serve as an ICCV 2019 reviewer.
  • 12/2018: Two papers are posted on ArXiv. Please watch the corresponding demos.
  • 07/2018: One paper is accepted by ECCV 2018! AVE dataset and codes have been released.
  • 02/2018: One paper is accepted by CVPR 2018. Congratulations to Yulun!
  • 07/2017: I recieve 'Outstanding Graduate of Tsinghua university' and 'Outstanding Master Thesis Award'.
  • 03/2017: I will join Prof. Chenliang Xu's lab to pursue a PhD degree at University of Rochester!

Students

(Co-)advised Students:
Shentong Mo (PhD student at Carnegie Mellon University)
Guangyao Li (PhD student at Renmin University of China)
Shijian Deng (Graduate student at University of Rochester)
Yuxin Ye (Graduate student at Tsinghua University)

Alumni:
Hai Wang (Graduate student at Tsinghua University; next: PhD student at UCL)
Sizhe Li (Undergraduate student at University of Rochester; next: Visiting student at MIT )
Yiyang Su (Undergraduate student at University of Rochester; next: PhD student at Michigan State University)
Rohan Sharma (Graduate student at University of Rochester; next: PhD student at SUNY Buffalo)
Chenxiao Guan (Undergraduate student at University of Rochester; next: Graduate student at CMU)

Publications

Most recent publications on Google Scholar.
‡ indicates equal contribution.

  • All
  • Selected
  • Vision+Sound
  • Video Restoration
  • Image Restoration

Learning Spatio-Temporal Downsampling for Effective Video Upscaling

Xiaoyu Xiang, Yapeng Tian, Vijay Rengarajan, Lucas Young, Bo Zhu, Rakesh Ranjan

ECCV'22: European Conference on Computer Vision.

Paper

DuDoCAF: Dual-Domain Cross-Attention Fusion with Recurrent Transformer for Fast Multi-contrast MR Imaging

Jun Lyu, Bin Sui, Chengyan Wang, Yapeng Tian, Qi Dou, and Jing Qin

MICCAI'22: Medical Image Computing and Computer Assisted Intervention.

Paper

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li‡, Yake Wei‡, Yapeng Tian‡, Chenliang Xu, Ji-Rong Wen, and Di Hu

CVPR'22 Oral: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code Project

Transformer-empowered Multi-contrast MRI Super-Resolution

Guangyuan Li, Jun Lv, Yapeng Tian, Qi Dou, Chengyan Wang, Chenliang Xu, Jing Qin

CVPR'22: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

Bin Xia, Yapeng Tian, Yucheng Hang, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Efficient Non-Local Contrastive Attention for Image Super-Resolution

Bin Xia‡, Yucheng Hang‡, Yapeng Tian, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Space-Time Memory Network for Sounding Object Localization in Videos

Sizhe Li‡, Yapeng Tian‡, and Chenliang Xu

BMVC'21: The British Machine Vision Conference.

Paper Code Project

Video Matting via Consistency-Regularized Graph Neural Networks

Tiantian Wang, Sifei Liu, Yapeng Tian, Kai Li, and Ming-Hsuan Yang

ICCV'21: IEEE/CVF International Conference on Computer Vision.

Paper Code

Can audio-visual integration strengthen robustness under multimodal attacks?

Yapeng Tian and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

Yapeng Tian, Di Hu, and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu

ECCV'20 Spotlight: European Conference on Computer Vision.

Paper Code Data

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiaoyu Xiang‡, Yapeng Tian‡, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

This is the first work that uses deformable alignment to address video restoration.

Paper Video Code

Deep Audio Prior

Yapeng Tian, Chenliang Xu, and Dingzeyu Li

CVPRW'20: CVPR Workshops.

Paper Code Project

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

TPAMI'20: IEEE Transactions on Pattern Analysis and Machine Intelligence.

Paper Code

CFSNet: Toward a Controllable Feature Space for Image Restoration

Wei Wang‡, Ruiming Guo‡, Yapeng Tian, and Wenming Yang

ICCV'19: IEEE/CVF International Conference on Computer Vision.

Paper Code

Interpretable and Controllable Audio-Visual Video Captioning

Yapeng Tian, Chenxiao Guan, Goodman Justin, Marc Moore, and Chenliang Xu

CVPRW'19: CVPR Workshops.

Multisensory interpretability in terms of the audio-visual video captioning task.

Paper

LCSCNet: Linear Compressing Based Skip-Connecting Network for ISR

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, Qingmin Liao

TIP'19: IEEE Trans. Image Processing.

Paper

Deep Learning for Single Image Super-Resolution: A Brief Review

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, JingHao Xue, Qingmin Liao

TMM'19: IEEE Trans. Multimedia.

Paper

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

ECCV'18: European Conference on Computer Vision.

Paper Video Code Data Project

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

CVPR'18 Spotlight: IEEE/CVF Conf. on Computer Vision and Pattern Recognition.

Paper Code

NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results

Timofte et al.

CVPRW'17: CVPR Workshops.

Paper

Consistent Coding Scheme for Single-Image Super-Resolution

Wenming Yang, Yapeng Tian, Fei Zhou, Qingmin Liao, Hai Chen, Chenglin Zheng

TMM'16: EEE Trans. Multimedia. (First student author)

Paper

Anchored Neighborhood Regression based SISR from Self-examples

Yapeng Tian, Fei Zhou, Wenming Yang, Xuesen Shang, Qingmin Liao

ICIP'16: IEEE International Conference on Image Processing.

Paper Code

SISR Using Clustering-Based Global Regression and Propagation Filtering

Wenming Yang, Yapeng Tian, Fei Zhou, ..., Qingmin Liao

ACPR'15 Oral: Asian Conference on Pattern Recognition. (First student author)

Paper

Learning Spatio-Temporal Downsampling for Effective Video Upscaling

Xiaoyu Xiang, Yapeng Tian, Vijay Rengarajan, Lucas Young, Bo Zhu, Rakesh Ranjan

ECCV'22: European Conference on Computer Vision.

Paper

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li‡, Yake Wei‡, Yapeng Tian‡, Chenliang Xu, Ji-Rong Wen, and Di Hu

CVPR'22 Oral: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code Project

Can audio-visual integration strengthen robustness under multimodal attacks?

Yapeng Tian and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

Yapeng Tian, Di Hu, and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu

ECCV'20 Spotlight: European Conference on Computer Vision.

Paper Code Data

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiaoyu Xiang‡, Yapeng Tian‡, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

This is the first work that uses deformable alignment to address video restoration.

Paper Video Code

Deep Audio Prior

Yapeng Tian, Chenliang Xu, and Dingzeyu Li

CVPRW'20: CVPR Workshops.

Paper Code Project

Interpretable and Controllable Audio-Visual Video Captioning

Yapeng Tian, Chenxiao Guan, Goodman Justin, Marc Moore, and Chenliang Xu

CVPRW'19: CVPR Workshops.

Multisensory interpretability in terms of the audio-visual video captioning task.

Paper

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

ECCV'18: European Conference on Computer Vision.

Paper Video Code Data Project

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li‡, Yake Wei‡, Yapeng Tian‡, Chenliang Xu, Ji-Rong Wen, and Di Hu

CVPR'22 Oral: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code Project

Space-Time Memory Network for Sounding Object Localization in Videos

Sizhe Li‡, Yapeng Tian‡, and Chenliang Xu

BMVC'21: The British Machine Vision Conference.

Paper Code Project

Can audio-visual integration strengthen robustness under multimodal attacks?

Yapeng Tian and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

Yapeng Tian, Di Hu, and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu

ECCV'20 Spotlight: European Conference on Computer Vision.

Paper Code Data

Deep Audio Prior

Yapeng Tian, Chenliang Xu, and Dingzeyu Li

CVPRW'20: CVPR Workshops.

Paper Code Project

Interpretable and Controllable Audio-Visual Video Captioning

Yapeng Tian, Chenxiao Guan, Goodman Justin, Marc Moore, and Chenliang Xu

CVPRW'19: CVPR Workshops.

Multisensory interpretability in terms of the audio-visual video captioning task.

Paper

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

ECCV'18: European Conference on Computer Vision.

Paper Video Code Data Project

Learning Spatio-Temporal Downsampling for Effective Video Upscaling

Xiaoyu Xiang, Yapeng Tian, Vijay Rengarajan, Lucas Young, Bo Zhu, Rakesh Ranjan

ECCV'22: European Conference on Computer Vision.

Paper

Video Matting via Consistency-Regularized Graph Neural Networks

Tiantian Wang, Sifei Liu, Yapeng Tian, Kai Li, and Ming-Hsuan Yang

ICCV'21: IEEE/CVF International Conference on Computer Vision.

Paper Code

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiaoyu Xiang‡, Yapeng Tian‡, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

This is the first work that uses deformable alignment to address video restoration.

Paper Video Code

DuDoCAF: Dual-Domain Cross-Attention Fusion with Recurrent Transformer for Fast Multi-contrast MR Imaging

Jun Lyu, Bin Sui, Chengyan Wang, Yapeng Tian, Qi Dou, and Jing Qin

MICCAI'22: Medical Image Computing and Computer Assisted Intervention.

Paper

Transformer-empowered Multi-contrast MRI Super-Resolution

Guangyuan Li, Jun Lv, Yapeng Tian, Qi Dou, Chengyan Wang, Chenliang Xu, Jing Qin

CVPR'22: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

Bin Xia, Yapeng Tian, Yucheng Hang, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Efficient Non-Local Contrastive Attention for Image Super-Resolution

Bin Xia‡, Yucheng Hang‡, Yapeng Tian, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

TPAMI'20: IEEE Transactions on Pattern Analysis and Machine Intelligence.

Paper Code

CFSNet: Toward a Controllable Feature Space for Image Restoration

Wei Wang‡, Ruiming Guo‡, Yapeng Tian, and Wenming Yang

ICCV'19: IEEE/CVF International Conference on Computer Vision.

Paper Code

LCSCNet: Linear Compressing Based Skip-Connecting Network for ISR

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, Qingmin Liao

TIP'19: IEEE Trans. Image Processing.

Paper

Deep Learning for Single Image Super-Resolution: A Brief Review

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, JingHao Xue, Qingmin Liao

TMM'19: IEEE Trans. Multimedia.

Paper

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

CVPR'18 Spotlight: IEEE/CVF Conf. on Computer Vision and Pattern Recognition.

Paper Code

NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results

Timofte et al.

CVPRW'17: CVPR Workshops.

Paper

Consistent Coding Scheme for Single-Image Super-Resolution

Wenming Yang, Yapeng Tian, Fei Zhou, Qingmin Liao, Hai Chen, Chenglin Zheng

TMM'16: EEE Trans. Multimedia. (First student author)

Paper

Anchored Neighborhood Regression based SISR from Self-examples

Yapeng Tian, Fei Zhou, Wenming Yang, Xuesen Shang, Qingmin Liao

ICIP'16: IEEE International Conference on Image Processing.

Paper Code

SISR Using Clustering-Based Global Regression and Propagation Filtering

Wenming Yang, Yapeng Tian, Fei Zhou, ..., Qingmin Liao

ACPR'15 Oral: Asian Conference on Pattern Recognition. (First student author)

Paper

Teaching

  • Fall 2022 - CS 6334: Virtual Reality (UT Dallas)

Service

Organizer:

Senior Program Committee or Area Chair:

  • AAAI: AAAI Conference on Artificial Intelligence, 2023

Conference Program Committee/Reviewer:

  • CVPR: IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • ICCV: IEEE/CVF International Conference on Computer Vision
  • ECCV: European Conference on Computer Vision
  • NeurIPS: Conference on Neural Information Processing Systems
  • ICLR: International Conference on Learning Representations
  • AAAI: AAAI Conference on Artificial Intelligence
  • ICML: International Conference on Machine Learning
  • WACV: Winter Conference on Applications of Computer Vision
  • ACCV: Asian Conference on Computer Vision

Journal Reviewer:

  • TPAMI: IEEE Transactions on Pattern Analysis and Machine Intelligence
  • TMLR: The Transactions on Machine Learning Research
  • TIP: IEEE Transactions on Image Processing
  • TNNLS: IEEE Transactions on Neural Networks and Learning Systems
  • TMM: IEEE Transactions on Multimedia
  • TCSVT: IEEE Transcations on Circuits and Systems for Video Technology
  • TASLP: IEEE/ACM Transactions on Audio, Speech and Language Processing
  • Scientific Reports–Nature
  • CGF: Computer Graphics Forum
  • CVIU: Computer Vision and Image Understanding
  • SPIC: Signal Processing: Image Communication
  • IEEE Access

Talks and Seminars:

  • Audio-Visual Scene Understanding Towards Unified, Explainable, and Robust Multisensory Perception

    KTH Dive-Deep Seminar, Dec. 2021
         RIT PhD Colloquium Series, Oct. 2021

  • Audio-Visual Video Understanding, IIAI Seminar, Sep. 2021
  • The Future of Audio-Visual Research Panel Discussion, VALSE Webinar, Nov. 2021

Awards

CVPR Doctoral Consortium, 2022
Top 10% of High-Scoring Reviewers for NeurIPS, 2020
Invited attendee of Amazon Graduate Student Symposium, Seattle, USA, 2019
Outstanding Graduate of Tsinghua University, 2017
Outstanding Master Thesis Award, Tsinghua University, 2017
National Scholarship, Tsinghua University, 2016
Second-class Scholarship, Tsinghua University, 2015

Vitæ

Full CV in PDF.

  • University of Texas at Dallas 2022 - now
    Assistant Professor
    Department of Computer Science
  • University of Rochester 2017 - 2022
    Ph.D. Student
    Department of Computer Science
  • Meta Sep. 2021 - Jan. 2022
    Research Intern
    Reality Labs
  • Adobe Summer 2021
    Research Intern
    Creative Intelligence Lab
  • Adobe Summer 2019
    Research Intern
    Creative Intelligence Lab
  • Tsinghua University 2014-2017
    M.E. Student
    Department of Electronic Engineering
  • Chinese Academy of Sciences Nov. 2016- May 2017
    Visiting Student
    Shenzhen Institutes of Advanced Technology
  • Xidian University 2009 - 2013
    B.E. Student
    School of Electronic Engineering

This website was built with jekyll based on a template from Martin Saveski.

Yapeng Tian
yapeng.tian@utdallas.edu