Yapeng Tian

PhD student, University of Rochester

yapengtian [AT] rochester.edu

Bio

I am currently a final-year PhD student in the Department of Computer Science at the University of Rochester, advised by Prof. Chenliang Xu.

My research interests center around solving core computer vision and computer audition problems and applying the developed learning approaches to broad AI applications in multisensory perception, computational photography, robotics, AR/VR, and HCI. My recent work has focused on designing unified, explainable, and robust multisensory perception systems [ECCV'18, CVPRW'19, ECCV'20, CVPR'21a, CVPR'21b, CVPR'22a] and mitigating video motions in computational photograhy [CVPR'20a, CVPR'20b].

Previously, I received my master degree in the Department of Electronic Engineering, Tsinghua University, China, in 2017 under the supervision of Prof. Wenming Yang, and B.E degree from School of Electronic Engineering, Xidian University in 2013. I was a visiting student at Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, advised by Prof. Yu Qiao in 2016-2017. I did three internships at Adobe Research and Meta Reality Labs.

I will be joining the Computer Science Department of UT Dallas as a tenure-track assistant professor this fall. I am looking for students (Spring/Fall 2023)! Please email me directly with your CV if you are interested in working with me.

News

  • 04/2022: I will attend CVPR'22 Doctoral Consortium.
  • 03/2022: Two works: audio-visual question answering and MRI SR are accepted by CVPR 2022.
  • 12/2021: Two papers are accepted by AAAI 2022.
  • 10/2021: One paper on sounding object localization is accepted by BMVC 2021!
  • 07/2021: One paper on video matting is accepted by ICCV 2021!
  • 03/2021: Our two works: co-learn sounding object visual grounding and sound separation and audio-visual robustness are accepted by CVPR 2021!
  • 02/2021: We will co-organize a CVPR 2021 Tutorial on Audio-visual Scene Understanding!
  • 01/2021: Co-organized the WACV 2021 Tutorial on Audio-visual Scene Understanding. More details can be found in our website.
  • 10/2020: I was in the top 10% of high-scoring reviewers for NeurIPS 2020!
  • 07/2020: Our audio-visual video parsing work got accepted by ECCV 2020 as a Spotlight.
  • 05/2020: Our three papers will be presented in the CVPR 2020 Sight and Sound workshop.
  • 02/2020: Two papers on video restoration got accepted by CVPR 2020! Congratulations to all co-authors!
  • 01/2020: RDN is accepted by IEEE TPAMI! Congratulations to Yulun!
  • 12/2019: Please check our deep audio prior paper.
  • 08/2019: One paper is accepted by IEEE TIP. Congratulations to Xuechen!
  • 07/2019: One paper is accepted by ICCV 2019. Congratulations to Wei!
  • 05/2019: Our two works: audio-visual event localization and audio-visual video captioning will be presented in the CVPR 2019 Sight and Sound workshop.
  • 02/2019: I will serve as an ICCV 2019 reviewer.
  • 12/2018: Two papers are posted on ArXiv. Please watch the corresponding demos.
  • 07/2018: One paper is accepted by ECCV 2018! AVE dataset and codes have been released.
  • 02/2018: One paper is accepted by CVPR 2018. Congratulations to Yulun!
  • 07/2017: I recieve 'Outstanding Graduate of Tsinghua university' and 'Outstanding Master Thesis Award'.
  • 03/2017: I will join Prof. Chenliang Xu's lab to pursue a PhD degree at University of Rochester!

Publications

Most recent publications on Google Scholar.
‡ indicates equal contribution.

  • All
  • Selected
  • Vision+Sound
  • Video Restoration
  • Image Restoration

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li‡, Yake Wei‡, Yapeng Tian‡, Chenliang Xu, Ji-Rong Wen, and Di Hu

CVPR'22 Oral: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code Project

Transformer-empowered Multi-contrast MRI Super-Resolution

Guangyuan Li, Jun Lv, Yapeng Tian, Qi Dou, Chengyan Wang, Chenliang Xu, and Jing Qin

CVPR'22: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

Bin Xia, Yapeng Tian, Yucheng Hang, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Efficient Non-Local Contrastive Attention for Image Super-Resolution

Bin Xia‡, Yucheng Hang‡, Yapeng Tian, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Space-Time Memory Network for Sounding Object Localization in Videos

Sizhe Li‡, Yapeng Tian‡, and Chenliang Xu

BMVC'21: The British Machine Vision Conference.

Paper Code Project

Video Matting via Consistency-Regularized Graph Neural Networks

Tiantian Wang, Sifei Liu, Yapeng Tian, Kai Li, and Ming-Hsuan Yang

ICCV'21: IEEE/CVF International Conference on Computer Vision.

Paper Code

Can audio-visual integration strengthen robustness under multimodal attacks?

Yapeng Tian and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

Yapeng Tian, Di Hu, and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu

ECCV'20 Spotlight: European Conference on Computer Vision.

Paper Code Data

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiaoyu Xiang‡, Yapeng Tian‡, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

This is the first work that uses deformable alignment to address video restoration.

Paper Video Code

Deep Audio Prior

Yapeng Tian, Chenliang Xu, and Dingzeyu Li

CVPRW'20: CVPR Workshops.

Paper Code Project

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

TPAMI'20: IEEE Transactions on Pattern Analysis and Machine Intelligence.

Paper Code

CFSNet: Toward a Controllable Feature Space for Image Restoration

Wei Wang‡, Ruiming Guo‡, Yapeng Tian, and Wenming Yang

ICCV'19: IEEE/CVF International Conference on Computer Vision.

Paper Code

Interpretable and Controllable Audio-Visual Video Captioning

Yapeng Tian, Chenxiao Guan, Goodman Justin, Marc Moore, and Chenliang Xu

CVPRW'19: CVPR Workshops.

Multisensory interpretability in terms of the audio-visual video captioning task.

Paper

LCSCNet: Linear Compressing Based Skip-Connecting Network for ISR

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, Qingmin Liao

TIP'19: IEEE Trans. Image Processing.

Paper

Deep Learning for Single Image Super-Resolution: A Brief Review

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, JingHao Xue, Qingmin Liao

TMM'19: IEEE Trans. Multimedia.

Paper

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

ECCV'18: European Conference on Computer Vision.

Paper Video Code Data Project

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

CVPR'18 Spotlight: IEEE/CVF Conf. on Computer Vision and Pattern Recognition.

Paper Code

NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results

Timofte et al.

CVPRW'17: CVPR Workshops.

Paper

Consistent Coding Scheme for Single-Image Super-Resolution

Wenming Yang, Yapeng Tian, Fei Zhou, Qingmin Liao, Hai Chen, Chenglin Zheng

TMM'16: EEE Trans. Multimedia. (First student author)

Paper

Anchored Neighborhood Regression based SISR from Self-examples

Yapeng Tian, Fei Zhou, Wenming Yang, Xuesen Shang, Qingmin Liao

ICIP'16: IEEE International Conference on Image Processing.

Paper Code

SISR Using Clustering-Based Global Regression and Propagation Filtering

Wenming Yang, Yapeng Tian, Fei Zhou, ..., Qingmin Liao

ACPR'15 Oral: Asian Conference on Pattern Recognition. (First student author)

Paper

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li‡, Yake Wei‡, Yapeng Tian‡, Chenliang Xu, Ji-Rong Wen, and Di Hu

CVPR'22 Oral: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code Project

Space-Time Memory Network for Sounding Object Localization in Videos

Sizhe Li‡, Yapeng Tian‡, and Chenliang Xu

BMVC'21: The British Machine Vision Conference.

Paper Code Project

Can audio-visual integration strengthen robustness under multimodal attacks?

Yapeng Tian and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

Yapeng Tian, Di Hu, and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu

ECCV'20 Spotlight: European Conference on Computer Vision.

Paper Code Data

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiaoyu Xiang‡, Yapeng Tian‡, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

This is the first work that uses deformable alignment to address video restoration.

Paper Video Code

Deep Audio Prior

Yapeng Tian, Chenliang Xu, and Dingzeyu Li

CVPRW'20: CVPR Workshops.

Paper Code Project

Interpretable and Controllable Audio-Visual Video Captioning

Yapeng Tian, Chenxiao Guan, Goodman Justin, Marc Moore, and Chenliang Xu

CVPRW'19: CVPR Workshops.

Multisensory interpretability in terms of the audio-visual video captioning task.

Paper

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

ECCV'18: European Conference on Computer Vision.

Paper Video Code Data Project

Learning to Answer Questions in Dynamic Audio-Visual Scenarios

Guangyao Li‡, Yake Wei‡, Yapeng Tian‡, Chenliang Xu, Ji-Rong Wen, and Di Hu

CVPR'22 Oral: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code Project

Space-Time Memory Network for Sounding Object Localization in Videos

Sizhe Li‡, Yapeng Tian‡, and Chenliang Xu

BMVC'21: The British Machine Vision Conference.

Paper Code Project

Can audio-visual integration strengthen robustness under multimodal attacks?

Yapeng Tian and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

Yapeng Tian, Di Hu, and Chenliang Xu

CVPR'21: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu

ECCV'20 Spotlight: European Conference on Computer Vision.

Paper Code Data

Deep Audio Prior

Yapeng Tian, Chenliang Xu, and Dingzeyu Li

CVPRW'20: CVPR Workshops.

Paper Code Project

Interpretable and Controllable Audio-Visual Video Captioning

Yapeng Tian, Chenxiao Guan, Goodman Justin, Marc Moore, and Chenliang Xu

CVPRW'19: CVPR Workshops.

Multisensory interpretability in terms of the audio-visual video captioning task.

Paper

Audio-Visual Event Localization in Unconstrained Videos

Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, Chenliang Xu

ECCV'18: European Conference on Computer Vision.

Paper Video Code Data Project

Video Matting via Consistency-Regularized Graph Neural Networks

Tiantian Wang, Sifei Liu, Yapeng Tian, Kai Li, and Ming-Hsuan Yang

ICCV'21: IEEE/CVF International Conference on Computer Vision.

Paper Code

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiaoyu Xiang‡, Yapeng Tian‡, Yulun Zhang, Yun Fu, Jan Allebach, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Video Code

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu

CVPR'20: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

This is the first work that uses deformable alignment to address video restoration.

Paper Video Code

Transformer-empowered Multi-contrast MRI Super-Resolution

Guangyuan Li, Jun Lv, Yapeng Tian, Qi Dou, Chengyan Wang, Chenliang Xu, and Jing Qin

CVPR'22: IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Paper Code

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

Bin Xia, Yapeng Tian, Yucheng Hang, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Efficient Non-Local Contrastive Attention for Image Super-Resolution

Bin Xia‡, Yucheng Hang‡, Yapeng Tian, Wenming Yang, Qingmin Liao, Jie Zhou

AAAI'22: The AAAI Conference on Artificial Intelligence.

Paper Code

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

TPAMI'20: IEEE Transactions on Pattern Analysis and Machine Intelligence.

Paper Code

CFSNet: Toward a Controllable Feature Space for Image Restoration

Wei Wang‡, Ruiming Guo‡, Yapeng Tian, and Wenming Yang

ICCV'19: IEEE/CVF International Conference on Computer Vision.

Paper Code

LCSCNet: Linear Compressing Based Skip-Connecting Network for ISR

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, Qingmin Liao

TIP'19: IEEE Trans. Image Processing.

Paper

Deep Learning for Single Image Super-Resolution: A Brief Review

Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, JingHao Xue, Qingmin Liao

TMM'19: IEEE Trans. Multimedia.

Paper

Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian, Yu Kong , Bineng Zhong, Yun Fu

CVPR'18 Spotlight: IEEE/CVF Conf. on Computer Vision and Pattern Recognition.

Paper Code

NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results

Timofte et al.

CVPRW'17: CVPR Workshops.

Paper

Consistent Coding Scheme for Single-Image Super-Resolution

Wenming Yang, Yapeng Tian, Fei Zhou, Qingmin Liao, Hai Chen, Chenglin Zheng

TMM'16: EEE Trans. Multimedia. (First student author)

Paper

Anchored Neighborhood Regression based SISR from Self-examples

Yapeng Tian, Fei Zhou, Wenming Yang, Xuesen Shang, Qingmin Liao

ICIP'16: IEEE International Conference on Image Processing.

Paper Code

SISR Using Clustering-Based Global Regression and Propagation Filtering

Wenming Yang, Yapeng Tian, Fei Zhou, ..., Qingmin Liao

ACPR'15 Oral: Asian Conference on Pattern Recognition. (First student author)

Paper

Teaching

Courses:

  • Fall 2022 - CS 6334: Virtual Reality (UT Dallas)

Tutorials:

Professional Activities

Talks and Seminars:

  • Audio-Visual Scene Understanding Towards Unified, Explainable, and Robust Multisensory Perception

    KTH Dive-Deep Seminar, Dec. 2021
         RIT PhD Colloquium Series, Oct. 2021

  • Audio-Visual Video Understanding, IIAI Seminar, Sep. 2021
  • The Future of Audio-Visual Research Panel Discussion, VALSE Webinar, Nov. 2021

Conference Program Committee/Reviewer:

  • CVPR: IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • ICCV: IEEE/CVF International Conference on Computer Vision
  • ECCV: European Conference on Computer Vision
  • NeurIPS: Conference on Neural Information Processing Systems
  • ICLR: International Conference on Learning Representations
  • AAAI: AAAI Conference on Artificial Intelligence
  • ICML: International Conference on Machine Learning
  • WACV: Winter Conference on Applications of Computer Vision
  • ACCV: Asian Conference on Computer Vision

Journal Reviewer:

  • TPAMI: IEEE Transactions on Pattern Analysis and Machine Intelligence
  • TMLR: The Transactions on Machine Learning Research
  • TIP: IEEE Transactions on Image Processing
  • TNNLS: IEEE Transactions on Neural Networks and Learning Systems
  • TMM: IEEE Transactions on Multimedia
  • TCSVT: IEEE Transcations on Circuits and Systems for Video Technology
  • TASLP: IEEE/ACM Transactions on Audio, Speech and Language Processing
  • Scientific Reports–Nature
  • CGF: Computer Graphics Forum
  • CVIU: Computer Vision and Image Understanding
  • SPIC: Signal Processing: Image Communication
  • IEEE Access

Awards

CVPR Doctoral Consortium, 2022
Top 10% of High-Scoring Reviewers for NeurIPS, 2020
Invited attendee of Amazon Graduate Student Symposium, Seattle, USA, 2019
Outstanding Graduate of Tsinghua University, 2017
Outstanding Master Thesis Award, Tsinghua University, 2017
National Scholarship, Tsinghua University, 2016
Second-class Scholarship, Tsinghua University, 2015

Vitæ

Full CV in PDF.

  • University of Rochester 2017 - now
    Ph.D. Student
    Department of Computer Science
  • Meta Sep. 2021 - Jan. 2022
    Research Intern
    Reality Labs
  • Adobe Summer 2021
    Research Intern
    Creative Intelligence Lab
  • Adobe Summer 2019
    Research Intern
    Creative Intelligence Lab
  • Tsinghua University 2014-2017
    M.E. Student
    Department of Electronic Engineering
  • Chinese Academy of Sciences Nov. 2016- May 2017
    Visiting Student
    Shenzhen Institutes of Advanced Technology
  • Xidian University 2009 - 2013
    B.E. Student
    School of Electronic Engineering

This website was built with jekyll based on a template from Martin Saveski.

Yapeng Tian
yapengtian [AT] rochester.edu