Institute of Information Science Academia Sinica
Topic: Deep Learning for Video Frame Interpolation and Object Co-segmentation
Speaker: Dr. Yen-Yu Lin (Research Center for Information Technology Innovation, Academia Sinica)
Date: 2019-05-02 (Thu) 10:30 – 12:00
Location: Auditorium106 at IIS new Building
Host: Chu-Song Chen


In this talk, I will present our recent research results on video frame interpolation and object co-segmentation.

For video frame interpolation, it predicts intermediateframes to produce videos with higher frame rates and smoothview transitions given two consecutive frames as inputs. Weobserve that synthesized frames are more reliable if theycan be used to reconstruct the input frames with high quality.Based on this observation, we introduce a new loss term, thecycle consistency loss. The cycle consistency loss can betterutilize the training data to not only enhance the interpolationresults, but also maintain the performance better withless training data. It can be integrated into any frame interpolationnetwork and trained in an end-to-end manner. Both qualitative andquantitative experiments demonstrate that our model outperformsthe state-of-the-art methods.

For object co-segmentation, it aims to segment the commonobjects in images. We present a CNN-basedmethod that is unsupervised and end-to-endtrainable to better solve this task. Our methodis unsupervised in the sense that it does not requireany training data in the form of object masksbut merely a set of images jointly covering objectsof a specific class. Our method comprisestwo collaborative CNN modules, a feature extractorand a co-attention map generator. The formermodule extracts the features of the estimated objectsand backgrounds. The latter moduleis learned to generate co-attention maps by whichthe estimated figure-ground segmentation can betterfit the former module. Experiments show thatour method achieves superior results, even outperformingthe state-of-the-art, supervised methods.


Yen-Yu Lin received the B.B.A. degree in Information Management, and the M.S. and Ph.D. degrees in Computer Science and Information Engineering from National Taiwan University, Taipei, Taiwan, in 2001, 2003, and 2010, respectively. He is currently an Associate Research Fellow with the Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan. His research interests include computer vision, machine learning, and artificial intelligence.


Lab URL: