[IIS&CITI]Visionary Technology Seminar Series_A human-computer duet system for music performance
- LecturerDr. Li Su (Institute of Information Science, Academia Sinica)
Host: K.M. Chung, D.N. Yang, Li Su - Time2020-11-10 (Tue.) 10:00 ~ 12:00
- LocationAuditorium106 at IIS new Building(actual); virtual link is available in the abstract
Live Stream
Abstract
**This is the last session of Visionary Technology Series in 2020**
Virtual musicians have become a remarkable phenomenon in contemporary multimedia arts. However, most of the virtual musicians nowadays have not been endowed with the ability to create their own behaviors or to perform music with human musicians. In this paper, we firstly create a virtual violinist, who can collaborate with a human pianist to perform chamber music automaticallywithout any intervention. The system incorporates techniques from various fields, including real-time music tracking, pose estimation, body movement generation, and style transfer from music to visual arts. In our system, the virtualmusician’s behavior is generated based on the given music audio alone, and such a system results in a low-cost, efficient and scalable way to produce human and virtual musicians’ co-performance. The proposed system has been validated in public concerts. Objective quality assessment approaches and possible ways to systematically improve the system are also discussed.
BIO
Li Su received double B. S. degrees in Electronic Engineering and Mathematics in National Taiwan University in 2008, and Ph. D. in communication engineering in National Taiwan University in 2012. Since then, he had served as a postdoctoral research fellow in the Center of Information and Technology Innovation, Since 2017, he has served as an Assistant Research Fellow in the Institute of Information Science, Academia Sinica. Li Su is dedicated to the research of AI in multimedia applications. Specifically, his research focuses are on automatic recognition and analysis of musical signals, style transfer of multimedia contents, and interactive virtual musicians. Since he started his research in the fields, he has more than 50 academic research papers being published in music information retrieval and multimedia technology and has received the Best Paper Award at the International Music Information Retrieval Society (ISMIR) conference in 2019. The technology he developed has been transferred to several concerts, some of which were held in national-level venues including the National Concert Hall of Taiwan and Weiwuying Kaohsiung National Center of the Arts.