Institute of Information Science, Academia Sinica

Music is a compound of hierarchical semantics. By leveraging the multi-task learning utilities which can be easily performed by modern deep learning packages, joint learning of multiple musical attributes at a time has become feasible.  Building high-quality datasets with fine-grained labels and learning all of them is the key to achieving high-level music understanding. In this talk, we will discuss some recent research on music understanding in our lab. First, we will introduce Omnizart, the first toolkit that offers transcription models for various music content including piano solo, instrument ensembles, percussion and vocal. We will take the vocal transcription model as an example and show how multi-task learning improves the performance. Second, we will introduce the voice segregation task, which seems to be simple but actually requires several layers of music understanding process in symbolic music. We will show how a simple model can serve as a general solution to this task.