中央研究院 資訊科學研究所

Movie summarization, aiming at condensing a full-length movie to a significantly shortened version that still preserves the movie’s major semantic content, is challenging since a large variety of movie scenarios and film styles complicate the problem. In this talk, we will first present a two-stage movie summarization method based on mining the relationship between role-communities. Since the role-communities in earlier scenes are usually used to develop the role relationship in later scenes, in the analysis stage, we construct a social network to characterize the interactions between role-communities. As a result, the social power of each role-community is evaluated by the community’s centrality value and the role communities are clustered into relevant groups based on the centrality values. In the summarization stage, a set of feasible summary combinations of scenes is identified and an information-rich summary is selected from these candidates based on social power preservation.

Our results show that in most test cases the proposed role-community-based method outperforms attention-based and role-based summarization methods in semantic content preservation for a movie summary.

Then, we will show that the movie summarization performance can be further effectively improved by fusing visual saliency information (based on low-level features and high-level cognitive process for an input movie) and high-level semantic analysis (based on the global and local social networks constructed from the movie) using a learning-based method.