Matching Slides to Presentation Videos

Abstract

Video streaming is becoming a major channel for distance learning (or e-learning). A tremendous number of videos for educational purpose are capturedand archived in various e-learning systems today throughout schools, corporations and over the Internet. However, making information searchable and browsable, and presenting results optimally for a wide range of users and systems, remains a challenge.In this work two core algorithms have been developedto support effective browsing and searching of educational videos. The first is a fully automatic approach that recognizes slides in the videowith high accuracy. Built upon SIFT (scale invariant feature transformation) keypoint matching using RANSAC (random sample consensus), the approach is independent of capture systems and can handle a variety of videos with different styles and plentiful ambiguities. In particular, we propose a multi-phase matching pipeline that incrementally identifies slides from the easy ones to the difficult ones. We achieve further robustness by using the matching confidence as part of a dynamic Hidden Markov model (HMM) that integrates temporal information, taking camera operations into account as well.The second algorithm locates slides in the video. We develop a non-linear optimization method (bundle adjustment) to accurately estimate the projective transformations (homographies) between slides and video frames. Different from estimating homography from a single image, our method solves a set of homographies jointly in a frame sequence that is related to a single slide.These two algorithms open up a series of possibilities for making the video content more searchable, browsable and understandable, thus greatly enriching the user's learning experience. Their usefulness has been demonstrated in the SLIC (Semantically Linking Instructional Content) system, which aims to turnsimple video content into fully interactive learning experience for students and scholars.