Combining Content-based Analysis and Crowdsourcing to Improve User Interaction with Zoomable Video

This paper introduces a new paradigm for interacting with zoomable video. Our interaction technique reduces the number of zooms and pans required by providing recommended viewports to the users, and replaces multiple zoom and pan actions with a simple click on the recommended viewport. The efficacy of our technique lies on the quality of the recommended viewport, which needs to match the user intention, track movement in the scene, and frame the scene in the video properly. To this end, we propose a hybrid method where content analysis and crowdsourcing are used to com- plement each other to recommend the viewports. We first compute a preliminary sets of recommended viewports by analyzing the con- tent of the video. These viewports allow tracking of moving objects in the scene, and are framed without violating basic aesthetic rules. To improve the relevance of the recommended viewport, we collect viewing statistics as users view a video, and use the viewports they select to reinforce the importance of certain recommendations and penalize others. New recommendations that are not previously rec- ognized by content analysis may also emerge. The resulting recom- mended viewports converge towards the regions in the video that are relevant to users. A user study involving 70 participants shows that an user interface build with our paradigm leads to more zooms into more informative regions with fewer interactions required.

The work consists in several steps:

A. Carlier, R. Guntur, V. Charvillat, W.T. Ooi : Combining Content-based Analysis and Crowdsourcing to Improve User Interaction with Zoomable Video ACMMM'11, 43-52

A. Carlier, A. Shafiei, J. Badie, S. Bensiali, W.T. Ooi : COZI: Crowdsourced and Content-based Zoomable Video Player ACMMM'11, 829-830