Ask'nSeek : a game for object detection and segmentation

Object Detection

This paper proposes a novel approach to solving computer vision problems using games and describes a two-player web-based guessing game, Ask’nSeek, that asks users to guess the location of a hidden region within an image with the help of semantic and topological clues (e.g., “to the left of the house”). The information collected from game logs (coordinates of clicked points and spatial relationships between points and regions) is combined with results from selected computer vision algorithms and used to feed a machine learning algorithm that outputs the outline of the most relevant regions within the image and their names. Two noteworthy aspects of the proposed game are: (i) it solves more than one computer vision problem – namely object detection and object la- beling – in a single game; and (ii) it learns spatial relations within the image from game logs. Ask’nSeek was designed to conceal the desired tasks expected to be performed by the users (labeling regions, clicking on relevant points within the image, and establishing meaningful spatial relationships between points and re- gions) while keeping it quick and entertaining. In this paper we also demonstrate that the underlying semi-supervised machine learning algorithm performs very well for an object detection and labeling task using images and ground truth from publicly available datasets. The game has been evaluated through user studies, which confirmed that the game was easy to understand, intuitive, and fun to play.

A. Carlier, V. Charvillat, O. Marques : Ask'nSeek, A New Game for Object Detection and Labeling ECCV Workshops(1) 2012, 249-258

Object Segmentation

We introduce a new algorithm for image segmentation based on crowdsourcing through a game : Ask’nSeek. The game provides information on the objects of an image, under the form of clicks that are either on the object, or on the back- ground. These logs are then used in order to determine the best segmentation for an object among a set of candidates generated by the state-of-the-art CPMC algorithm. We also introduce a simulator that allows the generation of game logs and therefore gives insight about the number of games needed on an image to perform acceptable segmentation.

A. Salvador, A. Carlier, X. Giro i Nieto, O. Marques, V. Charvillat : Crowdsourced Object Segmentation with a Game CrowdMM'13