Games With A Purpose
I N V I S I B L E C O M P U T I N G
Unfortunately, the text near an image
is often scarce or misleading and can
Games with
be hard to process.
Manual labeling is traditionally the
only method for obtaining precise
a Purpose
image descriptions, but this tedious
and labor-intensive process is
extremely costly.
Luis von Ahn
The ESP Game (www.espgame.org)
Carnegie Mellon University
accomplishes the same task through a
simple online game that randomly
pairs players together. Players don’t
know who their partner is, nor can
Through online games,
they communicate with each other.
people can collectively
The only thing partners have in com-
mon is an image they can both see.
solve large-scale
The games’ goal is to guess what
computational problems.
label your partner would give to the
image. Players type a word or phrase
and then press the enter key to submit
it to the game. Once both players have
typed the exact same string, a new
Each year, people around the can be analyzed, a more efficient ver- image appears; they don’t have to type
world spend billions of hours
sion can supersede a less efficient one,
the string at the same time, but each
playing computer games.
and so on. Instead of using a silicon
must type the same string at some
What if all this time and
processor, these “algorithms” run on
point while the image is onscreen.
energy could be channeled into
a processor consisting of ordinary
Players can submit as many words
useful work? What if people playing
humans interacting with computers
or phrases as they want; in fact, the
computer games could, without con-
over the Internet.
more strings they submit, the better
sciously doing so, simultaneously
“Games with a purpose” have a
their chance of getting a match.
solve large-scale problems?
vast range of applications in areas as
The process of typing the same
Despite colossal advances over the
diverse as security, computer vision,
string is called “agreeing on an
past 50 years, computers still don’t
Internet accessibility, adult content fil-
image,” illustrated in Figure 1.
possess the basic conceptual intelli-
tering, and Internet search. Two such
Partners strive to agree on as many
gence or perceptual capabilities that
games under development at Carnegie
images as they can up to a total of 15
most humans take for granted. If we
Mellon University, the ESP Game and
in two and one-half minutes, receiv-
treat human brains as processors in a
Peekaboom, demonstrate how hu-
ing a certain number of points for
distributed system, each can perform
mans, as they play, can solve problems
each match as well as a bonus for
a small part of a massive computation.
that computers can’t yet solve.
matching all 15. A meter at the bot-
Such a “human computation” para-
tom of the screen indicates the part-
digm has enormous potential to
LABELING RANDOM IMAGES
ners’ progress.
address problems that computers
Several important online applica-
Some images have “taboo words”
can’t yet tackle on their own and even-
tions such as search engines and acces-
that players can’t use; the more taboo
tually teach computers many of these
sibility programs for the visually
words an image has, the more points
human talents.
impaired require accurate image
a string match is worth. Players can
Unlike computer processors, humans
descriptions. However, there are no
also choose to pass on difficult images.
require some incentive to become part
guidelines about providing appropri-
To increase the chances of coming
of a collective computation. Online
ate textual descriptions for the mil-
up with the same string for images
games are a seductive method for
lions of images on the Web, and
over the course of a game, partners
encouraging people to participate in
computer vision can’t yet accurately
must learn to “think like each
the process. Such games constitute a
determine their content.
other”—thus the name ESP. It turns
general mechanism for using brain
Current techniques used to catego-
out that the string on which the two
power to solve open problems.
rize images for these applications are
players agree is typically a good label
In fact, designing such a game is
inadequate, largely because they
for the image.
much like designing an algorithm—it
assume that image content on a Web
The ESP Game is extremely popu-
must be proven correct, its efficiency
page is related to adjacent text.
lar, with many people playing more
96
Computer
than 40 hours per week. Within a few
months of initial deployment on 25
October 2003, the game collected
more than 10 million image labels; if
hosted on a major site like MSN
Games or Yahoo! Games, all images
on the Web could be labeled in a mat-
ter of weeks.
Player 1 guesses: purse
Player 2 guesses: handbag
LOCATING OBJECTS IN IMAGES
Player 1 guesses: bag
Player 1 guesses: brown
While the ESP Game can determine
Player 2 guesses: purse
what objects are in an image, it can’t
Success! Agreement on “purse”
Success! Agreement on “purse”
determine where in the image each
object is. Such location information is
necessary for training and testing com-
Figure 1. Partners agreeing on an image in the ESP Game. Neither player can see the
puter vision algorithms.
other’s guesses.
The online game Peekaboom
(www.peekaboom.org) improves on
the data collected by the ESP Game
by obtaining precise location infor-
mation for each object in a given
image. More specifically, it identifies
which pixels belong to which object
in the image.
In this game, two randomly paired
players are assigned the roles of
“Peek” and “Boom.” Peek starts with
a blank screen while Boom sees an
image and a related word, as shown
in Figure 2. All image-word pairs in
Peekaboom come directly from the
ESP Game.
Figure 2. Peekaboom.“Peek” tries to guess the word associated with an image slowly
Peek’s goal is to guess the associated
revealed by “Boom.”
word as Boom slowly reveals the
image. Each time Boom clicks on the
sible. As in the ESP Game, players can
POTENTIAL APPLICATIONS
image, a circular area in a 20-pixel
pass on difficult images.
FOR FUTURE GAMES
radius around that click appears to
Intuitively, to maximize points,
Many other large-scale open prob-
Peek, who can guess what the word is
Boom has an incentive to reveal only
lems can be solved using collective
by typing it in the box below the image.
the areas of the image necessary for
human brainpower in this unique
Boom can see Peek’s guesses and
Peek to guess the correct word. For
way. Examples include:
indicate whether they are “hot” or
example, if the image contains a car
“cold.” To help Peek identify the
and a dog and the associated word is
• Language translation. A game
word, Boom can also “ping” or point
“dog,” Boom would reveal only those
could challenge two players who
to part of the image as well as provide
parts of the image that contain the dog.
don’t speak the same language to
hints indicating whether the word is a
Thus, for a given image-word pair, data
translate text from one language to
verb or refers to a noun in the image,
from multiple games yields the area of
the other.
a noun related to the image, or text in
the image pertaining to the word.
• Monitoring of security cameras.
the image.
Since the release of Peekaboom to a
The rapidly decreasing cost of dig-
When Peek correctly guesses the
general audience on 1 August 2005,
ital video cameras is making it fea-
word, the players receive a certain
nearly 30,000 different people have
sible to install security cameras
number of points—using the hint but-
played the game, generating roughly
almost everywhere. In the context
tons adds points—and then switch
2 million pieces of data—a “piece of
of a game, players could monitor
“booming” and “peeking” roles using
data” is a successful round of
such cameras and alert authorities
a new image-word pair.
Peekaboom in which Peek correctly
about suspected illegal activity.
Players have four minutes to go
guessed the word given Boom’s
• Improving Web search. People
through as many combinations as pos-
revealed region.
have varying degrees of skill at
June 2006
97
I N V I S I B L E C O M P U T I N G
searching for information on the
Web. A game could be designed in
which the players perform searches
for other people.
• Text summarization. Imagine a
game in which people summarize
important documents for the rest
of the world. Solving this problem
would require an intelligent way to
break up such documents into
small “bite-size” chunks.
Any game designed to address
these and other problems must
ensure that game play results in a cor-
rect solution and, at the same time, is
enjoyable. People will play such
games to be entertained, not to solve
a problem—no matter how laudable
the objective.
For the first time in human history,
hundreds of millions of people
can, via the Internet, easily col-
laborate on the same problem. At
CMU, we continue working on ways
to solve complex computational chal-
lenges through the medium of online
entertainment. Two additional “games
with a purpose” under development
include Phetch, which annotates
images with descriptive paragraphs,
and Verbosity, which collects com-
monsense facts to train reasoning algo-
rithms. Log in soon! ■
Luis von Ahn is a postdoctoral associate
at Carnegie Mellon University’s Center
for Algorithm Adaptation, Dissemina-
tion, and Integration (ALADDIN).
Contact him at biglou@cs.cmu.edu.
Editor: Bill Schilit, Intel Research
Seattle; bill.schilit@intel.com
98
Computer
