Processing Sample (pdf)

Processing:
a programming
handbook for
visual designers
and artists
Casey Reas
Ben Fry
The MIT Press
Cambridge, Massachusetts
London, England
Reas_00_i-xxvi.indd Sec1:iii
Reas_00_i-xxvi.indd Sec1:iii
5/16/07 9:53:05 AM
5/16/07 9:53:05 AM

© 2007 Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means
(including photocopying, recording, or information storage and retrieval) without permission in writing from
the publisher.
MIT Press books may be purchased at special quantity discounts for business or sales promotional use. For
information, please email special_sales@mitpress.mit.edu or write to Special Sales Department, The MIT Press,
55 Hayward Street, Cambridge, MA 02142.
Printed and bound in the United States of America.
Library of Congress Cataloging-in-Publication Data
Reas, Casey.

Processing : a programming handbook for visual designers and artists / Casey Reas & Ben Fry ;

foreword by John Maeda.

p. cm.
Includes bibliographical references and index.
ISBN 978-0-262-18262-1 (hardcover : alk. paper)

1. Computer programming. 2. Computer graphics—Computer programs. 3. Digital art—Computer programs.

4. Art—Data processing. 5. Art and technology. I. Fry, Ben. II. Title.
QA76.6.R4138 2007
005.1—dc22

2006034768
10 9 8 7 6 5 4 3 2 1
Reas_00_i-xxvi.indd Sec1:iv
Reas_00_i-xxvi.indd Sec1:iv
5/16/07 9:53:06 AM
5/16/07 9:53:06 AM

29
34
45
57
67
72
84
91
99
113
121
131
141
189
192
204
208
221
225
233
244
247
289
297
307
320
324
331
336
344
352
354
359
409
415
447
451
472
493
530
535
551
Reas_00_i-xxvi.indd Sec1:vi
Reas_00_i-xxvi.indd Sec1:vi
5/16/07 9:53:06 AM
5/16/07 9:53:06 AM

Contents
Fore
xix
word

279
Motion 1: Lines, Curves
Preface
xxi
291
Motion 2: Machine, Organism
301
Data 4: Arrays

1
Processing . . .
315
Image 2: Animation
Using
9
Processing
321
Image 3: Pixels
327
Typography 2: Motion
17
Structure 1: Code Elements
333
Typography 3: Response
23
Shape 1: Coordinates, Primitives
337
Color 2: Components
37
Data 1: Variables
347
Image 4: Filter, Blend, Copy, Mask
43
Math 1: Arithmetic, Functions
355
Image 5: Image Processing
51
Control 1: Decisions
367
Output 1: Images
61
Control 2: Repetition
371
Synthesis 3: Motion and Arrays
69
Shape 2: Vertices
377
Interviews 3: Animation, Video
79
Math 2: Curves
85
Color 1: Color by Numbers
395
Structure 4: Objects I
95
Image 1: Display, Tint
413
Drawing 2: Kinetic Forms
101
Data 2: Text
421
Output 2: File Export
105
Data 3: Conversion, Objects
427
Input 6: File Import
111
Typography 1: Display
435
Input 7: Interface
117
Math 3: Trigonometry
453
Structure 5: Objects II
127
Math 4: Random
461
Simulate 1: Biology
133
Transform 1: Translate, Matrices
477
Simulate 2: Physics
137
Transform 2: Rotate, Scale
495
Synthesis 4: Structure, Interface
145
Development 1: Sketching, Techniques
501
Interviews 4: Performance, Installation
149
Synthesis 1: Form and Code
155
Interviews 1: Print
519
Extension 1: Continuing . . .
525
Extension 2: 3D
173
Structure 2: Continuous
547
Extension 3: Vision
181
Structure 3: Functions
563
Extension 4: Network
197
Shape 3: Parameters, Recursion
579
Extension 5: Sound
205
Input 1: Mouse I
603
Extension 6: Print
217
Drawing 1: Static Forms
617
Extension 7: Mobile
223
Input 2: Keyboard
633
Extension 8: Electronics
229
Input 3: Events
237
Input 4: Mouse II
Appendixes
661
245
Input 5: Time, Date
Rela
693
ted
Media
251
Development 2: Iteration, Debugging
Gl
699
ossary
255
Synthesis 2: Input and Response
Code
703
Index
261
Interviews 2: Software, Web
Index
705
vii

Reas_00_i-xxvi.indd Sec1:vii
Reas_00_i-xxvi.indd Sec1:vii
5/16/07 9:53:09 AM
5/16/07 9:53:09 AM

88
342
55
65
305
220
415
98
319
323
351
353
359
207
225
232
240
247
444
44
83
124
129
288
296
29
32
75
202
470
488
184
190
407
455
141
113
329
335
530
535
551
viii

Reas_00_i-xxvi.indd Sec1:viii
Reas_00_i-xxvi.indd Sec1:viii
5/16/07 9:53:10 AM
5/16/07 9:53:10 AM

Contents by category
Fore
xix
word
23
Shape 1: Coordinates, Primitives
Preface
xxi
69
Shape 2: Vertices
197
Shape 3: Parameters, Recursion

1
Processing . . .
461
Simulate 1: Biology
Using
9
Processing
477
Simulate 2: Physics
17
Structure 1: Code Elements
85
Color 1: Color by Numbers
173
Structure 2: Continuous
337
Color 2: Components
181
Structure 3: Functions
51
Control 1: Decisions
395
Structure 4: Objects I
61
Control 2: Repetition
453
Structure 5: Objects II
37
Data 1: Variables
149
Synthesis 1: Form and Code
101
Data 2: Text
255
Synthesis 2: Input and Response
105
Data 3: Conversion, Objects
371
Synthesis 3: Motion and Arrays
301
Data 4: Arrays
495
Synthesis 4: Structure, Interface
145
Development 1: Sketching, Techniques
133
Transform 1: Translate, Matrices
251
Development 2: Iteration, Debugging
137
Transform 2: Rotate, Scale
217
Drawing 1: Static Forms
111
Typography 1: Display
413
Drawing 2: Kinetic Forms
327
Typography 2: Motion
95
Image 1: Display, Tint
333
Typography 3: Response
315
Image 2: Animation
321
Image 3: Pixels
155
Interviews 1: Print
347
Image 4: Filter, Blend, Copy, Mask
261
Interviews 2: Software, Web
355
Image 5: Image Processing
377
Interviews 3: Animation, Video
205
Input 1: Mouse I
501
Interviews 4: Performance, Installation
223
Input 2: Keyboard
229
Input 3: Events
519
Extension 1: Continuing . . .
237
Input 4: Mouse II
525
Extension 2: 3D
245
Input 5: Time, Date
547
Extension 3: Vision
427
Input 6: File Import
563
Extension 4: Network
435
Input 7: Interface
579
Extension 5: Sound
43
Math 1: Arithmetic, Functions
603
Extension 6: Print
79
Math 2: Curves
617
Extension 7: Mobile
117
Math 3: Trigonometry
633
Extension 8: Electronics
127
Math 4: Random
279
Motion 1: Lines, Curves
Appendixes
661
291
Motion 2: Machine, Organism
Rela
693
ted
Media
367
Output 1: Images
Gl
699
ossary
421
Output 2: File Export
Code
703
Index
Index
705
ix

Reas_00_i-xxvi.indd Sec1:ix
Reas_00_i-xxvi.indd Sec1:ix
5/18/07 9:44:32 AM
5/18/07 9:44:32 AM

29
30
44
55
63
70
83
88
97
113
124
128
137
174
186
200
206
219
225
231
239
246
281
293
306
316
322
329
334
340
349
353
356
406
414
441
458
464
484
530
535
551
x

Reas_00_i-xxvi.indd Sec1:x
Reas_00_i-xxvi.indd Sec1:x
5/16/07 9:53:14 AM
5/16/07 9:53:14 AM

Extended contents
xix
Foreword by John Maeda
23
Shape 1: Coordinates, Primitives

23
Coordinates
Preface
xxi

size()

xxi
Contents

25
Primitive
shapes
xxii
How to read this book

point(),
line(),

xxiii
Casey’s
introduction

triangle(), quad(), rect(),

xxiv
Ben’s
introduction

ellipse(),
bezier()

xxv
Acknowledgments

31
Drawing
order

31
Gray
values

1
Processing . . .

background(),

1
Software

fill(),stroke(),

3
Literacy

noFill(),
noStroke()

4
Open

33
Drawing
attributes

4
Education

smooth(),
noSmooth(),

6
Network

strokeWeight(), strokeCap(),

7
Context

strokeJoin()

34
Drawing
modes
Using
9
Processing

ellipseMode(),
rectMode()

9
Download,
Install

9
Environment
37
Data 1: Variables

10
Export

37
Data
types

11
Example
walk-through

int, float, boolean,

16
Reference

true,
false

38
Variables
17
Structure 1: Code Elements

=

17

Comments

40
Processing
variables

//, /* */

width,
height

18

Functions
18
Expressions, Statements
43
Math 1: Arithmetic, Functions

“;”,
“,”

43
Arithmetic

20
Case
sensitivity

+, -, *, /, %

20

Whitespace
47
Operator precedence, Grouping
20
Console

()

print(),
println()

48
Shortcuts

++, --, +=, -=, *=, /=, -

49
Constraining
numbers

ceil(), floor(), round(),

min(),
max()
xi

Reas_00_i-xxvi.indd Sec1:xi
Reas_00_i-xxvi.indd Sec1:xi
5/16/07 9:53:17 AM
5/16/07 9:53:17 AM

51
Control 1: Decisions

101
Data 2: Text

51
Relational
expressions

102
Characters

>, <, >=, <=, ==, !=

char

53
Conditionals

103
Words,
Sentences

if, else, {}

String

57
Logical
operators

||, &&, !

105
Data 3: Conversion, Objects

105
Data
conversion
61
Control 2: Repetition

boolean(), byte(), char(),

61
Iteration

int(), float(), str()

for

107
Objects

65
Nested
iteration

“.”,
67
Formatting code blocks

PImage.width,
PImage.height,

String.length,
69
Shape 2: Vertices

String.startsWith(),

69
Vertex

String.endsWith();

beginShape(), endShape(),

String.charAt(),

vertex()

String.toCharArray(),

71
Points,
Lines

String.subString(),

72
Shapes

String.toLowerCase(),

74
Curves

String.toUpperCase(),

curveVertex(),
bezierVertex()
String.equals()
79
Math 2: Curves

111
Typography 1: Display

79
Exponents,
Roots

112
Loading fonts, Drawing text

sq(), sqrt(), pow()

PFont, loadFont(),

80
Normalizing,
Mapping

textFont(),
text()

norm(), lerp(), map()

114
Text
attributes

83
Simple
curves

textSize(), textLeading(),

textAlign(),
textWidth()
85
Color 1: Color by Numbers

86
Setting
colors

117
Math 3: Trigonometry

89
Color
data

117
Angles,
Waves

color,
color()

PI, QUARTER_PI, HALF_PI,

89
RGB,
HSB

TWO_PI, sin(), cos(),

colorMode()

radians(),
degrees()

93
Hexadecimal

123
Circles, Arcs, Spirals

arc()
95
Image 1: Display, Tint

96
Display

127
Math 4: Random

PImage, loadImage(), image()

127
Unexpected
numbers
97
Image color, Transparency

random(),
randomSeed()

tint(), noTint()

130
Noise

noise(),
noiseSeed()
xii

Reas_00_i-xxvi.indd Sec1:xii
Reas_00_i-xxvi.indd Sec1:xii
5/16/07 9:53:18 AM
5/16/07 9:53:18 AM

133
Transform 1: Translate, Matrices

181
Structure 3: Functions

133
Translation

182
Abstraction

translate()

183
Creating
functions

134
Controlling
transformations

void

pushMatrix(),
popMatrix()

193
Function
overloading

194
Calculating and returning values

137
Transform 2: Rotate, Scale

return

137
Rotation,
Scaling

rotate(),
scale()

197
Shape 3: Parameters, Recursion

139
Combining
transformations

197
Parameterized
form

142
New
coordinates

201
Recursion

145
Development 1: Sketching, Techniques

205
Input 1: Mouse I

145
Sketching
software

205
Mouse
data

146
Programming
techniques

mouseX,
mouseY,

pmouseX,
pmouseY

149
Synthesis 1: Form and Code

212
Mouse
buttons

150
Collage
Engine

mousePressed,
mouseButton

151
Riley
Waves

213
Cursor
icon

152
Wilson
Grids

cursor(),
noCursor()

153
Mandelbrot
Set

217
Drawing 1: Static Forms

155
Interviews 1: Print

218
Simple
tools

157
Jared Tarbell.

221
Drawing with images

Fractal.Invaders, Substrate

161
Martin Wattenberg.

223
Input 2: Keyboard

Shape of Song

224
Keyboard
data

165
James
Paterson.

keyPressed,
key

The Objectivity Engine

227
Coded
keys

169
LettError.

keyCode

RandomFont Beowolf

229
Input 3: Events

173
Structure 2: Continuous

229
Mouse
events

173
Continuous
evaluation

mousePressed(),

draw(), frameRate(),

mouseReleased(),

frameCount

mouseMoved(),
mouseDragged()

177
Controlling the ﬂ ow

232
Key
events

setup(),
noLoop(),

keyPressed(),
keyReleased()

178
Variable
scope

235
Controlling the ﬂ ow

loop(),
redraw()
xiii

Reas_00_i-xxvi.indd Sec1:xiii
Reas_00_i-xxvi.indd Sec1:xiii
5/16/07 9:53:18 AM
5/16/07 9:53:18 AM

237
Input 4: Mouse II

301
Data 4: Arrays

237
Constrain

303
Using
arrays

constrain()

Array, [], new, Array.length

238
Distance

306
Storing mouse data

dist()

309
Array
functions

239
Easing

append(), shorten(),

abs()

expand(),
arraycopy()

242
Speed

312
Two-dimensional
arrays

243
Orientation

atan2()

315
Image 2: Animation

316
Sequential
images

245
Input 5: Time, Date

319
Images in motion

245
Seconds, Minutes, Hours

second(), minute(), hour(),

321
Image 3: Pixels

millis()

321
Reading
pixels

249
Date

get()

day(), month(), year()

324
Writing
pixels

set()

251
Development 2: Iteration, Debugging

251
Iteration

327
Typography 2: Motion

252
Debugging

327
Words in motion

331
Letters in motion

255
Synthesis 2: Input and Response

256
Tennis

333
Typography 3: Response

257
Cursor. Peter Cho

333
Responsive
words

258
Typing

335
Responsive
letters

259
Banded Clock. Golan Levin

337
Color 2: Components

261
Interviews 2: Software, Web

337
Extracting
color

263
Ed
Burton.
Sodaconstructor

red(), blue(), green(),

267
Josh
On.
They Rule

alpha(), hue(), saturation(),

271
Jürg
Lehni.
Hektor and Scriptographer

brightness(),

275
Auriea Harvey and Michaël Samyn.

341
Dynamic color palettes

The Endless Forest

347
Image 4: Filter, Blend, Copy, Mask

279
Motion 1: Lines, Curves

347
Filtering,
Blending

279
Controlling
motion

filter(), blend(),

284
Moving along curves

blendColor()

287
Motion through transformation

353
Copying
pixels

copy()

291
Motion 2: Machine, Organism

354
Masking

291
Mechanical
motion

mask()

295
Organic
motion
xiv

Reas_00_i-xxvi.indd Sec1:xiv
Reas_00_i-xxvi.indd Sec1:xiv
5/16/07 9:53:18 AM
5/16/07 9:53:18 AM

355
Image 5: Image Processing

421
Output 2: File Export

356
Pixels

421
Formatting
data

pixels[], loadPixels(),

nf()

updatePixels(),
createImage()

422
Exporting
ﬁ les

359
Pixel
components

saveStrings(), PrintWriter,

360
Convolution

createWriter(),

364
Image as data

PrintWriter.flush(),

PrintWriter.close(),
exit()

367
Output 1: Images

368
Saving
images

427
Input 6: File Import

save()

428
Loading
numbers

369
Saving sequential images

loadStrings(),

saveFrame()

split(), splitTokens()

431
Loading
characters

371
Synthesis 3: Motion and Arrays

WHITESPACE

372
Centipede. Ariel Malka

373
Chronodraw. Andreas Gysin

435
Input 7: Interface

374
AmoebaAbstract_03. Marius Watz

436
Rollover,
Button,
Dragging

375
Mr. Roboto. Leon Hong

442
Check boxes, Radio buttons

448
Scrollbar

377
Interviews 3: Animation, Video

379
Motion
Theory.
R.E.M. “Animal”

453
Structure 5: Objects II

383
Bob
Sabiston.
Waking Life

453
Multiple
constructors

387
Jennifer
Steinkamp. Eye Catching

454
Composite
objects

391
Semiconductor.
The Mini-Epoch Series

456
Inheritance

extends,
super

395
Structure 4: Objects I

395
Object-oriented
programming

461
Simulate 1: Biology

398
Using classes and objects

461
Cellular
automata

class,
Object

469
Autonomous
agents

406
Arrays of objects

409
Multiple
ﬁ les

477
Simulate 2: Physics

477
Motion
simulation

413
Drawing 2: Kinetic Forms

481
Particle
systems

414
Active
tools

487
Springs

416
Active
drawings

495
Synthesis 4: Structure, Interface

496
WithoutTitle.
Lia

497
Pond. William Ngan

498
Swingtree.
ART+COM,

Andreas
Schlegel

499
SodaProcessing. Ed Burton
xv

Reas_00_i-xxvi.indd Sec1:xv
Reas_00_i-xxvi.indd Sec1:xv
5/16/07 9:53:19 AM
5/16/07 9:53:19 AM

501
Interviews 4: Performance, Installation
579
Extension 5: Sound. R. Luke DuBois

503
SUE.C.
Mini Movies
579
Music and sound programming
507
Chris Csikszentmihályi.

in the arts

DJ I, Robot Sound System
582
Sound and musical informatics
511
Golan Levin, Zachary Lieberman.
584
Digital representation of sound

Messa di Voce

and
music

515
Marc
Hansen.
Listening Post
588
Music as information
591
Tools for sound programming
519
Extension 1: Continuing . . .

592
Conclusion

519
Extending
Processing

593
Code
521
Processing and Java

599
Resources
522
Other programming languages
603
Extension 6: Print. Casey Reas
525
Extension 2: 3D. Simon Greenwold
603
Print and computers
525
A short history of 3D software

606
High-resolution
ﬁ le export

526
3D
form

608
Production

531
Camera

612
Conclusion
532
Material and lights

613
Code
536
Tools for 3D

615
Resources

538
Conclusion

539
Code
617
Extension 7: Mobile. Francis Li

545
Resources
617
Mobile software applications
619
The mobile platform
547
Extension 3: Vision. Golan Levin
622
Programming for mobile phones
547
Computer vision in interactive art
624
Mobile programming platforms
549
Elementary computer vision

625
Conclusion

techniques

626
Code
552
Computer vision in the physical world

631
Resources
554
Tools for computer vision

555
Conclusion
633
Extension 8: Electronics.

556
Code

Hernando Barragán and Casey Reas

561
Resources
633
Electronics in the arts

635
Electricity
563
Extension 4: Network.

637
Components

Alexander R. Galloway

638
Circuits
563
The Internet and the arts
639
Microcontrollers and I/O boards
565
Internet protocols and concepts
642
Sensors and communication

569
Network
tools
646
Controlling physical media

571
Conclusion

648
Conclusion

572
Code

649
Code

576
Resources

658
Resources
xvi

Reas_00_i-xxvi.indd Sec1:xvi
Reas_00_i-xxvi.indd Sec1:xvi
5/16/07 9:53:19 AM
5/16/07 9:53:19 AM

661
Appendix A: Order of Operations
663
Appendix B: Reserved Words
664
Appendix C: ASCII, Unicode
669
Appendix D: Bit, Binary, Hex
673
Appendix E: Optimization
679
Appendix F: Programming Languages
686
Appendix G: Code Comparison
Rela
693
ted
Media
Gl
699
ossary
Code
703
Index
Index
705
xvii

Reas_00_i-xxvi.indd Sec1:xvii
Reas_00_i-xxvi.indd Sec1:xvii
5/16/07 9:53:19 AM
5/16/07 9:53:19 AM

Processing…
Processing relates software concepts to principles of visual form, motion, and
interaction. It integrates a programming language, development environment,
and teaching methodology into a uniﬁ ed system. Processing was created to teach
fundamentals of computer programming within a visual context, to serve as a software
sketchbook, and to be used as a production tool. Students, artists, design professionals,
and researchers use it for learning, prototyping, and production.

The Processing language is a text programming language speciﬁ cally designed to
generate and modify images. Processing strives to achieve a balance between clarity and
advanced features. Beginners can write their own programs after only a few minutes
of instruction, but more advanced users can employ and write libraries with additional
functions. The system facilitates teaching many computer graphics and interaction
techniques including vector/raster drawing, image processing, color models, mouse and
keyboard events, network communication, and object-oriented programming. Libraries
easily extend Processing’s ability to generate sound, send/receive data in diverse
formats, and to import/export 2D and 3D ﬁ le formats.

Software

A group of beliefs about the software medium set the conceptual foundation for
Processing and inform decisions related to designing the software and environment.
Software is a unique medium with unique qualities
Concepts and emotions that are not possible to express in other media may be expressed
in this medium. Software requires its own terminology and discourse and should not
be evaluated in relation to prior media such as ﬁ lm, photography, and painting. History
shows that technologies such as oil paint, cameras, and ﬁ lm have changed artistic
practice and discourse, and while we do not claim that new technologies improve art,
we do feel they enable different forms of communication and expression. Software holds
a unique position among artistic media because of its ability to produce dynamic forms,
process gestures, deﬁ ne behavior, simulate natural systems, and integrate other media
including sound, image, and text.
Every programming language is a distinct material
As with any medium, different materials are appropriate for different tasks. When
designing a chair, a designer decides to use steel, wood or other materials based on
the intended use and on personal ideas and tastes. This scenario transfers to writing
software. The abstract animator and programmer Larry Cuba describes his experience
this way: “Each of my ﬁ lms has been made on a different system using a different
1
Reas_01_001-084.indd Sec2:1
Reas_01_001-084.indd Sec2:1
5/16/07 9:54:02 AM
5/16/07 9:54:02 AM

programming language. A programming language gives you the power to express some
ideas, while limiting your abilities to express others.”1 There are many programming
languages available from which to choose, and some are more appropriate than others
depending on the project goals. The Processing language utilizes a common computer
programming syntax that makes it easy for people to extend the knowledge gained
through its use to many diverse programming languages.
Sketching is necessary for the development of ideas
It is necessary to sketch in a medium related to the ﬁ nal medium so the sketch can
approximate the ﬁ nished product. Painters may construct elaborate drawings and
sketches before executing the ﬁ nal work. Architects traditionally work ﬁ rst in cardboard
and wood to better understand their forms in space. Musicians often work with a piano
before scoring a more complex composition. To sketch electronic media, it’s important
to work with electronic materials. Just as each programming language is a distinct
material, some are better for sketching than others, and artists working in software
need environments for working through their ideas before writing ﬁ nal code. Processing
is built to act as a software sketchbook, making it easy to explore and reﬁ ne many
different ideas within a short period of time.
Programming is not just for engineers
Many people think programming is only for people who are good at math and other
technical disciplines. One reason programming remains within the domain of this
type of personality is that the technically minded people usually create programming
languages. It is possible to create different kinds of programming languages and
environments that engage people with visual and spatial minds. Alternative languages
such as Processing extend the programming space to people who think differently. An
early alternative language was Logo, designed in the late 1960s by Seymour Papert as
a language concept for children. Logo made it possible for children to program many
different media, including a robotic turtle and graphic images on screen. A more
contemporary example is the Max programming environment developed by Miller
Puckette in the 1980s. Max is different from typical languages; its programs are created
by connecting boxes that represent the program code, rather than lines of text. It has
generated enthusiasm from thousands of musicians and visual artists who use it as a
base for creating audio and visual software. The same way graphical user interfaces
opened up computing for millions of people, alternative programming environments
will continue to enable new generations of artists and designers to work directly with
software. We hope Processing will encourage many artists and designers to tackle
software and that it will stimulate interest in other programming environments built
for the arts.
2 Processing...
Reas_01_001-084.indd Sec2:2
Reas_01_001-084.indd Sec2:2
5/16/07 9:54:03 AM
5/16/07 9:54:03 AM

Literacy
Processing does not present a radical departure from the current culture of
programming. It repositions programming in a way that is accessible to people who are
interested in programming but who may be intimidated by or uninterested in the type
taught in computer science departments. The computer originated as a tool for fast
calculations and has evolved into a medium for expression.

The idea of general software literacy has been discussed since the early 1970s.
In 1974, Ted Nelson wrote about the minicomputers of the time in Computer Lib /
Dream Machines. He explained “the more you know about computers . . . the better
your imagination can ﬂ ow between the technicalities, can slide the parts together,
can discern the shapes of what you would have these things do.”2 In his book, Nelson
discusses potential futures for the computer as a media tool and clearly outlines ideas
for hypertexts (linked text, which set the foundation for the Web) and hypergrams
(interactive drawings). Developments at Xerox PARC led to the Dynabook, a prototype
for today’s personal computers. The Dynabook vision included more than hardware.
A programming language was written to enable, for example, children to write
storytelling and drawing programs and musicians to write composition programs.
In this vision there was no distinction between a computer user and a programmer.

Thirty years after these optimistic ideas, we ﬁ nd ourselves in a different place.
A technical and cultural revolution did occur through the introduction of the personal
computer and the Internet to a wider audience, but people are overwhelmingly using
the software tools created by professional programmers rather than making their own.
This situation is described clearly by John Maeda in his book Creative Code: “To use a
tool on a computer, you need do little more than point and click; to create a tool, you
must understand the arcane art of computer programming.”3 The negative aspects of
this situation are the constraints imposed by software tools. As a result of being easy to
use, these tools obscure some of the computer’s potential. To fully explore the computer
as an artistic material, it’s important to understand this “arcane art of computer
programming.”

Processing strives to make it possible and advantageous for people within the visual
arts to learn how to build their own tools—to become software literate. Alan Kay, a
pioneer at Xerox PARC and Apple, explains what literacy means in relation to software:
The ability to “read” a medium means you can access materials and tools created by others. The ability to “write”
in a medium means you can generate materials and tools for others. You must have both to be literate. In print
writing, the tools you generate are rhetorical; they demonstrate and convince. In computer writing, the tools you
generate are processes; they simulate and decide.4
Making processes that simulate and decide requires programming.
3 Processing...
Reas_01_001-084.indd Sec2:3
Reas_01_001-084.indd Sec2:3
5/16/07 9:54:04 AM
5/16/07 9:54:04 AM

Open
The open source software movement is having a major impact on our culture and
economy through initiatives such as Linux, but it is having a smaller inﬂ uence on
the culture surrounding software for the arts. There are scattered small projects,
but companies such as Adobe and Microsoft dominate software production and
therefore control the future of software tools used within the arts. As a group, artists
and designers traditionally lack the technical skills to support independent software
initiatives. Processing strives to apply the spirit of open source software innovation
to the domain of the arts. We want to provide an alternative to available proprietary
software and to improve the skills of the arts community, thereby stimulating interest in
related initiatives. We want to make Processing easy to extend and adapt and to make it
available to as many people as possible.

Processing probably would not exist without its ties to open source software. Using
existing open source projects as guidance, and for important software components, has
allowed the project to develop in a smaller amount of time and without a large team of
programmers. Individuals are more likely to donate their time to an open source project,
and therefore the software evolves without a budget. These factors allow the software to
be distributed without cost, which enables access to people who cannot afford the high
prices of commercial software. The Processing source code allows people to learn from
its construction and by extending it with their own code.

People are encouraged to publish the code for programs they’ve written in
Processing. The same way the “view source” function in Web browsers encouraged the
rapid proliferation of website-creation skills, access to others’ Processing code enables
members of the community to learn from each other so that the skills of the community
increase as a whole. A good example involves writing software for tracking objects in a
video image, thus allowing people to interact directly with the software through their
bodies, rather than through a mouse or keyboard. The original submitted code worked
well but was limited to tracking only the brightest object in the frame. Karsten Schmidt
(a k a toxi), a more experienced programmer, used this code as a foundation for writing
more general code that could track multiple colored objects at the same time. Using this
improved tracking code as infrastructure enabled Laura Hernandez Andrade, a graduate
student at UCLA, to build Talking Colors, an interactive installation that superimposes
emotive text about the colors people are wearing on top of their projected image.
Sharing and improving code allows people to learn from one another and to build
projects that would be too complex to accomplish without assistance.
Education
Processing makes it possible to introduce software concepts in the context of the arts
and also to open arts concepts to a more technical audience. Because the Processing
syntax is derived from widely used programming languages, it’s a good base for future
learning. Skills learned with Processing enable people to learn other programming
4 Processing...
Reas_01_001-084.indd Sec2:4
Reas_01_001-084.indd Sec2:4
5/16/07 9:54:04 AM
5/16/07 9:54:04 AM

languages suitable for different contexts including Web authoring, networking,
electronics, and computer graphics.

There are many established curricula for computer science, but by comparison
there have been very few classes that strive to integrate media arts knowledge with
core concepts of computation. Using classes initiated by John Maeda as a model,
hybrid courses based on Processing are being created. Processing has proved useful
for short workshops ranging from one day to a few weeks. Because the environment
is so minimal, students are able to begin programming after only a few minutes of
instruction. The Processing syntax, similar to other common languages, is already
familiar to many people, and so students with more experience can begin writing
advanced syntax almost immediately.

In a one-week workshop at Hongik University in Seoul during the summer of 2003,
the students were a mix of design and computer science majors, and both groups worked
toward synthesis. Some of the work produced was more visually sophisticated and some
more technically advanced, but it was all evaluated with the same criteria. Students like
Soo-jeong Lee entered the workshop without any previous programming experience;
while she found the material challenging, she was able to learn the basic principles and
apply them to her vision. During critiques, her strong visual skills set an example for the
students from more technical backgrounds. Students such as Tai-kyung Kim from the
computer science department quickly understood how to use the Processing software,
but he was encouraged by the visuals in other students’ work to increase his aesthetic
sensibility. His work with kinetic typography is a good example of a synthesis between
his technical skills and emerging design sensitivity.

Processing is also used to teach longer introductory classes for undergraduates
and for topical graduate-level classes. It has been used at small art schools, private
colleges, and public universities. At UCLA, for example, it is used to teach a foundation
class in digital media to second-year undergraduates and has been introduced to the
graduate students as a platform for explorations into more advanced domains. In the
undergraduate Introduction to Interactivity class, students read and discuss the topic
of interaction and make many examples of interactive systems using the Processing
language. Each week new topics such as kinetic art and the role of fantasy in video
games are introduced. The students learn new programming skills, and they produce an
example of work addressing a topic. For one of their projects, the students read Sherry
Turkle’s “Video Games and Computer Holding Power”5 and were given the assignment to
write a short game or event exploring their personal desire for escape or transformation.
Leon Hong created an elegant ﬂ ying simulation in which the player ﬂ oats above a body
of water and moves toward a distant island. Muskan Srivastava wrote a game in which
the objective was to consume an entire table of desserts within ten seconds.

Teaching basic programming techniques while simultaneously introducing
basic theory allows the students to explore their ideas directly and to develop a deep
understanding and intuition about interactivity and digital media. In the graduate-
level Interactive Environments course at UCLA, Processing is used as a platform for
experimentation with computer vision. Using sample code, each student has one week
to develop software that uses the body as an input via images from a video camera.
5 Processing...
Reas_01_001-084.indd Sec2:5
Reas_01_001-084.indd Sec2:5
5/16/07 9:54:05 AM
5/16/07 9:54:05 AM

Zai Chang developed a provocative installation called White Noise where participants’
bodies are projected as a dense series of colored particles. The shadow of each person
is displayed with a different color, and when they overlap, the particles exchange,
thus appearing to transfer matter and infect each other with their unique essence.
Reading information from a camera is an extremely simple action within the Processing
environment, and this facility fosters quick and direct exploration within courses that
might otherwise require weeks of programming tutorials to lead up to a similar project.
Network

Processing takes advantage of the strengths of Web-based communities, and this has
allowed the project to grow in unexpected ways. Thousands of students, educators,
and practitioners across ﬁ ve continents are involved in using the software. The project
website serves as the communication hub, but contributors are found remotely in cities
around the world. Typical Web applications such as bulletin boards host discussions
between people in remote locations about features, bugs, and related events.

Processing programs are easily exported to the Web, which supports networked
collaboration and individuals sharing their work. Many talented people have been
learning rapidly and publishing their work, thus inspiring others. Websites such
as Jared Tarbell’s Complexiﬁ cation.net and Robert Hodgin’s Flight404.com present
explorations into form, motion, and interaction created in Processing. Tarbell creates
images from known algorithms such as Henon Phase diagrams and invents his own
algorithms for image creation, such as those from Substrate, which are reminiscent of
urban patterns (p. 157). On sharing his code from his website, Tarbell writes, “Opening
one’s code is a beneﬁ cial practice for both the programmer and the community. I
appreciate modiﬁ cations and extensions of these algorithms.”6 Hodgin is a self-trained
programmer who uses Processing to explore the software medium. It has allowed him
to move deeper into the topic of simulating natural forms and motion than he could
in other programming environments, while still providing the ability to upload his
software to the Internet. His highly trafﬁ cked website documents these explorations by
displaying the running software as well as providing supplemental text, images, and
movies. Websites such as those developed by Jared and Robert are popular destinations
for younger artists and designers and other interested individuals. By publishing their
work on the Web in this manner they gain recognition within the community.

Many classes taught using Processing publish the complete curriculum on the Web,
and students publish their software assignments and source code from which others can
learn. The websites for Daniel Shiffman’s classes at New York University, for example,
include online tutorials and links to the students’ work. The tutorials for his Procedural
Painting course cover topics including modular programming, image processing, and 3D
graphics by combining text with running software examples. Each student maintains
a web page containing all of their software and source code created for the class. These
pages provide a straightforward way to review performance and make it easy for
members of the class to access each others's work.
6 Processing...
Reas_01_001-084.indd Sec2:6
Reas_01_001-084.indd Sec2:6
5/16/07 9:54:05 AM
5/16/07 9:54:05 AM

The Processing website, www.processing.org, is a place for people to discuss their
projects and share advice. The Processing Discourse section of the website, an online
bulletin board, has thousands of members, with a subset actively commenting on each
others’ work and helping with technical questions. For example, a recent post focused
on a problem with code to simulate springs. Over the course of a few days, messages
were posted discussing the details of Euler integration in comparison to the Runge-
Kutta method. While this may sound like an arcane discussion, the differences between
the two methods can be the reason a project works well or fails. This thread and many
others like it are becoming concise Internet resources for students interested in detailed
topics.
Context

The Processing approach to programming blends with established methods. The core
language and additional libraries make use of Java, which also has elements identical to
the C programming language. This heritage allows Processing to make use of decades of
programming language reﬁ nements and makes it understandable to many people who
are already familiar with writing software.

Processing is unique in its emphasis and in the tactical decisions it embodies with
respect to its context within design and the arts. Processing makes it easy to write
software for drawing, animation, and reacting to the environment, and programs
are easily extended to integrate with additional media types including audio, video,
and electronics. Modiﬁ ed versions of the Processing environment have been built
by community members to enable programs to run on mobile phones (p. 617) and to
program microcontrollers (p. 633).

The network of people and schools using the software continues to grow. In the
ﬁ ve years since the origin on the idea for the software, it has evolved organically
through presentations, workshops, classes, and discussions around the globe. We plan to
continually improve the software and foster its growth, with the hope that the practice
of programming will reveal its potential as the foundation for a more dynamic media.
Notes
1. Larry Cuba, “Calculated Movements,” in Prix Ars Electronica Edition ’87: Meisterwerke der Computerkunst

(H. S. Sauer, 1987), p. 111.
2. Theodore Nelson, “Computer Lib / Dream Machines,” in The New Media Reader, edited by

Noah Wardrip-Fruin and Nick Montfort (MIT Press, 2003), p. 306.
3. John Maeda, Creative Code (Thames & Hudson, 2004), p. 113.
4. Alan Kay, “User Interface: A Personal View,” in The Art of Human-Computer Interface Design,

edited by Brenda Laurel (Addison-Wesley, 1989), p. 193.
5. Chapter 2 in Sherry Turkle, The Second Self: Computers and the Human Spirit (Simon & Schuster, 1984),
pp.
64–92.
6. Jared Tarbell, Complexiﬁ cation.net (2004), http://www.complexiﬁ cation.net/medium.html.
7 Processing...
Reas_01_001-084.indd Sec2:7
Reas_01_001-084.indd Sec2:7
5/16/07 9:54:06 AM
5/16/07 9:54:06 AM

Lines
Processing
File Edit Sketch Tools Help
Menu
Toolbar
Tabs
Lines
void setup() {
Display window
size(100, 100);
noLoop();
}
void draw() {
diagonals(40, 90);
diagonals(60, 62);
diagonals(20, 40);
Text editor
}
void diagonals(int x, int y) {
line(x, y, x+20, y-40);
line(x+10, y, x+30, y-40);
line(x+20, y, x+40, y-40);
}
Message area
Console
Processing Development Environment (PDE)
Use the PDE to create programs. Write the code in the text editor and use
the buttons in the toolbar to run, save, and export the code.
Reas_01_001-084.indd Sec2:8
Reas_01_001-084.indd Sec2:8
5/16/07 9:54:06 AM
5/16/07 9:54:06 AM

Using Processing
Download, Install
The Processing software can be downloaded from the Processing website. Using a
Web browser, navigate to www.processing.org/download and click on the link for your
computer’s operating system. The Processing software is available for Linux, Macintosh,
and Windows. The most up-to-date installation instructions for your operating system
are linked from this page.
Environment
The Processing Development Environment (PDE) consists of a simple text editor for
writing code, a message area, a text console, tabs for managing ﬁ les, a toolbar with
buttons for common actions, and a series of menus. When programs are run, they open
in a new window called the display window.

Pieces of software written using Processing are called sketches. These sketches are
written in the text editor. It has features for cutting/pasting and for searching/replacing
text. The message area gives feedback while saving and exporting and also displays
errors. The console displays text output by Processing programs including complete error
messages and text output from programs with the print() and println() functions.
The toolbar buttons allow you to run and stop programs, create a new sketch, open, save,
and export.

Run
Compiles the code, opens a display window, and runs the program inside.

Stop
Terminates a running program, but does not close the display window.

New
Creates a new sketch.

Open
Provides a menu with options to open ﬁ les from the sketchbook, open an example,

or open a sketch from anywhere on your computer or network.

Save
Saves the current sketch to its current location. If you want to give the sketch a different

name, select “Save As” from the File menu.

Export
Exports the current sketch as a Java applet embedded in an HTML ﬁ le. The folder

containing the ﬁ les is opened. Click on the index.html ﬁ le to load the software in the

computer’s default Web browser.
The menus provide the same functionality as the toolbar in addition to actions for ﬁ le
management and opening reference materials.

File
Commands to manage and export ﬁ les

Edit
Controls for the text editor (Undo, Redo, Cut, Copy, Paste, Find, Replace, etc.)
9
Reas_01_001-084.indd Sec2:9
Reas_01_001-084.indd Sec2:9
5/16/07 9:54:08 AM
5/16/07 9:54:08 AM

Sketch
Commands to run and stop programs and to add media ﬁ les and code libraries.

Tools
Tools to assist in using Processing (automated code formatting, creating fonts, etc.)
Help

Reference
ﬁ les for the environment and language
All Processing projects are called sketches. Each sketch has its own folder. The main
program ﬁ le for each sketch has the same name as the folder and is found inside.
For example, if the sketch is named Sketch_123, the folder for the sketch will be called
Sketch_123 and the main ﬁ le will be called Sketch_123.pde. The PDE ﬁ le extension stands
for the Processing Development Environment.

A sketch folder sometimes contains other folders for media ﬁ les and code libraries.
When a font or image is added to a sketch by selecting “Add File” from the Sketch menu,
a data folder is created. You can also add ﬁ les to your Processing sketch by dragging
them into the text editor. Image and sound ﬁ les dragged into the application window
will automatically be added to the current sketch’s data folder. All images, fonts, sounds,
and other data ﬁ les loaded in the sketch must be in this folder. Sketches are stored in
the Processing folder, which will be in different places on your computer or network
depending on whether you use PC, Mac, or Linux and on how the preferences are set.
To locate this folder, select the “Preferences” option from the File menu (or from the
Processing menu on the Mac) and look for the “Sketchbook location.”

It is possible to have multiple ﬁ les in a single sketch. These can be Processing text
ﬁ les (with the extension .pde) or Java ﬁ les (with the extension .java). To create a new ﬁ le,
click on the arrow button to the right of the ﬁ le tabs. This button enables you to create,
delete, and rename the ﬁ les that comprise the current sketch. You can write functions
and classes in new PDE ﬁ les and you can write any Java code in ﬁ les with the JAVA
extension. Working with multiple ﬁ les makes it easier to reuse code and to separate
programs into small subprograms. This is discussed in more detail in Structure 4 (p. 395).
Export
The export feature packages a sketch to run within a Web browser. When code is
exported from Processing it is converted into Java code and then compiled as a Java
applet. When a project is exported, a series of ﬁ les are written to a folder named applet
that is created within the sketch folder. All ﬁ les from the sketch folder are exported
into a single Java Archive (JAR) ﬁ le with the same name as the sketch. For example, if
the sketch is named Sketch_123, the exported ﬁ le will be called Sketch_123.jar. The applet
folder contains the following:
index.html
HTML
ﬁ le with the applet embedded and a link to the source code and the Processing

homepage. Double-click this ﬁ le to open it in the default Web browser.

Sketch_123.jar
Java Archive containing all necessary ﬁ les for the sketch to run. Includes the Processing

core classes, those written for the sketch, and all included media ﬁ les from the data

folder such as images, fonts, and sounds.
10
Using
Processing
Reas_01_001-084.indd Sec2:10
Reas_01_001-084.indd Sec2:10
5/16/07 9:54:09 AM
5/16/07 9:54:09 AM

Sketch_123.java The
JAVA
ﬁ le generated by the preprocessor from the PDE ﬁ le. This is the actual ﬁ le that

is compiled into the applet by the Java compiler used in Processing.

Sketch_123.pde
The original program ﬁ le. It is linked from the index.html ﬁ le.

loading.gif

An image ﬁ le displayed while the program is loading in a Web browser.
Every time a sketch is exported, the contents of the applet folder are deleted and the ﬁ les
are written from scratch. Any changes previously made to the index.html ﬁ le are lost.
Media ﬁ les not needed for the applet should be deleted from the data folder before it is
exported to keep the ﬁ le size small. For example, if there are unused images in the data
folder, they will be added to the JAR ﬁ le, thus needlessly increasing its size.

In addition to exporting Java applets for the Web, Processing can also export
Java applications for the Linux, Macintosh, and Windows platforms. When “Export
Application” is selected from the File menu, folders will be created for each of the
operating systems speciﬁ ed in the Preferences. Each folder contains the application, the
source code for the sketch, and all required libraries for a speciﬁ c platform.

Additional and updated information about the Processing environment is available
at www.processing.org/reference/environment or by selecting the “Environment” item
from the Help menu of the Processing application.
Example walk-through
A Processing program can be be as short as one line of code and as long as thousands of
lines. This scalability is one of the most important aspects of the language. The following
example walk-through presents the modest goal of animating a sequence of diagonal
lines as a means to explore some of the basic components of the Processing language. If
you are new to programming, some of the terminology and symbols in this section will
be unfamiliar. This walk-through is a condensed overview of the entire book, utilizing
ideas and techniques that are covered in detail later. Try running these programs inside
the Processing application to better understand what the code is doing.

Processing was designed to make it easy to draw graphic elements such as lines,
ellipses, and curves in the display window. These shapes are positioned with numbers
that deﬁ ne their coordinates. The position of a line is deﬁ ned by four numbers, two for
each endpoint. The parameters used inside the line() function determine the position
where the line appears. The origin of the coordinate system is in the upper-left corner,
and numbers increase right and down. Coordinates and drawing different shapes are
discussed on pages 23–30.

line(10, 80, 30, 40); // Left line
0-01

line(20, 80, 40, 40);

line(30, 80, 50, 40); // Middle line

line(40, 80, 60, 40);

line(50, 80, 70, 40); // Right line
11
Using
Processing
Reas_01_001-084.indd Sec2:11
Reas_01_001-084.indd Sec2:11
5/16/07 9:54:09 AM
5/16/07 9:54:09 AM

The visual attributes of shapes are controlled with other code elements that set color and
gray values, the width of lines, and the quality of the rendering. Drawing attributes are
discussed on pages 31–35.

background(0); // Set the black background
0-02

stroke(255); // Set line value to white

strokeWeight(5); // Set line width to 5 pixels

smooth(); // Smooth line edges

line(10, 80, 30, 40); // Left line

line(20, 80, 40, 40);

line(30, 80, 50, 40); // Middle line

line(40, 80, 60, 40);

line(50, 80, 70, 40); // Right line
A variable, such as x, represents a value; this value replaces the symbol x when the
code is run. One variable can then control many features of the program. Variables are
introduced on page 37-41.

int x = 5; // Set the horizontal position
0-03

int y = 60; // Set the vertical position

line(x, y, x+20, y-40); // Line from [5,60] to [25,20]

line(x+10,
y,
x+30,
y-40);
// Line from [15,60] to [35,20]

line(x+20,
y,
x+40,
y-40);
// Line from [25,60] to [45,20]

line(x+30,
y,
x+50,
y-40);
// Line from [35,60] to [55,20]

line(x+40,
y,
x+60,
y-40);
// Line from [45,60] to [65,20]
Adding more structure to a program opens further possibilities. The setup() and
draw() functions make it possible for the program to run continuously—this is required
to create animation and interactive programs. The code inside setup() runs once when
the program ﬁ rst starts, and the code inside draw() runs continuously. One image
frame is drawn to the display window at the end of each loop through draw().

In the following example, the variable x is declared as a global variable, meaning
it can be assigned and accessed anywhere in the program. The value of x increases by 1
each frame, and because the position of the lines is controlled by x, they are drawn to a
different location each time the value changes. This moves the lines to the right.

Line 14 in the code is an if structure. It contains a relational expression comparing
the variable x to the value 100. When the expression is true, the code inside the
block (the code between the { and } associated with the if structure) runs. When the
relational expression is false, the code inside the block does not run. When the value
of x becomes greater than 100, the line of code inside the block sets the variable x to
-40, causing the lines to jump to the left edge of the window. The details of draw() are
discussed on pages 173–175, programming animation is discussed on pages 315–320, and
the if structure is discussed on pages 53–56.
12
Using
Processing
Reas_01_001-084.indd Sec2:12
Reas_01_001-084.indd Sec2:12
5/16/07 9:54:10 AM
5/16/07 9:54:10 AM

int x = 0; // Set the horizontal position
0-04

int y = 55; // Set the vertical position

void
setup() {

size(100, 100); // Set the window to 100 x 100 pixels

}

void
draw() {

background(204);

line(x, y, x+20, y-40); // Left line

line(x+10, y, x+30, y-40); // Middle line

line(x+20, y, x+40, y-40); // Right line

x = x + 1; // Add 1 to x

if (x > 100) { // If x is greater than 100,

x = -40; // assign -40 to x

}

}
When a program is running continuously, Processing stores data from input devices such
as the mouse and keyboard. This data can be used to affect what is happening in the
display window. Programs that respond to the mouse are discussed on pages 205–244.

void
setup() {
0-05

size(100, 100);

}

void
draw() {

background(204);

// Assign the horizontal value of the cursor to x

float x = mouseX;

// Assign the vertical value of the cursor to y

float y = mouseY;

line(x, y, x+20, y-40);

line(x+10, y, x+30, y-40);

line(x+20, y, x+40, y-40);

}
A function is a set of code within a program that performs a speciﬁ c task. Functions
are powerful programming tools that make programs easier to read and change. The
diagonals() function in the following example was written to draw a sequence of
three diagonal lines each time it is run inside draw(). Two parameters, the numbers in
the parentheses after the function name, set the position of the lines. These numbers are
passed into the function deﬁ nition on line 12 and are used as the values for the variables
x and y in lines 13–15. Functions are discussed in more depth on pages 181–196.
13
Using
Processing
Reas_01_001-084.indd Sec2:13
Reas_01_001-084.indd Sec2:13
5/16/07 9:54:11 AM
5/16/07 9:54:11 AM

void setup() {
0-06

size(100, 100);

noLoop();

}

void
draw() {

diagonals(40, 90);

diagonals(60, 62);

diagonals(20, 40);

}

void diagonals(int x, int y) {

line(x, y, x+20, y-40);

line(x+10, y, x+30, y-40);

line(x+20, y, x+40, y-40);

}
The variables used in the previous programs each store one data element. If we want
to have 20 groups of lines on screen, it will require 40 variables: 20 for the horizontal
positions and 20 for the vertical positions. This can make programming tedious and can
make programs difﬁ cult to read. Instead of using multiple variable names, we can use
arrays. An array can store a list of data elements as a single name. A for structure can
be used to cycle through each array element in sequence. Arrays are discussed on pages
301–313, and the for structure is discussed on pages 61–68.

int num = 20;
0-07

int[] dx = new int[num]; // Declare and create an array

int[] dy = new int[num]; // Declare and create an array

void
setup() {

size(100, 100);

for (int i = 0; i < num; i++) {

dx[i] = i * 5;

dy[i] = 12 + (i * 6);

}

}

void
draw() {

background(204);

for (int i = 0; i < num; i++) {

dx[i] = dx[i] + 1;

if (dx[i] > 100) {

dx[i] = -100;

}
14
Using
Processing
Reas_01_001-084.indd Sec2:14
Reas_01_001-084.indd Sec2:14
5/16/07 9:54:12 AM
5/16/07 9:54:12 AM

diagonals(dx[i], dy[i]);
0-07
cont.

}

}

void diagonals(int x, int y) {

line(x, y, x+20, y-40);

line(x+10, y, x+30, y-40);

line(x+20, y, x+40, y-40);

}
Object-oriented programming is a way of structuring code into objects, units of code that
contain both data and functions. This style of programming makes a strong connection
between groups of data and the functions that act on this data. The diagonals()
function can be expanded by making it part of a class deﬁ nition. Objects are created using
the class as a template. The variables for positioning the lines and setting their drawing
attributes then move inside the class deﬁ nition to be more closely associated with
drawing the lines. Object-oriented programming is discussed further on pages 395–411.

Diagonals da, db;
0-08

void
setup() {

size(100, 100);

smooth();

// Inputs: x, y, speed, thick, gray

da = new Diagonals(0, 80, 1, 2, 0);

db = new Diagonals(0, 55, 2, 6, 255);

}

void
draw() {

background(204);

da.update();

db.update();

}

class
Diagonals
{

int x, y, speed, thick, gray;

Diagonals(int xpos, int ypos, int s, int t, int g) {

x = xpos;

y = ypos;

speed = s;

thick = t;

gray = g;

}
15
Using
Processing
Reas_01_001-084.indd Sec2:15
Reas_01_001-084.indd Sec2:15
5/16/07 9:54:12 AM
5/16/07 9:54:12 AM

void update() {
0-08
cont.

strokeWeight(thick);

stroke(gray);

line(x, y, x+20, y-40);

line(x+10, y, x+30, y-40);

line(x+20, y, x+40, y-40);

x = x + speed;

if (x > 100) {

x = -100;

}

}

}
This short walk-through serves to introduce, but not fully explain, some of the
core concepts explored in this text. Many key ideas of working with software were
mentioned only brieﬂ y and others were omitted. Each topic is covered in depth later
in the book.
Reference
The reference for the Processing language complements the text in this book. We advise
keeping the reference open and consulting it while programming. The reference can
be accessed by selecting the “Reference” option from the Help menu within Processing.
It’s also available online at www.processing.org/reference. The reference can also be
accessed within the text window. Highlight a word, right-click (or Ctrl-click in Mac OS
X), and select “Find in Reference” from the menu that appears. You can also select “Find
in Reference” from the Help menu. There are two versions of the Processing reference.
The Abridged Reference lists the elements of the Processing language introduced in this
book, and the Complete Reference documents additional features.
16
Using
Processing
Reas_01_001-084.indd Sec2:16
Reas_01_001-084.indd Sec2:16
5/16/07 9:54:13 AM
5/16/07 9:54:13 AM

Reas_01_001-084.indd Sec2:22
Reas_01_001-084.indd Sec2:22
5/16/07 9:54:15 AM
5/16/07 9:54:15 AM

Shape 1: Coordinates, Primitives
This unit introduces the coordinate system of the display window and a variety
of geometric shapes.
Syntax introduced:
size(), point(), line(), triangle(), quad(), rect(), ellipse(), bezier()
background(), fill(), stroke(), noFill(), noStroke()
strokeWeight(), strokeCap(), strokeJoin()
smooth(), noSmooth(), ellipseMode(), rectMode()
Drawing a shape with code can be difﬁ cult because every aspect of its location must be
speciﬁ ed with a number. When you’re accustomed to drawing with a pencil or moving
shapes around on a screen with a mouse, it can take time to start thinking in relation
to the screen’s strict coordinate grid. The mental gap between seeing a composition on
paper or in your mind and translating it into code notation is wide, but easily bridged.
Coordinates
Before making a drawing, it’s important to think about the dimensions and qualities of
the surface to which you’ll be drawing. If you’re making a drawing on paper, you can
choose from myriad utensils and papers. For quick sketching, newsprint and charcoal
are appropriate. For a reﬁ ned drawing, a smooth handmade paper and range of pencils
may be preferred. In contrast, when you are drawing to a computer’s screen, the primary
options available are the size of the window and the background color.

A computer screen is a grid of small light elements called pixels. Screens come in
many sizes and resolutions. We have three different types of computer screens in our
studios, and they all have a different number of pixels. The laptops have 1,764,000 pixels
(1680 wide * 1050 high), the ﬂ at panels have 1,310,720 pixels (1280 wide * 1024 high),
and the older monitors have 786,432 pixels (1024 wide * 768 high). Millions of pixels
may sound like a vast quantity, but they produce a poor visual resolution compared
to physical media such as paper. Contemporary screens have a resolution around 100
dots per inch, while many modern printers provide more than 1000 dots per inch. On
the other hand, paper images are ﬁ xed, but screens have the advantage of being able to
change their image many times per second.

Processing programs can control all or a subset of the screen’s pixels. When you
click the Run button, a display window opens and allows access to reading and writing
the pixels within. It’s possible to create images larger than the screen, but in most cases
you’ll make a window the size of the screen or smaller.
23
Reas_01_001-084.indd Sec2:23
Reas_01_001-084.indd Sec2:23
5/16/07 9:54:17 AM
5/16/07 9:54:17 AM

The size of the display window is controlled with the size() function:
size(width,
height)
The size() function has two parameters: the ﬁ rst sets the width of the window and the
second sets its height.

// Draw the display window 120 pixels
2-01

// wide and 200 pixels high

size(120,
200);

// Draw the display window 320 pixels 2-02

// wide and 240 pixels high

size(320, 240);

// Draw the display window 200 pixels
2-03

// wide and 200 pixels high

size(200, 200);
24
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:24
Reas_01_001-084.indd Sec2:24
5/16/07 9:54:18 AM
5/16/07 9:54:18 AM

A position on the screen is comprised of an x-coordinate and a y-coordinate. The
x-coordinate is the horizontal distance from the origin and the y-coordinate is the
vertical distance. In Processing, the origin is the upper-left corner of the display window
and coordinate values increase down and to the right. The image on the left shows the
coordinate system, and the image on the right shows a few coordinates placed on
the grid:
X
0
20 40 60 80 100
(0,0)
(99,0)
Y
0
20
40
(50,50)
(20,60)
60
(60,80)
80
100
(0,99)
(99,99)
A position is written as the x-coordinate value followed by the y-coordinate, separated
with a comma. The notation for the origin is (0,0), the coordinate (50,50) has an
x-coordinate of 50 and a y-coordinate of 50, and the coordinate (20,60) is an x-coordinate
of 20 and a y-coordinate of 60. If the size of the display window is 100 pixels wide and
100 pixels high, (0,0) is the pixel in the upper-left corner, (99,0) is the pixel in the upper-
right corner, (0,99) is the pixel in the lower-left corner, and (99,99) is the pixel in the
lower-right corner. This becomes clearer when we look at examples using point().
Primitive shapes
A point is the simplest visual element and is drawn with the point() function:

point(x, y)
This function has two parameters: the ﬁ rst is the x-coordinate and the second is the
y-coordinate. Unless speciﬁ ed otherwise, a point is the size of a single pixel.

// Points with the same X and Y parameters
2-04

// form a diagonal line from the

// upper-left corner to the lower-right corner

point(20,
20);

point(30,
30);

point(40,
40);

point(50,
50);

point(60,
60);
25
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:25
Reas_01_001-084.indd Sec2:25
5/16/07 9:54:19 AM
5/16/07 9:54:19 AM

// Points with the same Y parameter have the
2-05

// same distance from the top and bottom

// edges of the frame

point(50,
30);

point(55,
30);

point(60,
30);

point(65,
30);

point(70,
30);

// Points with the same X parameter have the
2-06

// same distance from the left and right

// edges of the frame

point(70,
50);

point(70,
55);

point(70,
60);

point(70,
65);

point(70,
70);

// Placing a group of points next to one
2-07

// another creates a line

point(50,
50);

point(50,
51);

point(50,
52);

point(50,
53);

point(50,
54);

point(50,
55);

point(50,
56);

point(50,
57);

point(50,
58);

point(50,
59);

// Setting points outside the display
2-08

// area will not cause an error,

// but the points won't be visible

point(-500,
100);

point(400,
-600);

point(140,
2500);

point(2500,
100);
26
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:26
Reas_01_001-084.indd Sec2:26
5/16/07 9:54:20 AM
5/16/07 9:54:20 AM

While it’s possible to draw any line as a series of points, lines are more simply drawn
with the line() function. This function has four parameters, two for each endpoint:

line(x1, y1, x2, y2)
The ﬁ rst two parameters set the position where the line starts and the last two set the
position where the line stops.

// When the y-coordinates for a line are the
2-09

// same, the line is horizontal

line(10, 30, 90, 30);

line(10, 40, 90, 40);

line(10, 50, 90, 50);

// When the x-coordinates for a line are the
2-10

// same, the line is vertical

line(40, 10, 40, 90);

line(50, 10, 50, 90);

line(60, 10, 60, 90);

// When all four parameters are different,
2-11

// the lines are diagonal

line(25, 90, 80, 60);

line(50, 12, 42, 90);

line(45, 30, 18, 36);

// When two lines share the same point they connect
2-12

line(15, 20, 5, 80);

line(90, 65, 5, 80);
The triangle() function draws triangles. It has six parameters, two for each point:

triangle(x1, y1, x2, y2, x3, y3)
The ﬁ rst pair deﬁ nes the ﬁ rst point, the middle pair the second point, and the last
pair the third point. Any triangle can be drawn by connecting three lines, but the
triangle() function makes it possible to draw a ﬁ lled shape. Triangles of all shapes
and sizes can be created by changing the parameter values.

triangle(60, 10, 25, 60, 75, 65); // Filled triangle
2-13

line(60, 30, 25, 80); // Outlined triangle edge

line(25, 80, 75, 85); // Outlined triangle edge

line(75, 85, 60, 30); // Outlined triangle edge
27
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:27
Reas_01_001-084.indd Sec2:27
5/16/07 9:54:21 AM
5/16/07 9:54:21 AM

point(x, y)
(x,y)
line(x1, y1, x2, y2)
(x1,y1)
(x2,y2)
triangle(x1, y1, x2, y2, x3, y3)
(x1,y1)
(x2,y2) (x3,y3)
quad(x1, y1, x2, y2, x3, y3, x4, y4)
(x1,y1) (x4,y4)
(x2,y2) (x3,y3)
rect(x, y, width, height)
(x,y)
height
width
ellipse(x, y, width, height)
(x,y)
height
width
bezier(x1, y1, cx1, cy1, cx2, cy2, x2, y2)
(x1,y1)
(cx1,cy1)
(x2,y2)
(cx2,cy2)
Geometry primitives
Processing has seven functions to assist in making simple shapes. These images show the format for each. Replace
the parameters with numbers to use them within a program. These functions are demonstrated in codes 2-04 to 2-22.
Reas_01_001-084.indd Sec2:28
Reas_01_001-084.indd Sec2:28
5/16/07 9:54:22 AM
5/16/07 9:54:22 AM

triangle(55, 9, 110, 100, 85, 100);
2-14

triangle(55, 9, 85, 100, 75, 100);

triangle(-1, 46, 16, 34, -7, 100);

triangle(16, 34, -7, 100, 40, 100);
The quad() function draws a quadrilateral, a four-sided polygon. The function has eight
parameters, two for each point.

quad(x1, y1, x2, y2, x3, y3, x4, y4)
Changing the parameter values can yield rectangles, squares, parallelograms, and
irregular quadrilaterals.

quad(38, 31, 86, 20, 69, 63, 30, 76);
2-15

quad(20, 20, 20, 70, 60, 90, 60, 40);
2-16

quad(20, 20, 70, -20, 110, 0, 60, 40);
Drawing rectangles and ellipses works differently than the shapes previously
introduced. Instead of deﬁ ning each point, the four parameters set the position and the
dimensions of the shape. The rect() function draws a rectangle:

rect(x, y, width, height)
The ﬁ rst two parameters set the location of the upper-left corner, the third sets the
width, and the fourth sets the height. Use the same value for the width and height
parameters to draw a square.

rect(15, 15, 40, 40); // Large square
2-17

rect(55, 55, 25, 25); // Small square

rect(0, 0, 90, 50);
2-18

rect(5, 50, 75, 4);

rect(24, 54, 6, 6);

rect(64, 54, 6, 6);

rect(20, 60, 75, 10);

rect(10, 70, 80, 2);
29
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:29
Reas_01_001-084.indd Sec2:29
5/16/07 9:54:22 AM
5/16/07 9:54:22 AM

The ellipse() function draws an ellipse in the display window:

ellipse(x, y, width, height)
The ﬁ rst two parameters set the location of the center of the ellipse, the third sets the
width, and the fourth sets the height. Use the same value for the width and height
parameters to draw a circle.

ellipse(40, 40, 60, 60); // Large circle
2-19

ellipse(75, 75, 32, 32); // Small circle

ellipse(35, 0, 120, 120);
2-20

ellipse(38, 62, 6, 6);

ellipse(40, 100, 70, 70);
The bezier() function can draw lines that are not straight. A Bézier curve is deﬁ ned by
a series of control points and anchor points. A curve is drawn between the anchor points,
and the control points deﬁ ne its shape:

bezier(x1, y1, cx1, cy1, cx2, cy2, x2, y2)
The function requires eight parameters to set four points. The curve is drawn between
the ﬁ rst and fourth points, and the control points are deﬁ ned by the second and third
points. In software that uses Bézier curves, such as Adobe Illustrator, the control points
are represented by the tiny handles that protrude from the edge of a curve.

bezier(32, 20, 80, 5, 80, 75, 30, 75);
2-21

// Draw the control points

line(32, 20, 80, 5);

ellipse(80, 5, 4, 4);

line(80, 75, 30, 75);

ellipse(80, 75, 4, 4);

bezier(85, 20, 40, 10, 60, 90, 15, 80);
2-22

// Draw the control points

line(85, 20, 40, 10);

ellipse(40, 10, 4, 4);

line(60, 90, 15, 80);

ellipse(60, 90, 4, 4);
30
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:30
Reas_01_001-084.indd Sec2:30
5/16/07 9:54:23 AM
5/16/07 9:54:23 AM

Drawing order
The order in which shapes are drawn in the code deﬁ nes which shapes appear on top of
others in the display window. If a rectangle is drawn in the ﬁ rst line of a program, it is
drawn behind an ellipse drawn in the second line of the program. Reversing the order
places the rectangle on top.

rect(15, 15, 50, 50); // Bottom
2-23

ellipse(60, 60, 55, 55); // Top

ellipse(60, 60, 55, 55); // Bottom
2-24

rect(15, 15, 50, 50); // Top
Gray values
The examples so far have used the default light-gray background, black lines, and white
shapes. To change these default values, it’s necessary to introduce additional syntax. The
background() function sets the color of the display window with a number between 0
and 255. This range may be awkward if you’re not familiar with drawing software on the
computer. The value 255 is white and the value 0 is black, with a range of gray values in
between. If no background value is deﬁ ned, the default value 204 (light gray) is used.

background(0);
2-25

background(124);
2-26

background(230);
2-27
31
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:31
Reas_01_001-084.indd Sec2:31
5/16/07 9:54:24 AM
5/16/07 9:54:24 AM

The fill() function sets the ﬁ ll value of shapes, and the stroke() function sets the
outline value of the drawn shapes. If no ﬁ ll value is deﬁ ned, the default value of 255
(white) is used. If no stroke value is deﬁ ned, the default value of 0 (black) is used.

rect(10, 10, 50, 50);
2-28

fill(204);

// Light gray

rect(20, 20, 50, 50);

fill(153);

// Middle gray

rect(30, 30, 50, 50);

fill(102);

// Dark gray

rect(40, 40, 50, 50);

background(0);
2-29

rect(10, 10, 50, 50);

stroke(102);
// Dark gray

rect(20, 20, 50, 50);

stroke(153);
// Middle gray

rect(30, 30, 50, 50);

stroke(204);
// Light gray

rect(40, 40, 50, 50);
Once a ﬁ ll or stroke value is deﬁ ned, it applies to all shapes drawn afterward. To change
the ﬁ ll or stroke value, use the fill() or stroke() function again.

fill(255); // White
2-30

rect(10, 10, 50, 50);

rect(20, 20, 50, 50);

rect(30, 30, 50, 50);

fill(0);

// Black

rect(40, 40, 50, 50);
An optional second parameter to fill() and stroke() controls transparency. Setting
the parameter to 255 makes the shape entirely opaque, and 0 is totally transparent:

background(0);
2-31

fill(255,
220);

rect(15, 15, 50, 50);

rect(35, 35, 50, 50);

fill(0);
2-32

rect(0, 40, 100, 20);

fill(255, 51); // Low opacity

rect(0, 20, 33, 60);

fill(255, 127); // Medium opacity
32
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:32
Reas_01_001-084.indd Sec2:32
5/16/07 9:54:25 AM
5/16/07 9:54:25 AM

rect(33, 20, 33, 60);
2-32
cont.

fill(255, 204); // High opacity

rect(66, 20, 33, 60);
The stroke and ﬁ ll of a shape can be disabled. The noFill() function stops Processing
from ﬁ lling shapes, and the noStroke() function stops lines from being drawn and
shapes from having outlines. If noFill() and noStroke() are both used, nothing will
be drawn to the screen.

rect(10, 10, 50, 50);
2-33

noFill();

// Disable the fill

rect(20, 20, 50, 50);

rect(30, 30, 50, 50);

rect(20, 15, 20, 70);
2-34

noStroke();

// Disable the stroke

rect(50, 15, 20, 70);

rect(80, 15, 20, 70);
Setting color ﬁ ll and stroke values is introduced in Color 1 (p. 85).
Drawing attributes
In addition to changing the ﬁ ll and stroke values of shapes, it’s also possible to change
attributes of the geometry. The smooth() and noSmooth() functions enable and
disable smoothing (also called antialiasing). Once these functions are used, all shapes
drawn afterward are affected. If smooth() is used ﬁ rst, using noSmooth() cancels the
setting, and vice versa.

ellipse(30, 48, 36, 36);
2-35

smooth();

ellipse(70, 48, 36, 36);

smooth();
2-36

ellipse(30, 48, 36, 36);

noSmooth();

ellipse(70, 48, 36, 36);
Line attributes are controlled by the strokeWeight(), strokeCap(), and
strokeJoin() functions. The strokeWeight() function has one numeric parameter
that sets the thickness of all lines drawn after the function is used. The strokeCap()
function requires one parameter that can be either ROUND, SQUARE, or PROJECT.
33
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:33
Reas_01_001-084.indd Sec2:33
5/16/07 9:54:25 AM
5/16/07 9:54:25 AM

ROUND makes round endpoints, and SQUARE squares them. PROJECT is a mix of the two
that extends a SQUARE endpoint by the radius of the line. The strokeJoin() function
has one parameter that can be either BEVEL, MITER, or ROUND. These parameters
determine the way line segments or the stroke around a shape connects. BEVEL causes
lines to join with squared corners, MITER is the default and joins lines with pointed
corners, and ROUND creates a curve.

smooth();
2-37

line(20, 20, 80, 20); // Default line weight of 1

strokeWeight(6);

line(20, 40, 80, 40); // Thicker line

strokeWeight(18);

line(20, 70, 80, 70); // Beastly line

smooth();
2-38

strokeWeight(12);

strokeCap(ROUND);

line(20, 30, 80, 30); // Top line

strokeCap(SQUARE);

line(20, 50, 80, 50); // Middle line

strokeCap(PROJECT);

line(20, 70, 80, 70); // Bottom line

smooth();
2-39

strokeWeight(12);

strokeJoin(BEVEL);

rect(12, 33, 15, 33); // Left shape

strokeJoin(MITER);

rect(42, 33, 15, 33); // Middle shape

strokeJoin(ROUND);

rect(72, 33, 15, 33); // Right shape
Shape 2 (p. 69) and Shape 3 (p. 197) show how to draw shapes with more ﬂ exibility.
Drawing modes
By default, the parameters for ellipse() set the x-coordinate of the center, the
y-coordinate of the center, the width, and the height. The ellipseMode() function
changes the way these parameters are used to draw ellipses. The ellipseMode()
function requires one parameter that can be either CENTER, RADIUS, CORNER, or
CORNERS. The default mode is CENTER. The RADIUS mode also uses the ﬁ rst and second
parameters of ellipse() to set the center, but causes the third parameter to set half of
34
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:34
Reas_01_001-084.indd Sec2:34
5/16/07 9:54:27 AM
5/16/07 9:54:27 AM

the width and the fourth parameter to set half of the height. The CORNER mode makes
ellipse() work similarly to rect(). It causes the ﬁ rst and second parameters to
position the upper-left corner of the rectangle that circumscribes the ellipse and uses
the third and fourth parameters to set the width and height. The CORNERS mode has a
similar affect to CORNER, but is causes the third and fourth parameters to ellipse()
to set the lower-right corner of the rectangle.

smooth();
2-40

noStroke();

ellipseMode(RADIUS);

fill(126);

ellipse(33, 33, 60, 60); // Gray ellipse

fill(255);

ellipseMode(CORNER);

ellipse(33, 33, 60, 60); // White ellipse

fill(0);

ellipseMode(CORNERS);

ellipse(33, 33, 60, 60); // Black ellipse
In a similar fashion, the rectMode() function affects how rectangles are drawn. It
requires one parameter that can be either CORNER, CORNERS, or CENTER. The default
mode is CORNER, and CORNERS causes the third and fourth parameters of rect()
to draw the corner opposite the ﬁ rst. The CENTER mode causes the ﬁ rst and second
parameters of rect() to set the center of the rectangle and uses the third and fourth
parameters as the width and height.

noStroke();
2-41

rectMode(CORNER);

fill(126);

rect(40, 40, 60, 60); // Gray ellipse

rectMode(CENTER);

fill(255);

rect(40, 40, 60, 60); // White ellipse

rectMode(CORNERS);

fill(0);

rect(40, 40, 60, 60); // Black ellipse
Exercises
1. Create a composition by carefully positioning one line and one ellipse.
2. Modify the code for exercise 1 to change the ﬁ ll, stroke, and background values.
3. Create a visual knot using only Bézier curves.
35
Shape 1: Coordinates, Primitives
Reas_01_001-084.indd Sec2:35
Reas_01_001-084.indd Sec2:35
5/16/07 9:54:27 AM
5/16/07 9:54:27 AM

Color 1: Color by Numbers
This unit introduces code elements and concepts for working with color in software.
Syntax introduced:
color, color(), colorMode()
When Casey and Ben studied color in school, they spent hours carefully mixing paints
and applying it to sheets of paper. They cut paper into perfect squares and carefully
arranged them into precise gradations from blue to orange, white to yellow, and many
other combinations. Over time, they developed an intuition that allowed them to
achieve a speciﬁ c color value by mixing the appropriate components. Through focused
labor, they learned how to isolate properties of color, understand the interactions
between colors, and discuss qualities of color.

Working with color on screen is different from working with color on paper or
canvas. While the same rigor applies, knowledge of pigments for painting (cadmium
red, Prussian blue, burnt umber) and from printing (cyan, yellow, magenta) does not
translate into the information needed to create colors for digital displays. For example,
adding all the colors together on a computer monitor produces white, while adding all
the colors together with paint produces black (or a strange brown). A computer monitor
mixes colors with light. The screen is a black surface, and colored light is added. This is
known as additive color, in contrast to the subtractive color model for inks on paper and
canvas. This image presents the difference between these models:

Additive color
Subtractive color
The most common way to specify color on the computer is with RGB values. An RGB
value sets the amount of red, green, and blue light in a single pixel of the screen. If you
look closely at a computer monitor or television screen, you will see that each pixel is
comprised of three separate light elements of the colors red, green, and blue; but because
our eyes can see only a limited amount of detail, the three colors mix to create a single
color. The intensities of each color element are usually speciﬁ ed with values between 0
and 255 where 0 is the minimum and 255 is the maximum. Many software applications
85
Reas_02_085-100.indd Sec2:85
Reas_02_085-100.indd Sec2:85
5/16/07 10:51:07 AM
5/16/07 10:51:07 AM

also use this range. Setting the red, green, and blue components to 0 creates black.
Setting these components to 255 creates white. Setting red to 255 and green and blue
to 0 creates an intense red.

Selecting colors with convenient numbers can save effort. For example, it’s
common to see the parameters (0, 0, 255) used for blue and (0, 255, 0) for green. These
combinations are often responsible for the garish coloring associated with technical
images produced on the computer. They seem extreme and unnatural because they
don’t account for the human eye’s ability to distinguish subtle values. Colors that appeal
to our eyes are usually not convenient numbers. Rather than picking numbers like 0 and
255, try using a color selector and choosing colors. Processing’s color selector is opened
from the Tools menu. Colors are selected by clicking a location on the color ﬁ eld or by
entering numbers directly. For example, in the ﬁ gure on the facing page, the current
blue selected is deﬁ ned by an R value of 35, a G value of 211, and a B value of 229. These
numbers can be used to recreate the chosen color in your code.
Setting colors
In Processing, colors are deﬁ ned by the parameters to the background(), fill(), and
stroke() functions:

background(value1, value2, value3)

fill(value1, value2, value3)

fill(value1, value2, value3, alpha)

stroke(value1, value2, value3)

stroke(value1, value2, value3, alpha)
By default, the value1 parameter deﬁ nes the red color component, value2 the green
component, and value3 the blue. The optional alpha parameter to fill() or stroke()
deﬁ nes the transparency. The alpha parameter value 255 means the color is entirely
opaque, and the value 0 means it’s entirely transparent (it won’t be visible).

background(242, 204, 47);
9-01

background(174, 221, 60);
9-02
86
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:86
Reas_02_085-100.indd Sec2:86
5/16/07 10:51:08 AM
5/16/07 10:51:08 AM

Color Selector
Drag the cursor inside the window or
input numbers to select a color. The large
square area determines the saturation
and brightness, and the thin vertical strip
determines the hue. The numeric value of
the selected color is displayed in HSB, RGB,
and hexadecimal notation.

background(129, 130, 87);
9-03

noStroke();

fill(174, 221, 60);

rect(17, 17, 66, 66);

background(129, 130, 87);
9-04

noFill();

strokeWeight(4);

stroke(174, 221, 60);

rect(19, 19, 62, 62);

background(116, 193, 206);
9-05

noStroke();

fill(129, 130, 87, 102); // More transparent

rect(20, 20, 30, 60);

fill(129, 130, 87, 204); // Less transparent

rect(50, 20, 30, 60);

background(116, 193, 206);
9-06

int x = 0;

noStroke();

for (int i = 51; i <= 255; i += 51) {

fill(129, 130, 87, i);

rect(x, 20, 20, 60);

x += 20;

}
87
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:87
Reas_02_085-100.indd Sec2:87
5/16/07 10:51:08 AM
5/16/07 10:51:08 AM

background(56, 90, 94);
9-07

smooth();

strokeWeight(12);

stroke(242, 204, 47, 102); // More transparency

line(30, 20, 50, 80);

stroke(242, 204, 47, 204); // Less transparency

line(50, 20, 70, 80);

background(56, 90, 94);
9-08

smooth();

int x = 0;

strokeWeight(12);

for (int i = 51; i <= 255; i += 51) {

stroke(242, 204, 47, i);

line(x, 20, x+20, 80);

x += 20;

}
Transparency can be used to create new colors by overlapping shapes. The colors
originating from overlaps depend on the order in which the shapes are drawn.

background(0);
9-09

noStroke();

smooth();

fill(242, 204, 47, 160); // Yellow

ellipse(47, 36, 64, 64);

fill(174, 221, 60, 160); // Green

ellipse(90, 47, 64, 64);

fill(116, 193, 206, 160); // Blue

ellipse(57, 79, 64, 64);

background(255);
9-10

noStroke();

smooth();

fill(242, 204, 47, 160); // Yellow

ellipse(47, 36, 64, 64);

fill(174, 221, 60, 160); // Green

ellipse(90, 47, 64, 64);

fill(116, 193, 206, 160); // Blue

ellipse(57, 79, 64, 64);
88
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:88
Reas_02_085-100.indd Sec2:88
5/16/07 10:51:09 AM
5/16/07 10:51:09 AM

Color data
The color data type is used to store colors in a program, and the color() function
is used to assign a color variable. The color() function can create gray values, gray
values with transparency, color values, and color values with transparency. Variables of
the color data type can store all of these conﬁ gurations:
color(gray)
color(gray,
alpha)

color(value1, value2, value3)

color(value1, value2, value3, alpha)
The parameters of the color() function deﬁ ne a color. The gray parameter used alone
or with alpha deﬁ nes tones ranging from white to black. The alpha parameter deﬁ nes
transparency with values ranging from 0 (transparent) to 255 (opaque). The value1,
value2, and value3 parameters deﬁ ne values for the different components. Variables
of the color data type are deﬁ ned and assigned in the same way as the int and float
data types discussed in Data 1 (p. 37).
color c1 = color(51); // Creates gray
9-11
color c2 = color(51, 204); // Creates gray with transparency
color c3 = color(51, 102, 153); // Creates blue
color c4 = color(51, 102, 153, 51); // Creates blue with transparency
After a color variable has been deﬁ ned, it can be used as the parameter to the
background(), fill(), and stroke() functions.

color ruby = color(211, 24, 24, 160);
9-12

color pink = color(237, 159, 176);

background(pink);

noStroke();

fill(ruby);

rect(35, 0, 20, 100);
RGB, HSB
Processing uses the RGB color model as its default for working with color, but the HSB
speciﬁ cation can be used instead to deﬁ ne colors in terms of their hue, saturation, and
brightness. The hue of a color is what most people normally think of as the color name:
yellow, red, blue, orange, green, violet. A pure hue is an undiluted color at its most
intense. The saturation is the degree of purity in a color. It is the continuum from the
undiluted, pure hue to its most diluted and dull. The brightness of a color is its relation
to light and dark.
89
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:89
Reas_02_085-100.indd Sec2:89
5/16/07 10:51:09 AM
5/16/07 10:51:09 AM

RGB
HSB
HEX
255
0
0
360 100 100
#FF0000
252
9
45
351
96
99
#FC0A2E
249
16
85
342
93
98
#F91157
249
23 126
332
90
98
#F91881
246
31 160
323
87
97
#F720A4
244
38 192
314
84
96
#F427C4
244
45 226
304
81
96
#F42EE7
226
51 237
295
78
95
#E235F2
196
58 237
285
75
95
#C43CF2
171
67 234
276
71
94
#AB45EF
148
73 232
267
68
93
#944BED
126
81 232
257
65
93
#7E53ED
108
87 229
248
62
92
#6C59EA
95
95 227
239
59
91
#5F61E8
102 122 227
229
56
91
#667DE8
107 145 224
220
53
90
#6B94E5
114 168 224
210
50
90
#72ACE5
122 186 221
201
46
89
#7ABEE2
127 200 219
192
43
88
#7FCDE0
134 216 219
182
40
88
#86DDE0
139 216 207
173
37
87
#8BDDD4
144 214 195
164
34
86
#90DBC7
151 214 185
154
31
86
#97DBBD
156 211 177
145
28
85
#9CD8B5
162 211 172
135
25
85
#A2D8B0
169 209 169
126
21
84
#A9D6AD
175 206 169
117
18
83
#AFD3AD
185 206 175
107
15
83
#BAD3B3
192 204 180
98
12
82
#C1D1B8
197 201 183
89
9
81
#C5CEBB
202 201 190
79
6
81
#CACEC2
202 200 193
70
3
80
#CACCC5
Color by numbers
Every color within a program is set by numbers, and there are more than 16 million colors to choose from.
This diagram presents a few colors and their corresponding numbers for the RGB and HSB color models.
The RGB column is in relation to colorMode(RGB, 255) and the HSB column is in relation to
colorMode(HSB, 360, 100, 100).
Reas_02_085-100.indd Sec2:90
Reas_02_085-100.indd Sec2:90
5/16/07 10:51:09 AM
5/16/07 10:51:09 AM

The colorMode() function sets the color space for a program:
colorMode(mode)
colorMode(mode,
range)

colorMode(mode, range1, range2, range3)
The parameters to colorMode() change the way Processing interprets color data. The
mode parameter can be either RGB or HSB. The range parameters allow Processing to
use different values than the default of 0 to 255. A range of values frequently used in
computer graphics is between 0.0 and 1.0. Either a single range parameter sets the range
for all the color components, or the range1, range2, and range3 parameters set the
range for each—either red, green, blue or hue, saturation, brightness, depending on the
value of the mode parameter.
// Set the range for the red, green, and blue values from 0.0 to 1.0
9-13
colorMode(RGB, 1.0);
A useful setting for HSB mode is to set the range1, range2, and range3 parameters
respectively to 360, 100, and 100. The hue values from 0 to 360 are the degrees around
the color wheel, and the saturation and brightness values from 0 to 100 are percentages.
This setting matches the values used in many color selectors and therefore makes it easy
to transfer color data between other programs and Processing:
// Set the range for the hue to values from 0 to 360 and the
9-14
// saturation and brightness to values between 0 and 100
colorMode(HSB, 360, 100, 100);
The following examples reveal the differences between hue, saturation, and brightness.

// Change the hue, saturation and brightness constant
9-15

colorMode(HSB);

for (int i = 0; i < 100; i++) {

stroke(i*2.5, 255, 255);

line(i, 0, i, 100);

}

// Change the saturation, hue and brightness constant
9-16

colorMode(HSB);

for (int i = 0; i < 100; i++) {

stroke(132, i*2.5, 204);

line(i, 0, i, 100);

}
91
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:91
Reas_02_085-100.indd Sec2:91
5/16/07 10:51:09 AM
5/16/07 10:51:09 AM

// Change the brightness, hue and saturation constant
9-17

colorMode(HSB);

for (int i = 0; i < 100; i++) {

stroke(132, 108, i*2.5);

line(i, 0, i, 100);

}

// Change the saturation and brightness, hue constant
9-18

colorMode(HSB);

for (int i = 0; i < 100; i++) {

for (int j = 0; j < 100; j++) {

stroke(132, j*2.5, i*2.5);

point(i, j);

}

}
It’s easy to make smooth transitions between colors by changing the values used for
color(), fill(), and stroke(). The HSB model has an enormous advantages over
the RGB model when working with code because it’s more intuitive. Changing the values
of the red, green, and blue components often has unexpected results, while estimating
the results of changes to hue, saturation, and brightness follows a more logical path. The
following examples show a transition from green to blue. The ﬁ rst example makes this
transition using the RGB model. It requires calculating all three color values, and the
saturation of the color unexpectedly changes in the middle. The second example makes
the transition using the HSB model. Only one number needs to be altered, and the hue
changes smoothly and independently from the other color properties.

// Shift from blue to green in RGB mode
9-19

colorMode(RGB);

for (int i = 0; i < 100; i++) {

float r = 61 + (i*0.92);

float g = 156 + (i*0.48);

float b = 204 - (i*1.43);

stroke(r, g, b);

line(i, 0, i, 100);

}

// Shift from blue to green in HSB mode
9-20

colorMode(HSB, 360, 100, 100);

for (int i = 0; i < 100; i++) {

float newHue = 200 - (i*1.2);

stroke(newHue, 70, 80);

line(i, 0, i, 100);

}
92
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:92
Reas_02_085-100.indd Sec2:92
5/16/07 10:51:10 AM
5/16/07 10:51:10 AM

Hexadecimal
Hexadecimal (hex) notation is an alternative notation for deﬁ ning color. This method
is popular with designers working on the Web because standards such as HyperText
Markup Language (HTML) and Cascading Style Sheets (CSS) use this notation. Hex
notation for color encodes each of the numbers from 0 to 255 into a two-digit value
using the numbers 0 through 9 and the letters A through F. In this way three RGB values
from 0 to 255 can be written as a single six-digit hex value. A few sample conversions
demonstrate this notation:

RGB

Hex

255, 255, 255

#FFFFFF

0, 0, 0

#000000

102, 153, 204

#6699CC

195, 244, 59

#C3F43B

116, 206, 206

#74CECE
Converting color values from RGB to hex notation is not intuitive. Most often, the value
is taken from a color selector. For instance, you can copy and paste a hex value from
Processing’s color selector into your code. When using color values encoded in hex
notation, you must place a # before the value to distinguish it within the code.

// Code 9-03 rewritten using hex numbers
9-21

background(#818257);

noStroke();

fill(#AEDD3C);

rect(17, 17, 66, 66);
There’s more information about hex notation in Appendix D (p. 669).
Exercises
1. Explore a wide range of color combinations within one composition.
2. Use HSB color and a for structure to design a gradient between two colors.
3. Redraw your composition from exercise 1 using hexadecimal color values.
93
Color 1: Color by Numbers
Reas_02_085-100.indd Sec2:93
Reas_02_085-100.indd Sec2:93
5/16/07 10:51:10 AM
5/16/07 10:51:10 AM

Reas_03_101-172.indd Sec2:148
Reas_03_101-172.indd Sec2:148
5/16/07 9:58:40 AM
5/16/07 9:58:40 AM

Synthesis 1: Form and Code
This unit presents examples of synthesizing concepts from Structure 1 though Transform 2.
The previous units introduced concepts and techniques including coordinates,
drawing with vertices, variables, iteration, conditionals, trigonometry, and
transformations. Understanding each of these in isolation is the ﬁ rst step toward
learning how to program. Learning how to combine these elements is the second step.
There are many ways to combine the components of every programming language for
purposes of communication and expression. This programming skill is best acquired
through writing more ambitious software and reading more complex programs
written by others. This unit introduces four new programs that push beyond those on
the previous pages.

Artists and designers throughout the twentieth century practiced the ideas
and visual styles currently associated with software culture, long before personal
computers became a common tool. The aesthetic legacies of the Bauhaus, art deco,
modernist architecture, and op art movements retain a strong voice in contemporary
culture, while new forms have emerged through software explorations within the
scientiﬁ c and artistic communities. The programs in this unit reference images from
the last hundred years; sampling from Dadaist collage, optical paintings, a twenty-
year-old software program, and mathematics.
The software featured in this unit is longer than the brief examples given in this book. It’s not practical to print it
on these pages, but the code is included in the Processing code download at www.processing.org/learning.
149
Reas_03_101-172.indd Sec2:149
Reas_03_101-172.indd Sec2:149
5/16/07 9:58:41 AM
5/16/07 9:58:41 AM

Collage Engine. Reacting to the horror of World War I, European artists and poets within
the Dada cultural movement produced works that were deliberately irrational and absurd
and that rejected the current standards of art. The poet Tristan Tzara devised a technique
for writing that involved taking text from the newspaper, separating the individual words,
and putting them back together in random order.

The images shown here were produced using a similar technique with photographs
from the ﬁ rst section of The New York Times of 9 June 2006. The pictures were cut, scanned,
and then repositioned randomly to produce these collages.
150
Synthesis 1: Form and Code
Reas_03_101-172.indd Sec2:150
Reas_03_101-172.indd Sec2:150
5/16/07 9:58:41 AM
5/16/07 9:58:41 AM

Riley Waves. These images were inﬂ uenced by the paintings of Bridget Riley, a British
artist who has exhibited her work since the mid-1960s. Riley’s optically vibrant works
often have a strong emotional and visceral effect on the viewer. She works exclusively
with simple geometric shapes such as curves and lines and constructs visual vibrations
through repetition. Because each of the waves in these images transitions from thick
to thin, only the beginShape() and endShape() functions could create them. Like
code 14-09 (p. 122), each wave is comprised of a sequence of triangles drawn using the
TRIANGLE_STRIP parameter.
151
Synthesis 1: Form and Code
Reas_03_101-172.indd Sec2:151
Reas_03_101-172.indd Sec2:151
5/16/07 9:58:43 AM
5/16/07 9:58:43 AM

Interviews 1: Print

Jared Tarbell. Fractal.Invaders, Substrate

Martin
Wattenberg.
Shape of Song

James
Paterson.
The Objectivity Engine

LettError. RandomFont Beowolf
155
Reas_03_101-172.indd Sec2:155
Reas_03_101-172.indd Sec2:155
5/16/07 9:58:47 AM
5/16/07 9:58:47 AM

Reas_03_101-172.indd Sec2:156
Reas_03_101-172.indd Sec2:156
5/16/07 9:58:48 AM
5/16/07 9:58:48 AM

Fractal.Invaders, Substrate (Interview with Jared Tarbell)
Creator
Jared
Tarbell
Year

2004
Medium

Software,
Prints
Software

Flash,
Processing
URL

www.complexification.net
What
are
Fractal.Invaders and Substrate?

Fractal.Invaders and Substrate are unique programs that both generate space-filling
patterns on a two-dimensional surface. Each uses simplified algorithmic processes to render
a more complex whole.

Fractal.Invaders begins with a rectangular region and recursively fills it with little
“invader” objects. Each invader is a combination of black squares arranged in a 5 * 5 grid
generated at random during runtime. The only rule of construction requires that the left side
of the invader be a mirror copy of the right side. This keeps them laterally symmetric, which
endows them with a special attractiveness to the human eye.

There are a total of 32,768 (215) possible invaders. The magnitude of 15 comes from the
product of 3 columns and 5 rows (the last 2 columns of the grid are ignored since they are the
same as the first 2). The 2 comes from the fact that each space in the grid can be either black
or white.

A small bit of interactivity allows each invader to be clicked. Clicking an invader destroys it,
although the empty space left behind is quickly filled with smaller invaders. In this way, the user
is ultimately doomed.

Substrate begins similarly with an empty rectangular region. It has been compared
to crystal formation and the emergent patterns of urban landscapes. A single line (known
internally as a “crack” since the algorithm was inspired by sunbaked mud cracks) begins
drawing itself from some random point in some random direction. The line continues to draw
itself until it either (a) hits the edge of the screen or (b) hits another line, at which point it stops
and two more lines begin. The one simple rule used in the creation of new lines is that they
begin at tangents to existing lines. This process is repeated until there are too many lines to keep

track of or the program is stopped.
tist.

Before writing the program, I only had a vague idea of what it might look like. It wasn’t
until the first couple of bug-free executions that I realized something incredible was happening.
The resulting form was much more complex than the originating algorithm. This particular
tesy of the ar
quality of software is what keeps me interested.

Interesting effects can be created by introducing small variations in the way the first couple
of lines are drawn. One of my favorite initial conditions is the creation of three lines, each in
age cour
Im
its own localized space with a direction that varies from the others by about 30 degrees. After
growing for a short time into coherent lattices, they eventually crash into each other, creating
t
e
,
2004.
an affluence of odd shapes and unexpected mazes.

The watercolor quality of the rendering is achieved by placing large numbers of mostly
Substra
transparent pixels perpendicular to each line’s growth. The trick is to deposit precisely the same
157
Interviews 1: Print
Reas_03_101-172.indd Sec2:157
Reas_03_101-172.indd Sec2:157
5/16/07 9:58:50 AM
5/16/07 9:58:50 AM

number of pixels regardless of the length of the area being filled. This produces an interesting
density modulation across an even mass of pixels.

Why did you create this software?

For me, one of the most enjoyable subjects in computer science is combination. I ask myself
a question like, “Given some rules and a few simple objects, how many possible ways can they
be combined?” Seldom can I answer this using thought alone, mainly because the complexity
of even just a few elements is outside the realm of my imagination. Instead, I write computer
programs to solve it for me. Fractal.Invaders is definitely one of these questions, and is answered
completely with the rendering of every single invader. Substrate asks a similar question but with
results that, although beautiful, are a little less complete.

What software tools were used?
For
Fractal.Invaders, I used a combination of Flash and custom software to create and
capture the invaders, respectively. In Flash, all work was done using ActionScript. A single
symbolic element (black square) exists in the library. Code takes this square and duplicates
it hundreds of thousands of times. The entire generative process takes about five minutes
to complete, depending on the size of the region to be filled and the speed of the execution.
Capturing a high-resolution image of the result is accomplished with a program that scales the
Shockwave Flash (SWF) file very large and saves the screen image out to a file.

Substrate was created entirely in Processing. Processing was particularly well suited for this
as it excels at drawing, especially when dropping millions of deep-color pixels. Processing can
also save out extremely large graphic images in an automated fashion. Oftentimes I will run a
Processing project overnight. In the morning I awake to a vast collection of unique images, the
best of which are archived as print editions.

Why did you use these tools?

I use Flash because I am comfortable working within it. I use Processing because it enables
me to do things Flash simply cannot. Both environments allow me to take a program from
concept to completion in a number of hours. Complex visual logic can be built up without the
bulky overhead required in more traditional graphic programming languages.

Flash excels at rendering very high resolution images nicely, displaying native vector
objects with a high degree of precision and full antialiasing. Processing gives me the
computational speed to increase the number of objects in the system by a magnitude of 20
or more. Both programs allow me to produce work that is capable of being viewed by a large
number of people worldwide.

Why do you choose to work with software?

With software, anything that can be imagined can be built. Software has a mysterious,
undefined border. Programming is truly a process of creating something from nothing. I enjoy
most John Maeda’s perspective: “While engaged in the deepest trance of coding, all one needs to
wish for is any kind of numerical or symbolic resource, and in a flash of lightning it is suddenly
there, at your disposal.”
Fractal.Invaders, 2004. Image courtesy of the artist.
158
Interviews 1: Print
Reas_03_101-172.indd Sec2:158
Reas_03_101-172.indd Sec2:158
5/16/07 9:58:52 AM
5/16/07 9:58:52 AM

159
Interviews 1: Print
Reas_03_101-172.indd Sec2:159
Reas_03_101-172.indd Sec2:159
5/16/07 9:58:57 AM
5/16/07 9:58:57 AM

Input 1: Mouse I
This unit introduces mouse input as a way to control the position and attributes of shapes
on screen. It also explains how to change the cursor icon.
Syntax introduced:
mouseX, mouseY, pmouseX, pmouseY, mousePressed, mouseButton
cursor(), noCursor()
The screen forms a bridge between our bodies and the realm of circuits and electricity
inside computers. We control elements on screen through a variety of devices such as
touch pads, trackballs, and joysticks, but — aside from the keyboard — the most common
input device is the mouse. The computer mouse dates back to the late 1960s when
Douglas Engelbart presented the device as an element of the oN-Line System (NLS), one
of the ﬁ rst computer systems with a video display. The mouse concept was further
developed at the Xerox Palo Alto Research Center (PARC), but its introduction with the
Apple Macintosh in 1984 was the catalyst for its current ubiquity. The design of the
mouse has gone through many revisions in the last thirty years, but its function has
remained the same. In Engelbart’s original patent application in 1970 he referred to the
mouse as an “X-Y position indicator,” and this still accurately, but dryly, deﬁ nes its
contemporary use.

The physical mouse object is used to control the position of the cursor on screen and
to select interface elements. The cursor position is read by computer programs as two
numbers, the x-coordinate and the y-coordinate. These numbers can be used to control
attributes of elements on screen. If these coordinates are collected and analyzed, they
can be used to extract higher-level information such as the speed and direction of the
mouse. This data can in turn be used for gesture and pattern recognition.
Mouse data
The Processing variables mouseX and mouseY (note the capital X and Y) store the
x-coordinate and y-coordinate of the cursor relative to the origin in the upper-left corner
of the display window. To see the actual values produced while moving the mouse, run
this program to print the values to the console:
void draw() {
23-01
frameRate(12);
println(mouseX + " : " + mouseY);
}
205
Reas_04_173-278.indd Sec3:205
Reas_04_173-278.indd Sec3:205
5/16/07 10:05:12 AM
5/16/07 10:05:12 AM

When a program starts, mouseX and mouseY values are 0. If the cursor moves into the
display window, the values are set to the current position of the cursor. If the cursor is
at the left, the mouseX value is 0 and the value increases as the cursor moves to the right.
If the cursor is at the top, the mouseY value is 0 and the value increases as the cursor
moves down. If mouseX and mouseY are used in programs without a draw() or if
noLoop() is run in setup(), the values will always be 0.

The mouse position is most commonly used to control the location of visual
elements on screen. More interesting relations are created when the visual elements
relate differently to the mouse values, rather than simply mimicking the current
position. Adding and subtracting values from the mouse position creates relationships
that remain constant, while multiplying and dividing these values creates changing
visual relationships between the mouse position and the elements on the screen. To
invert the value of the mouse, simply subtract the mouseX value from the width of the
window and subtract the mouseY value from the height of the screen.

// Circle follows the cursor (the cursor position is
23-02

// implied by the crosshairs around the illustration)

void
setup() {

size(100, 100);

smooth();

noStroke();

}

void
draw() {

background(126);

ellipse(mouseX, mouseY, 33, 33);

}

// Add and subtract to create offsets
23-03

void
setup() {

size(100, 100);

smooth();

noStroke();

}

void
draw() {

background(126);

ellipse(mouseX, 16, 33, 33); // Top circle

ellipse(mouseX+20, 50, 33, 33); // Middle circle

ellipse(mouseX-20, 84, 33, 33); // Bottom circle

}
206
Input 1: Mouse I
Reas_04_173-278.indd Sec3:206
Reas_04_173-278.indd Sec3:206
5/16/07 10:05:12 AM
5/16/07 10:05:12 AM

// Multiply and divide to creates scaling offsets
23-04

void
setup() {

size(100, 100);

smooth();

noStroke();

}

void
draw() {

background(126);

ellipse(mouseX, 16, 33, 33); // Top circle

ellipse(mouseX/2, 50, 33, 33); // Middle circle

ellipse(mouseX*2, 84, 33, 33); // Bottom circle

}

// Invert cursor position to create a secondary response
23-05

void
setup() {

size(100, 100);

noStroke();

smooth();

}

void
draw() {

float x = mouseX;

float y = mouseY;

float ix = width - mouseX; // Inverse X

float iy = mouseY - height; // Inverse Y

background(126);

fill(255, 150);

ellipse(x, height/2, y, y);

fill(0, 159);

ellipse(ix, height/2, iy, iy);

}
207
Input 1: Mouse I
Reas_04_173-278.indd Sec3:207
Reas_04_173-278.indd Sec3:207
5/16/07 10:05:12 AM
5/16/07 10:05:12 AM

// Exponential functions can create nonlinear relations
23-06

// between the mouse and shapes affected by the mouse

void
setup() {

size(100, 100);

smooth();

noStroke();

}

void draw() {

background(126);

float normX = mouseX / float(width);

ellipse(mouseX, 16, 33, 33); // Top

ellipse(pow(normX, 4) * width, 50, 33, 33); // Middle

ellipse(pow(normX, 8) * width, 84, 33, 33); // Bottom

}
The Processing variables pmouseX and pmouseY store the mouse values from the
previous frame. If the mouse does not move, the values will be the same, but if the
mouse is moving quickly there can be large differences between the values. To see the
difference, run the following program and alternate moving the mouse slowly and
quickly. Watch the values print to the console.
void draw() {
23-07
frameRate(12);
println(pmouseX - mouseX);
}
Drawing a line from the previous mouse position to the current position shows the
changing position in one frame, revealing the speed and direction of the mouse. When the
mouse is not moving, a point is drawn, but quick mouse movements create long lines.

// Draw a line between the current and previous positions
23-08

void
setup() {

size(100, 100);

strokeWeight(8);

smooth();

}

void
draw() {

background(204);

line(mouseX, mouseY, pmouseX, pmouseY);

}
208
Input 1: Mouse I
Reas_04_173-278.indd Sec3:208
Reas_04_173-278.indd Sec3:208
5/16/07 10:05:13 AM
5/16/07 10:05:13 AM

The mouseX and mouseY values can control translation, rotation, and scale by using
them as parameters in the transformation functions. You can move a circle around the
screen by changing the parameters to translate() rather than by changing the x and
y parameters of ellipse().

// Use translate() to move a shape
23-09

void
setup() {

size(100, 100);

smooth();

noStroke();

}

void
draw() {

background(126);

translate(mouseX, mouseY);

ellipse(0, 0, 33, 33);

}
Before using mouseX and mouseY as parameters to transformation functions, it’s
important to think ﬁ rst about how they relate to the expected parameters. For example,
the rotate() function expects its parameters in units of radians (p. 117). To make a
shape rotate 360 degrees as the cursor moves from the left edge to the right edge of the
window, the values of mouseX must be converted to values from 0.0 to 2π. In the
following example, the map() function is used to make this conversion. The resulting
value is used as the parameter to rotate() to turn the line as the mouse moves back
and forth between the left and right edge of the display window.

// Use rotate() to move a shape
23-10

void
setup() {

size(100, 100);

strokeWeight(8);

smooth();

}

void
draw() {

background(204);

float angle = map(mouseX, 0, width, 0, TWO_PI);

translate(50, 50);

rotate(angle);

line(0, 0, 40, 0);

}
209
Input 1: Mouse I
Reas_04_173-278.indd Sec3:209
Reas_04_173-278.indd Sec3:209
5/16/07 10:05:13 AM
5/16/07 10:05:13 AM

Using the mouseX and mouseY variables with an if structure allows the cursor to
select regions of the screen. The following examples demonstrate the cursor making
a selection between different areas of the display window.

// Cursor position selects the left or right half
23-11

// of the display window

void
setup() {

size(100, 100);

noStroke();

fill(0);

}

void
draw() {

background(204);

if (mouseX < 50) {

rect(0, 0, 50, 100); // Left

} else {

rect(50, 0, 50, 100); // Right

}

}

// Cursor position selects the left, middle,
23-12

// or right third of the display window

void
setup() {

size(100, 100);

noStroke();

fill(0);

}

void
draw() {

background(204);

if (mouseX < 33) {

rect(0, 0, 33, 100); // Left

} else if ((mouseX >= 33) && (mouseX <= 66)) {

rect(33, 0, 33, 100); // Middle

} else {

rect(66, 0, 33, 100); // Right

}

}
210
Input 1: Mouse I
Reas_04_173-278.indd Sec3:210
Reas_04_173-278.indd Sec3:210
5/16/07 10:05:13 AM
5/16/07 10:05:13 AM

// Cursor position selects a quadrant of
23-13

// the display window

void
setup() {

size(100, 100);

noStroke();

fill(0);

}

void
draw() {

background(204);

if ((mouseX <= 50) && (mouseY <= 50)) {

rect(0, 0, 50, 50); // Upper-left

} else if ((mouseX <= 50) && (mouseY > 50)) {

rect(0, 50, 50, 50); // Lower-left

} else if ((mouseX > 50) && (mouseY < 50)) {

rect(50, 0, 50, 50); // Upper-right

} else {

rect(50, 50, 50, 50); // Lower-right

}

}

// Cursor position selects a rectangular area to
23-14

// change the fill color

void
setup() {

size(100, 100);

noStroke();

fill(0);

}

void
draw() {

background(204);

if ((mouseX > 40) && (mouseX < 80) &&

(mouseY > 20) && (mouseY < 80)) {

fill(255);

} else {

fill(0);

}

rect(40, 20, 40, 60);

}
211
Input 1: Mouse I
Reas_04_173-278.indd Sec3:211
Reas_04_173-278.indd Sec3:211
5/16/07 10:05:14 AM
5/16/07 10:05:14 AM

Mouse buttons
Computer mice and other similar input devices typically have between one and three
buttons, and Processing can detect when these buttons are pressed. The button status
and the cursor position together allow the mouse to perform different actions. For
example, pressing a button when the mouse is over an icon can select it, so the icon can
be moved to a different location on screen. The mousePressed variable is true if any
mouse button is pressed and false if no mouse button is pressed. The variable
mouseButton is LEFT, CENTER, or RIGHT depending on the mouse button most recently
pressed. The mousePressed variable reverts to false as soon as the button is released,
but the mouseButton variable retains its value until a different button is pressed. These
variables can be used independently or in combination to control your software. Run
these programs to see how the software responds to your ﬁ ngers.

// Set the square to white when a mouse button is pressed
23-15

void
setup() {

size(100, 100);

}

void
draw() {

background(204);

if (mousePressed == true) {

fill(255); // White

} else {

fill(0); // Black

}

rect(25, 25, 50, 50);

}

// Set the square to black when the left mouse button
23-16

// is pressed and white when the right button is pressed

void
setup() {

size(100, 100);

}

void
draw() {

if (mouseButton == LEFT) {

fill(0); // Black

} else if (mouseButton == RIGHT) {

fill(255); // White

} else {
212
Input 1: Mouse I
Reas_04_173-278.indd Sec3:212
Reas_04_173-278.indd Sec3:212
5/16/07 10:05:14 AM
5/16/07 10:05:14 AM

fill(126); // Gray
23-16
cont.

}

rect(25, 25, 50, 50);

}

// Set the square to black when the left mouse button
23-17

// is pressed, white when the right button is pressed,

// and gray when a button is not pressed

void
setup() {

size(100, 100);

}

void
draw() {

if (mousePressed == true) {

if (mouseButton == LEFT) {

fill(0); // Black

} else if (mouseButton == RIGHT) {

fill(255); // White

}

} else {

fill(126); // Gray

}

rect(25, 25, 50, 50);

}
Not all mice have multiple buttons, and if software is distributed widely, the interaction
should not rely on detecting which button is pressed. For example, if you are posting
your work on the Web, don’t rely on the middle or right button for using the software
because many users won’t have a two- or three-button mouse.
Cursor icon
The cursor can be hidden with the noCursor() function and can be set to appear as a
different icon with the cursor() function. When the noCursor() function is run, the
cursor icon disappears as it moves into the display window. To give feedback about the
location of the cursor within the software, a custom cursor can be drawn and controlled
with the mouseX and mouseY variables.
213
Input 1: Mouse I
Reas_04_173-278.indd Sec3:213
Reas_04_173-278.indd Sec3:213
5/16/07 10:05:14 AM
5/16/07 10:05:14 AM

// Draw an ellipse to show the position of the hidden cursor
23-18
void setup() {
size(100, 100);
strokeWeight(7);
smooth();
noCursor();
}
void draw() {
background(204);
ellipse(mouseX, mouseY, 10, 10);
}
If noCursor() is run, the cursor will be hidden while the program is running until the
cursor() function is run to reveal it.
// Hides the cursor until a mouse button is pressed
23-19
void setup() {
size(100, 100);
noCursor();
}
void draw() {
background(204);
if (mousePressed == true) {
cursor();
}
}
Adding a parameter to the cursor() function allows it to be changed to another icon.
The self-descriptive options for the MODE parameter are ARROW, CROSS, HAND, MOVE,
TEXT, and WAIT.
// Draws the cursor as a hand when a mouse button is pressed
23-20
void setup() {
size(100, 100);
smooth();
}
void draw() {
background(204);
214
Input 1: Mouse I
Reas_04_173-278.indd Sec3:214
Reas_04_173-278.indd Sec3:214
5/16/07 10:05:14 AM
5/16/07 10:05:14 AM

if (mousePressed == true) {
23-20
cont.
cursor(HAND);
} else {
cursor(MOVE);
}
line(mouseX, 0, mouseX, height);
line(0, mouseY, height, mouseY);
}
These cursor images are part of your computer’s operating system and will appear
differently on different machines.
Exercises
1. Control the position of a shape with the mouse. Strive to create a more interesting
relation than one directly mimicking the position of the cursor.
2. Invent three unique shapes that behave differently in relation to the mouse. Each
shape’s behavior should change when the mouse is pressed. Relate the form of
each shape to its behavior.
3. Create a custom cursor that changes as the mouse moves through the
display
window.
215
Input 1: Mouse I
Reas_04_173-278.indd Sec3:215
Reas_04_173-278.indd Sec3:215
5/16/07 10:05:15 AM
5/16/07 10:05:15 AM

Reas_09_519-710.indd Sec6:546
Reas_09_519-710.indd Sec6:546
5/16/07 12:33:13 PM
5/16/07 12:33:13 PM

Extension 3: Vision
Text by Golan Levin
A well-known anecdote relates how, sometime in 1966, the legendary artiﬁ cial intelli-
gence pioneer Marvin Minsky directed an undergraduate student to solve “the problem
of computer vision” as a summer project.1 This anecdote is often resuscitated to
illustrate how egregiously the difﬁ culty of computational vision has been
underestimated. Indeed, nearly forty years later the discipline continues to confront
numerous unsolved (and perhaps unsolvable) challenges, particularly with respect to
high-level “image understanding” issues such as pattern recognition and feature
recognition. Nevertheless, the intervening decades of research have yielded a great
wealth of well-understood, low-level techniques that are able, under controlled
circumstances, to extract meaningful information from a camera scene. These
c.
y
,
n
techniques are indeed elementary enough to be implemented by novice programmers at
ery
the undergraduate or even high-school level.
all
tforms g
Computer vision in interactive art
The ﬁ rst interactive artwork to incorporate computer vision was, interestingly enough,
tesy of the bi
also one of the ﬁ rst interactive artworks. Myron Krueger’s legendary Videoplace,
developed between 1969 and 1975, was motivated by his deeply felt belief that the entire
human body ought to have a role in our interactions with computers. In the Videoplace
ages cour
Im
installation, a participant stands in front of a backlit wall and faces a video projection
screen. The participant’s silhouette is then digitized and its posture, shape, and gestural
,
2004.
movements analyzed. In response, Videoplace synthesizes graphics such as small
rds
“critters” that climb up the participant’s projected silhouette, or colored loops drawn
nda
between the participant’s ﬁ ngers. Krueger also allowed participants to paint lines with
e Sta
their ﬁ ngers, and, indeed, entire shapes with their bodies; eventually, Videoplace offered
more than ﬁ fty compositions and interactions. Videoplace is notable for many “ﬁ rsts” in
nd Doubl
the history of human-computer interaction. Some of its interaction modules allowed two
rds a
participants in mutually remote locations to participate in the same shared video space,
nda
connected across the network—an implementation of the ﬁ rst multiperson virtual
.

Sta
reality, or, as Krueger termed it, an “artiﬁ cial reality.” Videoplace, it should be noted, was
developed before the mouse became the ubiquitous desktop device it is today, and was
(in part) created to demonstrate interface alternatives to the keyboard terminals that
dominated computing so completely in the early 1970s.

Messa di Voce (p. 511), created by this text’s author in collaboration with Zachary
Lieberman, uses whole-body vision-based interactions similar to Krueger’s, but combines
Rafael Lozano-Hemmer
them with speech analysis and situates them within a kind of projection-based
547
Reas_09_519-710.indd Sec6:547
Reas_09_519-710.indd Sec6:547
5/16/07 12:33:15 PM
5/16/07 12:33:15 PM

augmented reality. In this audiovisual performance, the speech, shouts, and songs
produced by two abstract vocalists are visualized and augmented in real time by
synthetic graphics. To accomplish this, a computer uses a set of vision algorithms to
track the locations of the performers’ heads; this computer also analyzes the audio
signals coming from the performers’ microphones. In response, the system displays
various kinds of visualizations on a projection screen located just behind the performers;
these visualizations are synthesized in ways that are tightly coupled to the sounds being
spoken and sung. With the help of the head-tracking system, moreover, these
visualizations are projected such that they appear to emerge directly from the
performers’ mouths.

Rafael Lozano-Hemmer’s installation Standards and Double Standards (2004)
incorporates full-body input in a less direct, more metaphorical context. This work
consists of ﬁ fty leather belts, suspended at waist height from robotic servomotors
mounted on the ceiling of the exhibition room. Controlled by a computer vision-based
tracking system, the belts rotate automatically to follow the public, turning their buckles
slowly to face passers-by. Lozano-Hemmer’s piece “turns a condition of pure surveillance
into an ‘absent crowd’ using a fetish of paternal authority: the belt.”2

The theme of surveillance plays a foreground role in David Rokeby’s Sorting Daemon
(2003). Motivated by the artist’s concerns about the increasing use of automated systems
for proﬁ ling people as part of the “war on terrorism,” this site-speciﬁ c installation works
toward the automatic construction of a diagnostic portrait of its social (and racial)
environment. Rokeby writes: “The system looks out onto the street, panning, tilting and
zooming, looking for moving things that might be people. When it ﬁ nds what it thinks
might be a person, it removes the person’s image from the background. The extracted
person is then divided up according to areas of similar colour. The resulting swatches of
colour are then organized [by hue, saturation and size] within the arbitrary context of
the composite image” projected onsite at the installation’s host location.3

Another project themed around issues of surveillance is Suicide Box, by the Bureau
of Inverse Technology (Natalie Jeremijenko and Kate Rich). Presented as a device for
measuring the hypothetical “despondency index” of a given locale, the Suicide Box
nevertheless records very real data regarding suicide jumpers from the Golden Gate
Bridge. According to the artists, “The Suicide Box is a motion-detection video system,
positioned in range of the Golden Gate Bridge, San Francisco, in 1996. It watched the
bridge constantly and when it recognized vertical motion, captured it to a video record.
The resulting footage displays as a continuous stream the trickle of people who jump off
the bridge. The Golden Gate Bridge is the premiere suicide destination in the United
States; a 100-day initial deployment period of the Suicide Box recorded 17 suicides.
During the same time period the Port Authority counted only 13.”4 Elsewhere,
Jeremijenko has explained that “the idea was to track a tragic social phenomenon which
was not being counted—that is, doesn’t count.”5 The Suicide Box has met with
considerable controversy, ranging from ethical questions about recording the suicides to
disbelief that the recordings could be real. Jeremijenko, whose aim is to address the
hidden politics of technology, has pointed out that such attitudes express a recurrent
theme—“the inherent suspicion of artists working with material evidence”—evidence
548
Extension 3: Vision
Reas_09_519-710.indd Sec6:548
Reas_09_519-710.indd Sec6:548
5/16/07 12:33:15 PM
5/16/07 12:33:15 PM

obtained, in this case, with the help of machine vision-based surveillance.

Considerably less macabre is Christian Möller’s clever Cheese installation (2003),
which the artist developed in collaboration with the California Institute of Technology
and the Machine Perception Laboratories of the University of California, San Diego.
Motivated, perhaps, by the culture shock of his relocation to Hollywood, the German-
born Möller directed “six actresses to hold a smile for as long as they could, up to one and
a half hours. Each ongoing smile is scrutinized by an emotion recognition system, and
whenever the display of happiness fell below a certain threshold, an alarm alerted them
to show more sincerity.”6 The installation replays recordings of the analyzed video on
six ﬂ at-panel monitors, with the addition of a ﬂ uctuating graphic level-meter to indicate
the strength of each actress’ smile. The technical implementation of this artwork’s
vision-based emotion recognition system is quite sophisticated.

As can be seen from these examples, artworks employing computer vision range
from the highly formal and abstract to the humorous and sociopolitical. They concern
themselves with the activities of willing participants, paid volunteers, or unaware
strangers. They track people of interest at a wide variety of spatial scales, from extremely
intimate studies of their facial expressions, to the gestures of their limbs, to the
movements of entire bodies. The examples above represent just a small selection of
notable works in the ﬁ eld and of the ways in which people (and objects) have been
tracked and dissected by video analysis. Other noteworthy artworks that use machine
vision include Marie Sester’s Access; Joachim Sauter and Dirk Lüsebrink’s Zerseher and
Bodymover; Scott Snibbe’s Boundary Functions and Screen Series; Camille Utterback and
Romy Achituv’s TextRain; Jim Campbell’s Solstice; Christa Sommerer and Laurent
Mignonneau’s A-Volve; Danny Rozin’s Wooden Mirror; Chico MacMurtrie’s Skeletal
Reﬂ ection, and various works by Simon Penny, Toshio Iwai, and numerous others. No
doubt many more vision-based artworks remain to be created, especially as these
techniques gradually become incorporated into developing ﬁ elds like physical
computing and robotics.
Elementary computer vision techniques
To understand how novel forms of interactive media can take advantage of computer
vision techniques, it is helpful to begin with an understanding of the kinds of problems
that vision algorithms have been developed to address, and of their basic mechanisms of
operation. The fundamental challenge presented by digital video is that it is
computationally “opaque.” Unlike text, digital video data in its basic form—stored solely
as a stream of rectangular pixel buffers—contains no intrinsic semantic or symbolic
information. There is no widely agreed upon standard for representing the content of
video, in a manner analogous to HTML, XML, or even ASCII for text (though some new
initiatives, notably the MPEG-7 description language, may evolve into such a standard in
the future). As a result, a computer, without additional programming, is unable to
answer even the most elementary questions about whether a video stream contains a
person or object, or whether an outdoor video scene shows daytime or nighttime, et
549
Extension 3: Vision
Reas_09_519-710.indd Sec6:549
Reas_09_519-710.indd Sec6:549
5/16/07 12:33:15 PM
5/16/07 12:33:15 PM

cetera. The discipline of computer vision has developed to address this need.

Many low-level computer vision algorithms are geared to the task of distinguishing
which pixels, if any, belong to people or other objects of interest in the scene. Three
elementary techniques for accomplishing this are frame differencing, which attempts to
locate features by detecting their movements; background subtraction, which locates
visitor pixels according to their difference from a known background scene; and
brightness thresholding, which uses hoped-for differences in luminosity between
foreground people and their background environment. These algorithms, described in
the following examples, are extremely simple to implement and help constitute a base
of detection schemes from which sophisticated interactive systems may be built.
Example 1: Detecting motion (p. 556)
The movements of people (or other objects) within the video frame can be detected and
quantiﬁ ed using a straightforward method called frame differencing. In this technique,
each pixel in a video frame F1 is compared with its corresponding pixel in the
subsequent frame F2. The difference in color and/or brightness between these two pixels
is a measure of the amount of movement in that particular location. These differences
can be summed across all of the pixels’ locations to provide a single measurement of the
aggregate movement within the video frame. In some motion detection
implementations, the video frame is spatially subdivided into a grid of cells, and the
values derived from frame differencing are reported for each of the individual cells. For
accuracy, the frame differencing algorithm depends on relatively stable environmental
lighting, and on having a stationary camera (unless it is the motion of the camera that is
being measured).
Example 2: Detecting presence (p. 557)
A technique called background subtraction makes it possible to detect the presence of
people or other objects in a scene, and to distinguish the pixels that belong to them from
those that do not. The technique operates by comparing each frame of video with a
stored image of the scene’s background, captured at a point in time when the scene was
known to be empty. For every pixel in the frame, the absolute difference is computed
between its color and that of its corresponding pixel in the stored background image;
areas that are very different from the background are likely to represent objects of
interest. Background subtraction works well in heterogeneous environments, but it is
very sensitive to changes in lighting conditions and depends on objects of interest
having sufﬁ cient contrast against the background scene.
Example 3: Detection through brightness thresholding (p. 559)
With the aid of controlled illumination (such as backlighting) and/or surface treatments
(such as high-contrast paints), it is possible to ensure that objects are considerably
darker or lighter than their surroundings. In such cases objects of interest can be
distinguished based on their brightness alone. To do this, each video pixel’s brightness is
compared to a threshold value and tagged accordingly as foreground or background.
550
Extension 3: Vision
Reas_09_519-710.indd Sec6:550
Reas_09_519-710.indd Sec6:550
5/16/07 12:33:16 PM
5/16/07 12:33:16 PM

Example 1. Detects motion by comparing each video frame to the previous frame. The change is visualized
and is calculated as a number.
Example 2. Detects the presence of someone or something in front of the camera by comparing each video
frame with a previously saved frame. The change is visualized and is calculated as a number.
Example 3. Distinguishes the silhouette of people or objects in each video frame by comparing each pixel
to a threshold value. The circle is ﬁ lled with white when it is within the silhouette.
Example 4. Tracks the brightest object in each video frame by calculating the brightest pixel. The light from
the ﬂ ashlight is the brightest element in the frame; therefore, the circle follows it.
551
Extension 3: Vision
Reas_09_519-710.indd Sec6:551
Reas_09_519-710.indd Sec6:551
5/16/07 12:33:16 PM
5/16/07 12:33:16 PM

Example 4: Brightness tracking (p. 560)
A rudimentary scheme for object tracking, ideal for tracking the location of a single
illuminated point (such as a ﬂ ashlight), ﬁ nds the location of the single brightest pixel in
every fresh frame of video. In this algorithm, the brightness of each pixel in the
incoming video frame is compared with the brightest value yet encountered in that
frame; if a pixel is brighter than the brightest value yet encountered, then the location
and brightness of that pixel are stored. After all of the pixels have been examined, then
the brightest location in the video frame is known. This technique relies on an
operational assumption that there is only one such object of interest. With trivial
modiﬁ cations, it can equivalently locate and track the darkest pixel in the scene, or track
multiple and differently colored objects.
Of course, many more software techniques exist, at every level of sophistication, for
detecting, recognizing, and interacting with people and other objects of interest. Each of
the tracking algorithms described above, for example, can be found in elaborated
versions that amend its various limitations. Other easy-to-implement algorithms can
compute speciﬁ c features of a tracked object, such as its area, center of mass, angular
orientation, compactness, edge pixels, and contour features such as corners and cavities.
On the other hand, some of the most difﬁ cult to implement algorithms, representing the
cutting edge of computer vision research today, are able (within limits) to recognize
unique people, track the orientation of a person’s gaze, or correctly identify facial
expressions. Pseudocodes, source codes, or ready-to-use implementations of all of these
techniques can be found on the Internet in excellent resources like Daniel Huber’s
Computer Vision Homepage, Robert Fisher’s HIPR (Hypermedia Image Processing
Reference), or in the software toolkits discussed on pages 554-555.
Computer vision in the physical world
Unlike the human eye and brain, no computer vision algorithm is completely general,
which is to say, able to perform its intended function given any possible video input.
Instead, each software tracking or detection algorithm is critically dependent on certain
unique assumptions about the real-world video scene it is expected to analyze. If any of
these expectations are not met, then the algorithm can produce poor or ambiguous
results or even fail altogether. For this reason, it is essential to design physical conditions
in tandem with the development of computer vision code, and to select the software
techniques that are most compatible with the available physical conditions.

Background subtraction and brightness thresholding, for example, can fail if the
people in the scene are too close in color or brightness to their surroundings. For these
algorithms to work well, it is greatly beneﬁ cial to prepare physical circumstances that
naturally emphasize the contrast between people and their environments. This can be
achieved with lighting situations that silhouette the people, or through the use of
specially colored costumes. The frame-differencing technique, likewise, fails to detect
people if they are stationary. It will therefore have very different degrees of success
552
Extension 3: Vision
Reas_09_519-710.indd Sec6:552
Reas_09_519-710.indd Sec6:552
5/16/07 12:33:17 PM
5/16/07 12:33:17 PM

detecting people in videos of ofﬁ ce waiting rooms compared with videos of the Tour de
France bicycle race.

A wealth of other methods exist for optimizing physical conditions in order to
enhance the robustness, accuracy, and effectiveness of computer vision software. Most
are geared toward ensuring a high-contrast, low-noise input image. Under low-light
conditions, for example, one of the most helpful such techniques is the use of infrared
(IR) illumination. Infrared, which is invisible to the human eye, can supplement the light
detected by conventional black-and-white security cameras. Using IR signiﬁ cantly
improves the signal-to-noise ratio of video captured in low-light circumstances and can
even permit vision systems to operate in (apparently) complete darkness. Another
physical optimization technique is the use of retroreﬂ ective marking materials, such as
those manufactured by 3M Corporation for safety uniforms. These materials are
remarkably efﬁ cient at reﬂ ecting light back toward their source of illumination and are
ideal aids for ensuring high-contrast video of tracked objects. If a small light is placed
coincident with the camera’s axis, objects with retroreﬂ ective markers will be detected
with tremendous reliability.

Finally, some of the most powerful physical optimizations for machine vision can be
made without intervening in the observed environment at all, through well-informed
selections of the imaging system’s camera, lens, and frame-grabber components. To take
one example, the use of a “telecentric” lens can signiﬁ cantly improve the performance of
certain kinds of shape-based or size-based object recognition algorithms. For this type of
lens, which has an effectively inﬁ nite focal length, magniﬁ cation is nearly independent
of object distance. As one manufacturer describes it, “an object moved from far away to
near the lens goes into and out of sharp focus, but its image size is constant. This
property is very important for gauging three-dimensional objects, or objects whose
distance from the lens is not known precisely.”7 Likewise, polarizing ﬁ lters offer a
simple, nonintrusive solution to another common problem in video systems, namely
glare from reﬂ ective surfaces. And a wide range of video cameras are available,
optimized for conditions like high-resolution capture, high-frame-rate capture, short
exposure times, dim light, ultraviolet light, and thermal imaging. It pays to research
imaging components carefully.

As we have seen, computer vision algorithms can be selected to negotiate best the
physical conditions presented by the world, and physical conditions can be modiﬁ ed to
be more easily legible to vision algorithms. But even the most sophisticated algorithms
and the highest-quality hardware cannot help us ﬁ nd meaning where there is none, or
track an object that cannot be described in code. It is therefore worth emphasizing that
some visual features contain more information about the world, and are also more
easily detected by the computer, than others. In designing systems to “see for us,” we
must not only become freshly awakened to the many things about the world that make
it visually intelligible to us, but also develop a keen intuition about their ease of
computability. The sun is the brightest point in the sky, and by its height also indicates
the time of day. The mouth cavity is easily segmentable as a dark region, and the
circularity of its shape is also closely linked to vowel sound. The pupils of the eyes emit
an easy-to-track infrared retroreﬂ ection, and they also indicate a person’s direction of
553
Extension 3: Vision
Reas_09_519-710.indd Sec6:553
Reas_09_519-710.indd Sec6:553
5/16/07 12:33:17 PM
5/16/07 12:33:17 PM

gaze. Simple frame differencing makes it easy to track motion in a video. The Suicide Box
(p. 548) uses this technique to dramatic effect.
Tools for computer vision
It can be a rewarding experience to implement machine vision techniques from scratch
using code such as the examples provided in this section. To make this possible, the only
requirement of one’s software development environment is that it should provide direct
read-access to the array of video pixels obtained by the computer’s frame-grabber.
Hopefully, the example algorithms discussed earlier illustrate that creating low-level
vision algorithms from ﬁ rst principles isn’t so hard. Of course, a vast range of
functionality can also be obtained immediately from readily available solutions. Some of
the most popular machine vision toolkits take the form of plug-ins or extension libraries
for commercial authoring environments geared toward the creation of interactive
media. Such plug-ins simplify the developer’s problem of connecting the results of the
vision-based analysis to the audio, visual, and textual affordances generally provided by
such authoring systems.

Many vision plug-ins have been developed for Max/MSP/Jitter, a visual
programming environment that is widely used by electronic musicians and VJs.
Originally developed at the Parisian IRCAM research center in the mid-1980s and now
marketed commercially by the California-based Cycling’74 company, this extensible
environment offers powerful control of (and connectivity between) MIDI devices, real-
time sound synthesis and analysis, OpenGL-based 3D graphics, video ﬁ ltering, network
communications, and serial control of hardware devices. The various computer vision
plug-ins for Max/MSP/Jitter, such as David Rokeby’s SoftVNS, Eric Singer’s Cyclops, and
Jean-Marc Pelletier’s CV.Jit, can be used to trigger any Max processes or control any
system parameters. Pelletier’s toolkit, which is the most feature-rich of the three, is also
the only one that is freeware. CV.Jit provides abstractions to assist users in tasks such as
image segmentation, shape and gesture recognition, and motion tracking, as well as
educational tools that outline the basics of computer vision techniques.

Some computer vision toolkits take the form of stand-alone applications and are
designed to communicate the results of their analyses to other environments (such as
Processing, Director, or Max) through protocols like MIDI, serial RS-232, UDP, or TCP/IP
networks. BigEye, developed by the STEIM (Studio for Electro-Instrumental Music) group
in Holland, is a simple and inexpensive example. BigEye can track up to 16 objects of
interest simultaneously, according to their brightness, color, and size. The software
allows for a simple mode of operation in which the user can quickly link MIDI messages
to many object parameters, such as position, speed, and size. Another example is the
powerful EyesWeb open platform, a free system developed at the University of Genoa.
Designed with a special focus on the analysis and processing of expressive gesture,
EyesWeb includes a collection of modules for real-time motion tracking and extraction of
movement cues from human full-body movement; a collection of modules for analysis of
occupation of 2D space; and a collection of modules for extraction of features from
554
Extension 3: Vision
Reas_09_519-710.indd Sec6:554
Reas_09_519-710.indd Sec6:554
5/16/07 12:33:17 PM
5/16/07 12:33:17 PM

trajectories in 2D space. EyesWeb’s extensive vision affordances make it highly
recommended for students.

The most sophisticated toolkits for computer vision generally demand greater
familiarity with digital signal processing, and they require developers to program in
compiled languages like C++ rather than languages like Java, Lingo, or Max. The Intel
Integrated Performance Primitives (IPP) library, for example, is among the most general
commercial solutions available for computers with Intel-based CPUs. The OpenCV
library, by contrast, is a free, open source toolkit with nearly similar capabilities and a
tighter focus on commonplace computer vision tasks. The capabilities of these tools, as
well as all of those mentioned above, are continually evolving.

Processing includes a basic video library that handles getting pixel information
from a camera or movie ﬁ le as demonstrated in the examples included with this text.
The computer vision capabilities of Processing are extended by libraries like Myron,
which handles video input and has basic image processing capabilities. Other libraries
connect Processing to EyesWeb and OpenCV. They can be found on the libraries page of
the Processing website: www.processing.org/reference/libraries.
Conclusion
Computer vision algorithms are increasingly used in interactive and other computer-
based artworks to track people’s activities. Techniques exist that can create real-time
reports about people’s identities, locations, gestural movements, facial expressions, gait
characteristics, gaze directions, and other attributes. Although the implementation of
some vision algorithms requires advanced understanding of image processing and
statistics, a number of widely used and highly effective techniques can be implemented
by novice programmers in as little as an afternoon. For artists and designers who are
familiar with popular multimedia authoring systems like Macromedia Director and
Max/MSP/Jitter, a wide range of free and commercial toolkits are also available that
provide ready access to more advanced vision functionalities.
Since the reliability of computer vision algorithms is limited according to the
quality of the incoming video scene and the deﬁ nition of a scene’s quality is determined
by the speciﬁ c algorithms that are used to analyze it, students approaching computer
vision for the ﬁ rst time are encouraged to apply as much effort to optimizing their
physical scenario as they do their software code. In many cases, a cleverly designed
physical environment can permit the tracking of phenomena that might otherwise
require much more sophisticated software. As computers and video hardware become
more available, and software-authoring tools continue to improve, we can expect to see
the use of computer vision techniques increasingly incorporated into media-art
education and into the creation of games, artworks, and many other applications.
Notes
1. http://mechanism.ucsd.edu/~bill/research/mercier/2ndlecture.pdf.
2. http://www.fundacion.telefonica.com/at/rlh/eproyecto.html.
555
Extension 3: Vision
Reas_09_519-710.indd Sec6:555
Reas_09_519-710.indd Sec6:555
5/16/07 12:33:17 PM
5/16/07 12:33:17 PM

3. http://homepage.mac.com/davidrokeby/sorting.html.
4. http://www.bureauit.org/sbox.
5. http://www.wired.com/news/culture/0,1284,64720,00.html.
6. http://www.christian-moeller.com.
7. http://www.mellesgriot.com/pdf/pg11-19.pdf.
Code
Video can be captured into Processing from USB cameras, IEEE 1394 cameras, or video
cards with composite or S-video input devices. The examples that follow assume you
already have a camera working with Processing. Before trying these examples, ﬁ rst get
the examples included with the Processing software to work. Sometimes you can plug a
camera into your computer and it will work immediately. Other times it’s a difﬁ cult
process involving trial-and-error changes. It depends on the operating system, the
camera, and how the computer is conﬁ gured. For the most up-to-date information, refer
to the Video reference on the Processing website: www.processing.org/reference/libraries.
Example 1: Detecting motion
// Quantify the amount of movement in the video frame using frame-differencing
import processing.video.*;
int numPixels;
int[] previousFrame;
Capture video;
void setup(){
size(640, 480); // Change size to 320 x 240 if too slow at 640 x 480
video = new Capture(this, width, height, 24);
numPixels = video.width * video.height;
// Create an array to store the previously captured frame
previousFrame = new int[numPixels];
}
void draw() {
if (video.available()) {
// When using video to manipulate the screen, use video.available() and
// video.read() inside the draw() method so that it's safe to draw to the screen
video.read(); // Read the new frame from the camera
video.loadPixels(); // Make its pixels[] array available
int movementSum = 0; // Amount of movement in the frame
loadPixels();

for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
556
Extension 3: Vision
Reas_09_519-710.indd Sec6:556
Reas_09_519-710.indd Sec6:556
5/16/07 12:33:17 PM
5/16/07 12:33:17 PM

color currColor = video.pixels[i];
color prevColor = previousFrame[i];
// Extract the red, green, and blue components from current pixel
int currR = (currColor >> 16) & 0xFF; // Like red(), but faster (see p. 673)
int currG = (currColor >> 8) & 0xFF;
int currB = currColor & 0xFF;
// Extract red, green, and blue components from previous pixel
int prevR = (prevColor >> 16) & 0xFF;
int prevG = (prevColor >> 8) & 0xFF;
int prevB = prevColor & 0xFF;
// Compute the difference of the red, green, and blue values
int diffR = abs(currR - prevR);
int diffG = abs(currG - prevG);
int diffB = abs(currB - prevB);
// Add these differences to the running tally
movementSum += diffR + diffG + diffB;
// Render the difference image to the screen
pixels[i] = color(diffR, diffG, diffB);
// The following line is much faster, but more confusing to read
//pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
// Save the current color into the 'previous' buffer
previousFrame[i] = currColor;
}

// To prevent flicker from frames that are all black (no movement),
// only update the screen if the image has changed.
if (movementSum > 0) {
updatePixels();
println(movementSum); // Print the total amount of movement to the console
}
}
}
Example 2: Detecting presence
// Detect the presence of people and objects in the frame using a simple
// background-subtraction technique. To initialize the background, press a key.
import processing.video.*;
int numPixels;
int[] backgroundPixels;
Capture video;
void setup() {
size(640, 480); // Change size to 320 x 240 if too slow at 640 x 480
video = new Capture(this, width, height, 24);
numPixels = video.width * video.height;
557
Extension 3: Vision
Reas_09_519-710.indd Sec6:557
Reas_09_519-710.indd Sec6:557
5/16/07 12:33:18 PM
5/16/07 12:33:18 PM

// Create array to store the background image
backgroundPixels = new int[numPixels];
// Make the pixels[] array available for direct manipulation
loadPixels();
}
void draw() {
if (video.available()) {
video.read(); // Read a new video frame
video.loadPixels(); // Make the pixels of video available

// Difference between the current frame and the stored background
int presenceSum = 0;
for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
// Fetch the current color in that location, and also the color
// of the background in that spot
color currColor = video.pixels[i];
color bkgdColor = backgroundPixels[i];

// Extract the red, green, and blue components of the current pixel’s color
int currR = (currColor >> 16) & 0xFF;
int currG = (currColor >> 8) & 0xFF;
int currB = currColor & 0xFF;
// Extract the red, green, and blue components of the background pixel’s color
int bkgdR = (bkgdColor >> 16) & 0xFF;
int bkgdG = (bkgdColor >> 8) & 0xFF;
int bkgdB = bkgdColor & 0xFF;
// Compute the difference of the red, green, and blue values
int diffR = abs(currR - bkgdR);
int diffG = abs(currG - bkgdG);
int diffB = abs(currB - bkgdB);
// Add these differences to the running tally
presenceSum += diffR + diffG + diffB;
// Render the difference image to the screen
pixels[i] = color(diffR, diffG, diffB);
// The following line does the same thing much faster, but is more technical
//pixels[i] = 0xFF000000 | (diffR << 16) | (diffG << 8) | diffB;
}
updatePixels(); // Notify that the pixels[] array has changed
println(presenceSum); // Print out the total amount of movement
}
}
// When a key is pressed, capture the background image into the backgroundPixels
// buffer by copying each of the current frame’s pixels into it.
void keyPressed() {
video.loadPixels();
arraycopy(video.pixels, backgroundPixels);
}
558
Extension 3: Vision
Reas_09_519-710.indd Sec6:558
Reas_09_519-710.indd Sec6:558
5/16/07 12:33:18 PM
5/16/07 12:33:18 PM

Example 3: Detection through brightness thresholding
// Determines whether a test location (such as the cursor) is contained within
// the silhouette of a dark object
import processing.video.*;
color black = color(0);
color white = color(255);
int numPixels;
Capture video;
void setup() {
size(640, 480); // Change size to 320 x 240 if too slow at 640 x 480
strokeWeight(5);
video = new Capture(this, width, height, 24);
numPixels = video.width * video.height;
noCursor();
smooth();
}
void draw() {
if (video.available()) {
video.read();
video.loadPixels();
int threshold = 127; // Set the threshold value
float pixelBrightness; // Declare variable to store a pixel's color
// Turn each pixel in the video frame black or white depending on its brightness
loadPixels();
for (int i = 0; i < numPixels; i++) {
pixelBrightness = brightness(video.pixels[i]);
if (pixelBrightness > threshold) { // If the pixel is brighter than the
pixels[i] = white; // threshold value, make it white
} else { // Otherwise,
pixels[i] = black; // make it black
}
}
updatePixels();
// Test a location to see where it is contained. Fetch the pixel at the test
// location (the cursor), and compute its brightness
int testValue = get(mouseX, mouseY);
float testBrightness = brightness(testValue);
if (testBrightness > threshold) { // If the test location is brighter than
fill(black); // the threshold set the fill to black
} else { // Otherwise,
fill(white); // set the fill to white
}
ellipse(mouseX, mouseY, 20, 20);
}
}
559
Extension 3: Vision
Reas_09_519-710.indd Sec6:559
Reas_09_519-710.indd Sec6:559
5/16/07 12:33:18 PM
5/16/07 12:33:18 PM

Example 4: Brightness tracking
// Tracks the brightest pixel in a live video signal
import processing.video.*;
Capture video;
void setup(){
size(640, 480); // Change size to 320 x 240 if too slow at 640 x 480
video = new Capture(this, width, height, 30);
noStroke();
smooth();
}
void draw() {
if (video.available()) {
video.read();
image(video, 0, 0, width, height); // Draw the webcam video onto the screen
int brightestX = 0; // X-coordinate of the brightest video pixel
int brightestY = 0; // Y-coordinate of the brightest video pixel
float brightestValue = 0; // Brightness of the brightest video pixel
// Search for the brightest pixel: For each row of pixels in the video image and
// for each pixel in the yth row, compute each pixel's index in the video
video.loadPixels();
int index = 0;
for (int y = 0; y < video.height; y++) {
for (int x = 0; x < video.width; x++) {
// Get the color stored in the pixel
int pixelValue = video.pixels[index];
// Determine the brightness of the pixel
float pixelBrightness = brightness(pixelValue);
// If that value is brighter than any previous, then store the
// brightness of that pixel, as well as its (x,y) location
if (pixelBrightness > brightestValue){
brightestValue = pixelBrightness;
brightestY = y;
brightestX = x;
}
index++;
}
}
// Draw a large, yellow circle at the brightest pixel
fill(255, 204, 0, 128);
ellipse(brightestX, brightestY, 200, 200);
}
}
560
Extension 3: Vision
Reas_09_519-710.indd Sec6:560
Reas_09_519-710.indd Sec6:560
5/16/07 12:33:18 PM
5/16/07 12:33:18 PM

Resources
Computer vision software toolkits
Camurri, Antonio, et al. Eyesweb. Vision-oriented software development environment. http://www.eyesweb.org.
Cycling’74 Inc. Max/MSP/Jitter. Graphic software development environment. http://www.cycling74.com.
Davies, Bob, et al. OpenCV. Open source computer vision library. http://sourceforge.net/projects/opencvlibrary.
Nimoy, Joshua. Myron (WebCamXtra). Library (plug-in) for Macromedia Director and Processing.
http://webcamxtra.sourceforge.net.
Pelletier, Jean-Marc. CV.Jit. Extension library for Max/MSP/Jitter. http://www.iamas.ac.jp/~jovan02/cv.
Rokeby, David. SoftVNS. Extension library for Max/MSP/Jitter.
http://homepage.mac.com/davidrokeby/softVNS.html.
Singer, Eric. Cyclops. Extension library for Max/MSP/Jitter. http://www.cycling74.com/products/cyclops.html.
STEIM (Studio for Electro-Instrumental Music). BigEye. Video analysis software. http://www.steim.org
Texts and artworks
Bureau of Inverse Technology. Suicide Box. http://www.bureauit.org/sbox.
Bechtel, William. The Cardinal Mercier Lectures at the Catholic University of Louvain. Lecture 2, An Exemplar.
Neural Mechanism: The Brain’s Visual Processing System. 2003, p.1.
http://mechanism.ucsd.edu/~bill/research/mercier/2ndlecture.pdf.
Fisher, Robert, et. al. HIPR (The Hypermedia Image Processing Reference).
http://homepages.inf.ed.ac.uk/rbf/HIPR2/index.htm.
Fisher, Robert, et al. CVonline: The Evolving, Distributed, Non-Proprietary, On-Line Compendium of

Computer Vision. http://homepages.inf.ed.ac.uk/rbf/CVonline.
Huber, Daniel, et al. The Computer Vision Homepage. http://www-2.cs.cmu.edu/~cil/vision.html.
Krueger, Myron. Artiﬁ cial Reality II. Addison-Wesley Professional, 1991.
Levin, Golan and Lieberman, Zachary. Messa di Voce. Interactive installation, 2003.
http://www.tmema.org/messa.
Levin, Golan, and Zachary Lieberman. “In-Situ Speech Visualization in Real-Time Interactive Installation

and
Performance.”
Proceedings of the Third International Symposium on Non-Photorealistic Animation and

Rendering. Annecy, France, June 7-9, 2004. http://www.ﬂ ong.com/writings/pdf/messa_NPAR_2004_150dpi.pdf.
Lozano-Hemmer, Rafael. Standards and Double Standards. Interactive installation.
http://www.fundacion.telefonica.com/at/rlh/eproyecto.html.
Melles Griot Corporation. Machine Vision Lens Fundamentals. http://www.mellesgriot.com/pdf/pg11-19.pdf.
Möller, Christian. Cheese. Installation artwork, 2003. http://www.christian-moeller.com.
Rokeby, David. Sorting Daemon. Computer-based installation, 2003.
http://homepage.mac.com/davidrokeby/sorting.html.
Shachtman, Noah. “Tech and Art Mix at RNC Protest.” Wired News, 27 August 2004.
http://www.wired.com/news/culture/0,1284,64720,00.html.
Sparacino, Flavia. “(Some) computer vision based interfaces for interactive art and entertainment installations.”

INTER_FACE Body Boundaries, issue edited by Emanuele Quinz. Anomalie no. 2. Anomos, 2001.
http://www.sensingplaces.com/papers/Flavia_isea2000.pdf.
561
Extension 3: Vision
Reas_09_519-710.indd Sec6:561
Reas_09_519-710.indd Sec6:561
5/16/07 12:33:18 PM
5/16/07 12:33:18 PM

Code Index
This index contains all of
abs(), 241
ellipse(), 30
the Processing language
alpha(), 338
ellipseMode(), 34
ambient(), 533
else, 55
elements introduced
ambientLight(), 533
else if, 56
within this book. The
append(), 309
endRaw(), 531
page numbers refer to
arc(), 124
endRecord(), 607
the ﬁ rst use.
arraycopy, 310
endShape(), 69
Array, 301
exit(), 422
length, 304
expand(), 309
! (logical NOT), 57
atan2(), 243
extends, 456
!= (inequality), 52
% (modulo), 45
background(), 31
false, 38
&& (logical AND), 57
beginRaw(), 531
fill(), 32
() (parentheses)
beginRecord(), 607
filter(), 347
for functions, 18
beginShape(), 69
float, 37
for precedence, 47
bezier(), 30
float(), 106
* (multiply), 44
bezierVertex(), 75
floor(), 49
*= (multiply assign), 49
blend(), 351
for, 61
+ (addition), 43
blendColor(), 352
frameCount, 173
++ (increment), 48
blue(), 337
frameRate(), 173
+= (add assign), 48
boolean, 38
, (comma), 18
boolean(), 106
get(), 321
- (minus), 44
brightness(), 338
green(), 337
-- (decrement), 48
byte, 38
-= (subtract assign), 48
byte(), 106
HALF_PI, 117
. (dot), 107
height, 40
/ (divide), 44
camera(), 531
hour(), 245
/= (divide assign), 49
Capture, 556
HSB, 89
/* */ (comment), 18
ceil(), 49
hue(), 338
// (comment), 17
char, 38, 102
; (semicolon), 19
char(), 106
if, 53
< (less than), 51
class, 395
image(), 96
<= (less than or
Client, 567
int, 37
equal to), 52
color, 38, 89
int(), 107
= (assign), 38
color(), 89
key, 225
== (equality), 52
colorMode(), 91
keyCode, 227
for String objects, 109
constrain(), 237
keyPressed, 224
> (greater than), 51
copy(), 353
keyPressed(), 232
>= (greater than
cos(), 118
keyReleased(), 232
or equal to), 52
createGraphics(), 614
[] (array access), 301
createImage(), 362
lerp(), 81
2D arrays, 312
createWriter(), 423
lightSpecular(), 536
arrays of objects, 406
cursor(), 213
line(), 27
{} (braces), 53
curveVertex(), 74
loadFont(), 112
and variable scope, 178
loadImage(), 96
|| (logical OR), 57
day(), 249
loadPixels(), 356
# (hex color), 93
degrees(), 117
loadStrings(), 428
directionalLight(), 536
loop(), 235
dist(), 238
draw(), 173
703
Reas_09_519-710.indd Sec6:703
Reas_09_519-710.indd Sec6:703
5/16/07 12:33:53 PM
5/16/07 12:33:53 PM

map(), 81
quad(), 29
text(), 112
mask(), 354
QUARTER_PI, 117
textAlign(), 115
max(), 50
textFont(), 112
millis(), 248
radians(), 117
textLeading(), 115
min(), 50
random(), 127
textSize(), 114
minute(), 245
randomSeed(), 129
texture(), 536
month(), 249
rect(), 29
textWidth(), 116
mouseButton, 212
rectMode(), 34
tint(), 97
mouseDragged(), 229
red(), 337
translate(), 133
mouseMoved(), 229
redraw(), 235
triangle(), 27
mousePressed, 212
return, 194
true, 38
mousePressed(), 229
RGB, 89
TWO_PI, 117
mouseReleased(), 229
rotate(), 137
mouseX, 205
round(), 50
updatePixels(), 356
mouseY, 205
saturation(), 338
vertex(), 69
new
save(), 368
void, 187
for arrays, 303
saveFrame(), 369
for objects, 399
saveStrings(), 422
width, 40
nf(), 422
scale(), 138
noCursor(), 213
second(), 245
year(), 249
noFill(), 33
Server, 567
noise(), 130
set(), 324
noiseSeed(), 131
setup(), 177
noLoop(), 178
shorten(), 309
norm(), 80
sin(), 118
noSmooth(), 33
size(), 24
noStroke(), 33
with P3D, 528
noTint(), 97
with OPENGL, 528
with PDF, 607
Object, 107, 395
smooth(), 33
specular(), 536
PFont, 112
split(), 429
PI, 117
splitTokens(), 430
PImage, 96
spotLight(), 536
pixels[], 356
sq(), 79
pmouseX, 208
sqrt(), 79
pmouseY, 208
str(), 107
point(), 25
String, 103
pointLight(), 536
length(), 108
popMatrix(), 134
endsWith(), 108
pow(), 80
equals(), 109
print(), 20
startsWith(), 108
println(), 20
substring(), 109
PrintWriter, 423
toCharArray(), 108
close(), 423
toLowerCase(), 109
flush(), 423
toUpperCase(), 109
println(), 424
stroke(), 32
pushMatrix(), 134
strokeCap(), 33
strokeJoin(), 33
strokeWeight(), 33
super, 456
704
Code Index
Reas_09_519-710.indd Sec6:704
Reas_09_519-710.indd Sec6:704
5/16/07 12:33:53 PM
5/16/07 12:33:53 PM

Index
This index contains mostly
Apple Computer, 3, 111, 537,
Bass, Saul, 327
people, software, artwork,
585,
699
Baumgärtel, Tilman, 564
Apple Logic Audio, 503, 591
Bauhaus, 149
and programming
Apple Mac G3, 383
BBC Acorn Archimedes, 264
languages. For topics,
Apple Mac G4, 383
Beach Culture, 605
see the table of contents
Apple Macintosh (Mac), 9–11, 95,
Beethoven, Ludwig van, 581
(pp. vii–xvii); for code, see
111–112, 169, 205, 227, 367, 383,
BEFLIX, 315, 681
521, 568–569, 574, 604, 639, 665,
Bell Laboratories, 315, 580–581, 604
the Code Index.
682,
685
Bentley Systems

Apple Mac Mini, 639
GenerativeComponents,
537
1:1 (Jevbratt), 566
Apple Mac OS, 264, 435, 665–666,
Berliner, Emile, 579
3M Corporation, 553
681
Berlow, David, 170
3 Stoppages Étalon (Duchamp), 127
Apple Mac OS X, 16, 170, 435, 645,
Bernard (a k a Flip 1), 508
7–11 Email list, 563
649,
684
BIAS Peak, 591
Apple QuickTime, 367, 383–384,

BigEye, 554
AARON, 218
387–388
Binary Runtime Environment for
Aesthetics and Computation

AppleScript, 681
Wireless (BREW), 625
Group (ACG), xxiii, 682
Arduino, 521, 633, 640, 641, 645–646,
Binary space partition (BSP), 527
Achituv, Romy, 549
648–649, 681, 685
Binder, Maurice, 327
ActionScript, 158, 166, 522–523, 565,
Arp, Jean, 127
bitforms gallery, 164, 166–167, 525,
680–681, 686–687, 689, 691
Ars Electronica Festival, 618
547, 603, 633
Adair, Sandra, 384
ART+COM, 498
Bittorent, 571
Adobe, 4, 169, 683
ASCII (American Standard Code for
Blackwell, Lewis, 605
Adobe After Effects, 166, 327,
Information Interchange), 102–103,
Blender, 276, 576
379,
387
226–227, 549, 565, 664–668, 670,
Blinkenlights (Chaos Computer

Adobe Flash, 157–158, 165–166,

691,
699
Club),
618
267–268, 275, 278, 327, 436,
Athena, 387
Blonk, Jaap, 511
564–565, 624, 629, 642,
ATI, 537
Bluetooth, 619, 621–622, 624, 641,
680–681, 683, 686, 701
AT&T/Bell , 564
645,
683
Adobe Flash Lite, 624, 681
Audacity, 591
Blyth, Steven, 512
Adobe Garamond (font), 112
AutoCAD, 217, 529, 537
Boids (Reynolds), 295, 473,

Adobe Illustrator, xxiii, 30, 77, 166,
Autodesk 3ds Max, 268, 276,

475,
497
143, 217, 271, 273, 607–608, 683
391–392,
537
Boole, George, 38, 61, 669
Adobe Photoshop, xxiii, 95, 166,
AutoDesk Revit, 537
Boolean algebra, 38
268, 276, 347, 355, 360, 384,

AutoLISP, 522, 681
Boulez, Pierre, 581
387–388, 391–392, 607–608,

Autonomedia, 564
Braitenberg, Valentino, 473–474
611,
683
Avid/Digidesign Pro Tools, 591
Brakhage, Stan, 413
Adobe Premiere, 391–392
AVR (Atmel), 640
Brecht, Bertolt, 564
Adobe Streamline, 166
awk, 517, 684
Brooklyn Academy of Music

AAC (Advanced Audio Coding),

(BAM),
515–516
585
Babbitt, Milton, 580–581
Brown, Robert, 295
AIFF (Audio Interchange File

Bach, J. S., 581
Brownian motion, 295
Format), 585–586, 699
Bailey, Chris, 581
Brunelleschi, Filippo, 525
Aldus PageMaker, 605
Balkin, Amy, 267
Bunting, Heath, 563–564
Alexander, Ryan, 380
Baran, Paul, 564
Bureau of Inverse Technology,

Alias Maya, 379, 387–388, 537, 680
Barr, Alfred, 291
548,
634
AltSys, 170
Barragán, Hernando, 633
Burke, Phil, 592
Andrade, Laura Hernandez, 4
BASIC, xxiii, xxiv, 152, 264, 522,
Burton, Ed, 263–264, 413, 499
Apple IIe, xxiii
604–605, 640, 642, 681
Byrne, David, 581
Apple Audio Units (AU), 591
BASIC Stamp 2 (Parallax), 640
BasicX–24 (NetMedia), 642
705
Reas_09_519-710.indd Sec6:705
Reas_09_519-710.indd Sec6:705
5/16/07 12:33:53 PM
5/16/07 12:33:53 PM

C, 7, 264, 515–517, 522–523, 592, 640,
Cosic, Vic 563–564
Electronic Arts, 585
642, 682–685, 693, 697
Costabile, Sue (SUE.C), 503–504
ELIZA, 101
C++, 264, 271, 383, 507–508, 511–512,
Craighead, Alison, 618
Emacs, 516
515–516, 522–523, 555, 592, 599,

Crawford, David, 316
Emigre, 605
640, 679, 681–682
Crystal Castle, 525
End of Print, The (Blackwell), 605
CAD (computer–aided drawing

Csikszentmihályi, Chris, 507–508,
Endless Forest, The (Tale of Tales),
software), 217, 526, 537–538
634
274–277
Cage, John, 127, 579
CSIRAC, 580
Engelbart, Douglas, 205
CalArts School of Art, 564
Csuri, Charles, 217
Eno, Brian, 581
California Institute of Technology
Cuba, Larry, 1, 315
Enron, 268
(Caltech), 388, 549
Cullen, Mathew, 379–380
Enzensberger, Hans Magnus, 564
Cameron, Dan, 387
CV.Jit, 554
EPS, 606
Campbell, Jim, 549
Cybernetic Serendipity, 101, 603
Euler’s method, 7, 494
Carmack, John ,525
Cycling ’74, 554, 592
Every Icon (Simon), 565
Carnegie Mellon University, xxi
Cyclops, 554
Evolved Virtual Creatures (Sims),
Carnivore, 566, 568–569
295
Carson, David, 605
Dada, 149–150
Experiments in Art and

Cascading Style Sheets (CSS), 93
Davies, Char, 526
Technology (E.A.T.), 633
CCRMA Synthesis ToolKit (STK),
Davis, Joshua, 564–565
Extend Script, 683
592
Deck, Barry, 112
Eye magazine, 605
Chang, Zai, 6
Deleuze and Guattari, 564
Eye Catching (Steinkamp),
Cheese (Möller), 549
Delvoye, Wim, 461
386–389
Cho, Peter, 257, 327
De Mol, Gerry, 275
EyesWeb, 554–555
CIA World Fact Book, 267
Design By Numbers (DBN), xxiv,
EZIO (NIQ), 642
Citron, Jack, 315
552–523,
682
CityPoems, 617, 624
Designers Republic, The, 605
Feingold, Ken, 633
ChucK, 592, 682
Dextro, 316
Ferro, Pablo, 327
Cloaca (Delvoye), 461
Dialtones (Levin et al.), 617–618
Final Cut Pro (FCP), 383, 503
Clash of the Titans, 387
Digidesign, 587, 591
Final Scratch, 507
Close, Chuck, 606
Dine, Jim, 606
Fischinger, Oskar, 413
CODE (Petzold), 648
DJ I, Robot Sound System, 506–509
Fisher, Robert, 552
Cohen, Harold, 218
Dodgeball, 617, 624
Flake, Gary William, 469
Columbia–Princeton Electronic

Domain Name System (DNS), 566
Flight404.com, 6
Music Center, 580
DrawBot, 169, 682, 684
Flight Simulator, 525
Commodore C–64, 272
Drawing with Computers (Wilson),
Foldes, Peter, 315
Commodore VC–20, 272
152, 217, 604
FontLab, 170
Common Lisp, 592
Drawn (Lieberman), 413
Fontographer, 170
Complexification.net, 6, 157
DuBois, R. Luke, 579
Fortran, 522
Computational Beauty of Nature,
Duchamp, Marcel, 127, 633
Fractal.Invaders (Tarbell), 156–159
The
(Flake), 469
Dunne, Tony, 634
Franceschini, Amy, 267
Computers and Automation, 603
Dürer, Albrecht 525, 612
Franke, Uli, 260, 271
Computer Clubhouse, 680
DXF, 520, 529–531
Free Radicals, 413
Computer Lib / Dream Machines
Dynabook, 3
Friendster, 617
(Nelson),
3
Fourier, Jean-Baptiste-Joseph, 584
Computer Vision Homepage

Eagle, 272
Fourier transform, 585, 588, 590
(Huber),
552
écal (école cantonale d’art de
Futurist, 279, 579
Coniglio, Mark, 512
Lausanne), 271
“Constituents for a Theory of the
Eclipse, 571, 625
Gabo, Nam, 633
Media” (Enzensberger), 564
ECMAScript, 681, 683
Galloway, Alexander R., 563
Conway, John, 461, 463, 467–468,
Edelweiss Series (Maywa Denki),
Game of Life, 461, 463, 465–466,

475
634
468,
475
Cook, Perry, 592
Edgerton, Harold, 295
Gardner, Martin, 461, 463
Cooper, Muriel , 327
Edison, Thomas, 579
Garton, Brad, 581
Cope, David, 581
Eighth Istanbul Biennial, 387
Gerhardt, Joseph, 391–392
CorelDRAW, 608
Eimart, Herbert, 580
Gestalt psychology, 584
706
Index
Reas_09_519-710.indd Sec6:706
Reas_09_519-710.indd Sec6:706
5/16/07 12:33:54 PM
5/16/07 12:33:54 PM

GIF, 95–96, 98–99, 421, 700–701
Huber, Daniel, 552
JavaScript, 268, 271, 522, 624, 680,
Girroir, Jonathan, 506–509
Huff, Kenneth A., 606
681,
683
Google, 568, 617
Hypermedia Image Processing
Java Virtual Machine (JVM), 680
GPS (Global positioning system),
Reference (HIPR), 552
Jeremijenko, Natalie, 548
619,
621
HyperTalk, 522
Jevbratt, Lisa, 566
Graffiti, 223
jMax, 592
GRASS, 681
IANA, 569
Jodi, 563–566
Groeneveld, Dirk, 333
IBM, 315, 537, 580, 585, 604,
Jones, Crispin, 634
GNU Image Manipulation

620,
702
Jones, Ronald, 275
Program (GIMP), 95, 347, 355,
IC (integrated circuit), 639, 647
Jonzun Crew, 508
607–608
I–Cube X (Infusion Systems), 642
JPEG, 95–96, 162, 421, 606, 611,

GNU Public License (GPL), 271
IEEE 1394 camera, 556
620,
701
Gnutella, 566, 571
If/Then (Feingold), 633
JSyn (Java Synthesis), 592
GPU (graphics processing unit),
Igarashi, Takeo, 538
Julesz, Bela, 603
536–537
Igoe, Tom, 635, 648
Graphomat Z64 (Zuse), 603
Ikarus M, 170
Kay, Alan, 3
Greenwold, Simon, 525
Incredibles, The, 315
Kim, Tai-kyung, 5
Greie, Antye (AGF), 503–504
Internet Explorer, 565
Kimura, Mari, 582
Grzinic, Marina, 563
Internet Protocol (IP), 566–567,

King’s Quest, 525
GUI (Graphical user interface),

569, 589, 645
Klee, Paul, 217
435–436, 448, 450, 499, 604,

Impressionist, 279
Knowlton, Kenneth C., 315, 604
634, 679–680, 683, 685, 700
Inaudible Cities: Part One

Krueger, Myron, 255, 512, 547
Gutenberg, Johannes, 111
(Semiconductor),
392
Kusaite, Lina, 275
Gutenberg archive, 433
InDesign, 683
Kuwakubo, Ryota, 634
Guttmann, Newmann, 580
Infrared, 553, 621
Gysin, Andreas, 373
Inge, Leif, 581
La Barbara, Joan, 511
Inkscape, 77, 607–608
Langton, Chris, 469, 471
Hall, Grady, 379
Installation (Greenwold), 526
Putto8 2.2.2.2 (Rees), 524, 526
Handel, George Frideric, 581
Institute of Contemporary Arts

LaserWriter, 111, 604
Hansen, Mark, 515–516, 634
(ICA), 101, 522
Lee, Soo-jeong, 5
Harmon, Leon, 604
Intel Integrated Performance

Led Zeppelin, 161
Harvard University, xxi
Primitives (IPP), 512, 555
Legible City, The (Shaw,

Harvey, Auriea, 275
Interaction Design Institute Ivrea
Groeneveld),
333
Hewlett-Packard (HP), 604, 610
(IDII), xxi, 634
Lehni, Jürg, 260, 271–273
Hawkinson, Tim, 633
i|o 360°, 565
Leibniz, Gottfried Wilhelm, 61
Hawtin, Richie, 507
I/O/D 4 (“The Webstalker”), 566
Letterscapes (Cho), 327
Hébert, Jean-Pierre, 217, 606
IRCAM, 554, 581, 592
LettError, 111, 168–170, 605
Hektor (Lehni, Franke), 260,

Ishii, Hiroshi, 634
Levin, Golan, 259, 333, 511–512,
270–273
Ishizaki, Suguru, 327
547,
617–618
Henry, John, 507
ISO 216 standard, 611
Lewis, George, 582
Henry, Pierre, 580
Iwai, Toshio, 512, 549
LeWitt, Sol, 217
Hiller, Lejaren , 581
Li, Francis, 617
Hoefler, Jonathan, 112
James, Richard (Aphex Twin), 582
Lia, 316, 496
Hodgin, Robert, 6, 692
Jarman, Ruth, 391–392
Lialina, Olia, 563–564
Hokusai, 612
Java, 7, 9–11, 146, 161–162, 263–264,
Licko, Zuzana, 112, 605
Hongik University, 5
271, 499, 521–523, 528, 555, 564–
Lieberman, Zachary, 413,
Hong, Leon, 5, 375
565, 571, 574, 592, 622, 625– 626,
512–512,
547
Hooke’s law, 263, 487
642, 663, 673, 677, 679–683,

Lifestreams, 425–426
Howard Wise gallery, 603
686–690,
699–700
Limewire, 571
HTML (HyperText Markup
Java 2 Micro Edition (J2ME), 625
Lingo, 522–523, 555, 565, 683,
Language), 9–11, 93, 268, 427, 549,
Java applet, 9–11, 264, 521, 656, 657,
686–687, 689, 691
564–565, 568–569, 621, 624,

675,
699
Linklater, Richard, 383
665–666,
684
Java Archive (JAR), 10–11, 700
Linotype, 111
HTTP (Hypertext Transfer

Java Core API, 271
Linux, 4, 9–11, 508, 521, 568–569,

Protocol), 567–569, 623
625, 645, 649
707
Index
Reas_09_519-710.indd Sec6:707
Reas_09_519-710.indd Sec6:707
5/16/07 12:33:54 PM
5/16/07 12:33:54 PM

Listening Post (Rubin, Hansen),

Metrowerks Codewarrior, 512
NetBeans, 625
514–517
Microsoft, 4, 111, 169, 436, 508,
Netscape Navigator, 565, 683
LISP, 101
525, 537, 585, 702
Newton, Isaac, 477, 488
LiveScript, 683
Microsoft Direct3D, 537
New York University (NYU), 6, 634
Local area network (LAN),
Microsoft Visual Basic, 436
New York Times, The, 150
568–569
Microsoft Windows, 9, 11, 264, 367,
Ngan, William, 497
Logo, xxiii, 2, 217, 522, 681
421, 435–436, 511, 521, 568, 625,

Nimoy, Josh, 512
Lovink, Geert, 564
645, 649, 665–666, 685
Noll, A. Michael , 217, 603
Lozano-Hemmer, Rafael, 546, 548
MIDI (Musical Instrument Digital
Nokia, 517, 618–619, 625
Lucent Technologies, 515
Interface) 162, 554, 588–589,

Nmap, 569
Lucier, Alvin, 590
591–592, 618, 621, 623, 642, 645,
NSA (National Security Agency),
Luening, Otto, 580
683,
685
268
Lüsebrink, Dirk, 549
Mignonneau, Laurent, 549
NTNTNT (Cal Arts), 564
Lye, Len, 413
MIME, 623
NTSC, 367
Mims, Forest M., III, 648
NTT DoCoMo’s i–Mode, 624
Machine Art exhibition, 291, 633
Mini-Epoch Series, The

Nuendo, Steinberg, 591
Machine Perception Laboratories,
(Semiconductor),
390–393
null, 40, 701
549
Mini Movies (AGF+SUE.C), 500,

NURBS (Non-uniform Rational
MacMurtrie, Chico, 549
502–505
B-splines),
526
Macromedia Director, 166,
Minitasking (Schoenerwissen/

nVidia, 537
387–388, 554–555, 642, 683, 686
OfCD), 562, 566
Nyquist theorem, 585
Maeda, John, xix, xxiii, xxiv, 3, 5,
Minsky, Marvin, 547
158, 333, 564, 606, 682
MIT Media Laboratory, xxiii, 327,
OBJ, 529–531
Malka, Ariel, 372
634, 680, 682, 702
Objectivity Engine, The (Paterson),
Makela, P. Scott, 605
MixViews, 591
164–167
Mandelbrot, Benoit, 153
MP3, 162, 421, 585, 621, 623
Oliveros, Pauline, 582
Manovich, Lev, 565
MPEG–7, 549
Olsson, Krister, 589
Marble Madness, 525
Mobile Processing, 521, 622–626,
Once-Upon-A-Forest (Davis), 564
Marconi, Guglielmo, 579
683
On, Josh, 267–268
Marey, Étienne-Jules, 295
Mohr, Manfred, 217, 602, 606
oN-Line System (NLS), 205
Mark of the Unicorn Digital

Möller, Christian, 549
OpenCV, 512, 555
Performer,
591
Moore, F. Richard, 592
OpenGL, 512, 520, 528, 531, 537,
Markov chain, 581
Mophun, 625
554,
684
Marx, Karl, 267–268
Morisawa, 605
Open source, 4, 268, 271, 512, 521,

Massachusetts Institute of

Motion Theory, 378–381
555, 591, 625–626, 640, 684
Technology (MIT), xix, xxiii,

MTV, 384
OpenType, 111, 169
xxiv, 327, 634, 680, 682, 693, 695
[murmur], 618
Oracle database, 264
Masterman, Margaret, 101
Museum of Modern Art, The

OSC (Open Sound Control),
Mathews, Max, 580, 586, 591, 683
(MOMA), 291, 633
516–517, 571, 589
MATLAB, 522
MUSIC, 580, 591
oscP5 (Schlegel), 571
Max/MSP/Jitter, 2, 503–504,
Musique concrète, 580–581
Osmose (Davies), 526
515–517, 522, 554–555, 571, 580,

Muybridge, Eadweard, 295, 373
O’Sullivan, Dan, 635, 648
592, 642, 683–685
Myron, 555
Oswald, John, 581
Maya Embedded Language (MEL),
MySQL, 267–268
Owens, Matt, 565
680,
683
Myst, 525
Maywa Denki, 634
Pad, 435
McCarthy, John, 101
Nakamura, Yugo, 565
Paik, Nam June, 633
McCartney, James, 592
Nake, Frieder, 217, 603
PAL, 367
McCay, Winsor, 315
Napier, Mark, 566
Palm Pilot, 223, 625
McLaren, Norman, 413
Napster, 507, 571
Palm OS, 625
Medusa, 387
Nees, Georg, 217, 603
Panasonic, 625
MEL, 680, 683
Nelson, Ted, 3
Papert, Seymour, 2, 217
Mendel, Lucy, 507
“net.art”, 563–564
Parallax, 640
Messa di Voce (Tmema et al.),
net.art (Baumgärtel), 564
Parallel Development, 516
510–513,
547
net.art 2.0 (Baumgärtel), 564
Pascal, 522
708
Index
Reas_09_519-710.indd Sec6:708
Reas_09_519-710.indd Sec6:708
5/16/07 12:33:54 PM
5/16/07 12:33:54 PM

Paterson, James, 165–166, 316,
Q*bert, 525
Schlegel, Andreas, 498, 571
565,
606
Quartz Composer, 684
Schmidt, Karsten (a k a toxi), 4, 518
Paul, Les, 580
Qualcomm, 625
Schoenerwissen/OfCD, 562
PBASIC, 642, 681
Quest3D, 275–276
Schöffer, Nicolas, 633
PC, 10, 227, 388, 625, 665, 682
Schumacher, Michael, 582
PCB (printed circuit board),
R, 515, 517
Schwartz, Lillian, 315
639,
640
Raby, Fiona, 634
Scientific American, 461, 463
PCM (pulse–code modulation),

Radial, 503–504
Scratch, 680
585–586, 699, 702
RAM, 701
Screen Series (Snibbe), 549
PDF, 520, 606–608, 682
RandomFont Beowolf (LettError),
Scriptographer (Lehni, Franke),

Pelletier, Jean-Marc, 554
111, 168–170, 605
270–273,
683
Penny, Simon, 549
Rauschenberg, Robert, 606
Seawright, James, 633
Perl, 146, 515–517, 522–523, 565, 571,
Ray Gun, 605
sed, 684
681,
684
Razorfish, 565
Semiconductor, 390–393, 646
Perlin, Ken, 130
RCA Mark II Sound Synthesizer,
Sessions, Roger 580
Personal area network (PAN),

580
Sester, Marie, 549
621–622
Readme!, 563
Shannon, Claude, 669
Petzold, Charles, 648
Real-Time Cmix, 592
Shape of Song (Wattenberg),
Phidgets, 642
Rees, Michael, 526
160–163
Philips, 634
Reeves, Alec 585
Shaw, Jeffrey, 333
PHP, 267–268, 522–523, 565,
Reich, Steve, 293
Shiffman, Daniel, 6
682,
684
Reichardt, Jasia, 522
Shockwave Flash (SWF), 158, 565
PHPMyAdmin, 268
Reiniger, Lotte, 315
Short Messaging Service (SMS),

Physical Computing (O’Sullivan,
RenderMan, 315
617, 619, 621
Igoe),
648
R.E.M. “Animal” (Motion Theory),
SHRDLU, 101
Piano Phases (Reich), 293
378–381
sh/tcsh, 515, 684
PIC (Microchip), 272, 640
ResEdit, 170
Shulgin, Alexi, 563–564
PIC Assembler, 271–272
Resnick, Mitchel, 471, 680
Silicon Graphics, 529, 537
PIC BASIC, 681
Reynolds, Craig, 295, 473, 497
Simon, John F. Jr., 413, 565
Pickard, Galen, 507
Rhino, 271, 537
SimpleTEXT, 618
Pickering, Will, 516
Rich, Kate, 548
Sims, Karl, 295
Pixar, 315
Riley, Bridget, 151
Sinclair Spectrum, 264
Pixillation (Schwartz), 315
Ringtail Studios, 275
Singer, Eric, 554
PNG (Portable Network Graphics),
Risset, Jean-Claude, 581
Sketchpad, 217
95–96, 98–99, 606, 622, 701
RoboFog, 170
SketchUp, 538
Pocket PC, 625
Rokeby, David, 548, 554
Slacker, 383
PoemPoints, 617
Rotoshop, 383–384, 413
Slimbach, Robert, 112
Pong, 256, 590, 618
Royal Academy of Arts, 169
Smalltalk, 685
PortAudio, 512
Royal College of Art, 634
Smith, Laura, 275
PostScript, 111, 143, 169–170, 522,

Rozin, Danny, 549
Snake, 618
604–605,
681
RS-232, 639, 554, 640, 645
Snibbe, Scott, 413, 549
Poynor, Rick, 605
Rubin, Ben, 515, 634
Social Mobiles (SoMo), 634
Practical Electronics for Inventors
Ruby, 681, 684
Sodaconstructor (Burton),
(Scherz),
648
Ruby on Rails, 684
262–265, 413, 499
Practice of Programming,The

Runge-Kutta method 7, 494
Soda Creative Ltd., 263–264
(Kernighan, Pike), 252
Russolo, Luigi, 579
SoftVNS, 554
Praystation (Davis), 564
Solidworks, 537
Public Enemy, 581
Sabiston, Bob, 383–384, 413
Sommerer, Christa, 549
Puckette, Miller, 2, 592, 684
Saito, Tatsuya, 198, 529, 568
Sonami, Laetitia, 582
Pulse–code modulation (PCM),

Samyn, Michaël, 275
Sonic Inc., 392
585–586, 699, 702
Sauter, Joachim, 549
Sony, 634
Pure Data (Pd), 592, 684–685
Schaeffer, Pierre, 580
Sony Ericsson, 625
Python, 146, 170, 517, 522–523,
Scheme, 522
Sorenson, 388
681–682,
684
Scherz, Paul, 648
Sorting Daemon (Rokeby), 548, 554
Schiele, Egon, 217
Sound Films, 392
709
Index
Reas_09_519-710.indd Sec6:709
Reas_09_519-710.indd Sec6:709
5/16/07 12:33:55 PM
5/16/07 12:33:55 PM

Spark Fun Electronics, 640
Toy Story, 315
von Neumann, John, 461
SQL (Structured Query Language),
Tmema, 510–513
Vorbis codec, 585
685
Transmission Control Protocol

Voxel, 527
Srivastava, Muskan, 5
(TCP),
569
vvvv, 685
Standards and Double Standards
Tron, 315
(Lozano-Hemmer), 547–548
Truax, Barry, 581
Wacom, 383
Star Wars, 315
TrueType, 111
Waking Life, 382–385, 413
Strausfeld, Lisa, 327
Tsai, Wen–Ying, 633
Walt Disney, 315, 379
Stedelijk Museum, 218
TurboGears, 684
Wang, Ge, 592
Stehura, John, 315
Turkle, Sherry, 5
Warner Bros. Records, 379
STEIM (Studio for Electro-

Turux (Lia, Dextro), 316
Wattenberg, Martin, 161–162, 606
Instrumental Music), 554
Type, Tap, Write (Maeda), 333
Watz, Marius, 374
Steinkamp, Jennifer, 387–388
Tzara, Tristan, 150
WAV, 585–586, 621, 623, 702
Stipe, Michael, 379–380
Wayfinder Systems, 619
Stockhausen, Karlheinz, 580
Überorgan (Hawkinson), 633
Wegman, William 606
Stone, Carl, 582
Unicode, 432, 665–668
Weizenbaum, Joseph, 101
Stop Motion Studies (Crawford),
University of California
Whitney, James, 315
316
Berkeley,
589
Whitney, John, 315
Studies in Perception I, (Knowlton,
Los Angeles (UCLA), xxi, 4, 5, 574
Whitney Museum of American

Harmon),
604
San Diego (UCSD), 549
Art,
516
Substrate (Tarbell), 6, 154, 156–159
University of Cincinnati (UC), xxiii
Wilhelm Imaging Research, 610
Sudol, Jeremi, 507
University of Genoa, 554
Wilson, Mark, 152, 217, 604
Suicide Box, 548, 554
UNIX, 227, 435, 517, 569, 645
Winograd, Terry, 101
Sun Java Wireless Toolkit, 625
U.S. Army Ballistic Missile

Wiring, 521, 633, 640, 641, 645–646,
Sun Microsystems, 521, 537,
Research Laboratories, 603
648–649,
685
625,
682
USB, 556, 640–645, 701
Wright, Frank Lloyd, 333
SuperCollider, 571, 592, 685
User Datagram Protocol (UDP),

Wrongbrowser (Jodi), 566
Sutherland, Ivan, 217
554, 569, 589
Wolfram, Steven, 461, 463–464,

SVG (Scalable Vector Graphics),
Ussachevsky, Vladimir, 580
467,
475
77, 520, 606, 624
UTF-8, 665
Wolfenstein 3D, 525
Symbian, 625
Utterback, Camille, 549
Wong, Andy, 507
Synergenix, 625
Wong, Yin Yin, 327
Valicenti, Rick, 605
Tale of Tales, 274–277
van Blokland, Erik, 169–170
Xenakis, Iannis, 581
Talmud Project (Small), 327
van Blokland, Petr, 170
Xerox Palo Alto Research Center
Takeluma (Cho), 327
VanDerBeek, Stan, 315
(PARC), 3, 205
Takis, 633
Vanderlans, Rudy, 605
Xerox, 507
Tarbell, Jared, 6, 155–156, 606
van Rossum, Just, 169
xHTML Mobile, 624
Tangible Media Group (TMG), 634
Vaucanson’s Duck, 461
XML, 421, 427–428, 520, 549, 621,

TARGA, 368, 606, 702
Vehicles: Experiments in Synthetic
624,
702
Tate Gallery, 218
Psychology
(Braitenberg), 473
T|C Electronics Powercore, 587
Venice Biennale, 391
Yamaha Digital Mixing Engine

tcpdump, 568–569
Verschoren, Jan, 275
(DME),
516
TCP/IP, 554, 569, 589
“Video Games and Computer

Yellow Arrow, 618
Technics, 507
Holding Power” (Turkle), 5
Youngblood, Gene, 388
Teddy (Igarashi), 538
Videoplace (Krueger), 547
Telephony (Thompson,
Visual Language Workshop (VLW),
Ziggurat (font), 112
Craighead),
618
327,
702
Zooming user interface (ZUI), 435
TeleNav, 619
Visual programming languages
Teleo (Making Things), 642
(VPL or VL) , 679–680
Tesla, Nikola, 579
Vitiello, Stephen, 582
Text-to-speech (TTS), 516–517
VLW font format, 112, 702
They Rule (On et al.), 266–269
Vogel, Peter, 632, 633
Thomson, Jon, 618
Von Ehr, Jim, 170
TIFF, 368, 507, 606, 608, 611, 702
Vonnegut, Kurt, 507
710
Index
Reas_09_519-710.indd Sec6:710
Reas_09_519-710.indd Sec6:710
5/16/07 12:33:55 PM
5/16/07 12:33:55 PM