On Three Layer Architectures
On Three-Layer Architectures
Erann Gat
Jet Propulsion Laboratory
California Institute of Technology
4800 Oak Grove Drive
Pasadena, CA 91109
(Appears in Artificial Intelligence and Mobile Robots, David Kortenkamp,
R. Peter Bonnasso, and Robin Murphy, eds., AAAI Press)
the
face
of
environmental
uncertainty
and
1. Historical Background
unpredictability. Several researchers in the mid 80's
suggested that a different execution mechanism was
In the mid-1980's Rodney Brooks touched off a
needed [Firby87, Agre87, Payton86]. The SPA
firestorm of interest in autonomous robots with the
approach was so dominant at the time that this new
introduction of the
Subsumption
architecture1
work was labeled with the self-contradictory buzzword
[Brooks86]. At the time, the dominant view in the
"reactive planning."
AI community was that a control system for an
Subsumption is the best known departure from SPA.
autonomous mobile robot should be decomposed into
It is also popularly perceived as the most radical of its
three functional elements: a sensing system, a
time. Ironically, in Brooks's seminal 1986 paper
planning
system,
and
an
execution
system
Subsumption is presented as a compatible extension
[Nilsson80]. The job of the sensing system is to
to SPA:
translate raw sensor input (usually sonar or vision
data) into a world model. The job of the planner is to
But what about each individual layer? Don't
take the world model and a goal and generate a plan to
we need to decompose a single layer in the
achieve the goal. The job of the execution system is
traditional manner? This is true to some
to take the plan and generate the actions it prescribes.
extent, but the key difference is that we don't
The sense-plan-act (SPA)
approach has
two
need to account for all desired perceptions and
significant architectural features. First, the flow of
processing and generated behaviors in a single
control among these components is unidirectional and
decomposition.
linear. Information flows from sensors to world
model to plan to effectors, never in the reverse
So in its original presentation, Subsumption is, at
direction. Second, the execution of an SPA plan is
least ostensibly, not a radical departure from SPA at
analogous to the execution of a computer program.
all, but rather an attempt to make SPA more efficient
Both are built of primitives composed using partial
by applying task-dependent constraints to the
orderings, conditionals, and loops. Executing a plan
Subsumption layers. This view is reinforced by the
or a program is easy (and therefore uninteresting)
canonical diagram of Subsumption, which shows all
when compared with generating one.
The
information flowing unidirectionally from sensors to
information content is in the composition structure,
actuators, just as in SPA.
not the primitives. The intelligence of the system
Where Subsumption departs radically from SPA is in
(such as it is) lives in the planner or the programmer,
its repudiation of plans (and indeed of symbolic
not the execution mechanism. Research efforts
representations in general [Brooks90]). From the
through about 1985 focused almost exclusively on
details of Brooks's examples it is clear that
planning and world modeling.
Subsumption's layers are not decomposed "in the
By 1985 it was becoming clear that SPA had
traditional manner" at all. Instead, they are simple
numerous shortcomings. Planning and world
networks of small finite state machines joined by
modeling turned out to be Very Hard Problems, and
"wires" which connect output ports to input ports.
open-loop plan execution was clearly inadequate in
Subsumption provides only one mechanism for
composing these basic building blocks to produce
complex control mechanisms: the ability to override
1There is no consensus on the definition of
the contents of one wire with a value from another
the word "architecture"in the context of
wire. This process is called suppression or inhibition
software systems. In this chapter I will use
depending on whether it takes place at an input or an
the word to mean a set of constraints on the
output port. Subsumption also advocates a
structure of a software system.
development methodology consisting of layering
progressively more complex task-specific control
behaviors by high-level ones is not always
programs (called behaviors) on top of each other.
appropriate. Sometimes the low level should
However, Subsumption provides no architectural
override higher levels. ...
mechanism to enforce (or even support) this
methodology.
Note that Hartley is taking issue with the most
fundamental tenet of Subsumption as a design
Subsumption achieved dramatic early success in the
methodology, saying in effect that the central (indeed
area of collision-free robot navigation. While SPA-
the only) architectural mechanism that Subsumption
based
robots
were
pondering
their
plans,
provides often doesn't work. (It should be noted that
Subsumption-based robots were zipping around the
Hartley's critique only addresses Subsumption as an
lab like R2D2. By the common metric that speed
engineering methodology, not as a model of human
equals intelligence, Subsumption appeared to be a
intelligence [Brooks91]. For such a critique, see
major breakthrough.
[Kirsh91].)
Subsumption reached a pinnacle with a robot called
The years following the introduction of Subsumption
Herbert, which was programmed to find and retrieve
in 1986 saw a profusion of new robot control
soda cans in an office environment [Connell89].
architectures, some
developed
more
or
less
(Brooks has recently launched an even more
independently
(e.g.
[Kaelbling88],
[Soldo90],
ambitious project called Cog, but as of this writing
[Arkin90], [Georgeff87], [Simmons90]) and others
no results have been published.) While Herbert's
introduced as a direct response to Subsumption's
capabilities were impressive even by
today's
shortcomings (e.g. [Rosenblatt89]). One of the first
standards, it also appeared to represent the limits of
robots to be built using one of these latter
what could be achieved with Subsumption. Herbert
alternatives was Tooth [Gat94] which was completed
was very unreliable (there is no record of it ever
in the summer of 1989 [Angle89]. Tooth was a
having performed a complete can-retrieval task
small robot (30 cm by 20 cm) with simple sensors
flawlessly), and no Subsumption-based robot has ever
and limited computation (two 8-bit microcontroller,
matched its capabilities since.
each with about 2000 bytes of memory), but it was a
One possible cause of Subsumption's apparent
very capable robot nonetheless. Tooth was
"capability ceiling" is that the architecture lacks
programmed to search for small objects (styrofoam
mechanisms for managing complexity. Quoting
coffee cups), pick them up in its gripper, and return
from [Hartley91]:
them to a light-bulb beacon. A similar capability
was demonstrated a year later on an outdoor robot,
The most important problem we found with
RockyIII [Miller91, Gat94] using the same control
the Subsumption architecture is that is it not
methodology. In contrast with Herbert, Tooth and
sufficiently modular. The other problems
Rocky III were extremely reliable, running many
described below are really side-effects of this
dozens of trials without failing. (To be fair, Herbert
one. Because upper layers interfere with the
was a much more ambitious robot, finding its soda
internal functions of lower-level behaviors
cans using a structured-light vision system.)
they cannot be designed independently and
The software that controlled Tooth and Rocky III
become increasingly complex. This also
(which I will refer to as T/R-III, not to be confused
means that even small changes to low-level
with the 3T architecture described later in this chapter)
behaviors or to the vehicle itself cannot be
was a layered design like Subsumption. However,
made
without
redesigning
the
whole
unlike Subsumption, T/R-III embraced abstraction
controller.
rather than rejecting it. In Subsumption higher-level
layers interface with lower level ones by suppressing
Brooks proposes to solve this problem by
the results of the lower-level computations and
reducing
or
even
eliminating
direct
superseding their results. In T/R-III, higher-level
communications between modules. Instead
layers interfaced with lower-level ones by providing
behaviors would "communicate through the
input or advice to the lower-level layers (cf.
world." Except in a few cases ... we did not
[Payton90, Agre90]). In other words, layers in T/R-
find this approach useful. The problem was
III provided layers of computational abstraction as
that very similar states of the world could
well as layers of functionality.
mean different things depending on the
Tooth and Rocky III were among the first
context. ...
autonomous robots capable of reliably performing a
more complex task than simply moving from place
Determining that one behavior is more high-
to place, but they had one serious drawback: they
level than another is sometimes completely
were not taskable, that is, it was not possible to
artificial. ... Subsumption of low-level
change the task they performed without rewriting
having exactly three major components. It has to do
their control program.
with the role of internal state.
At least three different groups of researchers working
By way of motivation, consider the classic SPA
more or less independently came up with very similar
architecture, and two of its associated difficulties.
solutions to this problem at about the same time
First, because planning is time-consuming, the world
[Connell91, Gat91, Bonasso91]. All three solutions
may change during the planning process in a way that
consisted of control architectures that comprised three
invalidates the resulting plan. (The same problem
main components: a reactive feedback control
exists for world modeling.) Second, an unexpected
mechanism, a slow deliberative planner, and a
outcome from the execution of a plan step can cause
sequencing mechanism that connected the first two
subsequent plan steps to be executed in an
components. Connell's sequencer was based on
inappropriate context. (This problem often manifests
Subsumption,
Bonasso
used
Kaelbling's
itself as "running researcher syndrome," characterized
REX/GAPPS system [Kaelbling89], and Gat's was
by having to chase the robot to push the emergency
based on Firby's Reactive Action Packages (RAPs)
stop button after it makes a wrong turn.)
system as described in his 1989 thesis [Firby89].
To be fair, let us also consider a problem associated
Bonasso's group later adopted RAPs as their
with Brooks-style reactive architectures. A reactive
sequencing component, while Gat's sequencer was
robot using ultrasonic sensors to control its motions
recently developed into a new language, ESL [Gat97].
sometimes can collide with obstacles when specular
Aside from the technical advances, there are two items
(mirror-like) reflections produce readings that fail to
of historical interest in Firby's thesis. The first is
indicate the obstacle's presence.
that the title catch phrase was changed from "reactive
All three of these problems can be viewed as a result
planning" to "reactive execution," heralding a clean
of the method used to manage stored internal state
break from the SPA tradition. The second is that it
information [Gat93]. Time-consuming computations
contains the earliest description of the three-layer
like planning and world modeling generate internal
architecture that has now become the de facto standard
state whose semantics reflect world states, whether
[Firby89, figures 1.1 and 7.1]. This original three-
they are past, present (in the case of world models) or
layer architecture was briefly implemented on JPL's
future (in the case of plans). Plan execution also
Robbie robot [Wilcox87], but there is no record of
involves internal state, the program counter, which
the results. RAPs has since been used to control a
implicitly encodes the planner's expectations about
number of real robots, including Uncle Bob
the state of the world as execution proceeds. SPA
[Elsaessar&Slack94], Homer [Gat&Dorais94], and
gets into trouble when its internal state loses sync
Chip [Firby96]. The RAP-based three-layer
with the reality that it is intended to represent.
architecture has come be called 3T [Bonnasso et al.
96]. Connell's Subsumption-based architecture is
The reactive solution to this problem is to minimize
called SSS. Gat's architecture is called ATLANTIS.
the use of internal state as much as possible. If there
It was first implemented on Robby in 1990
is no state, then it cannot lose sync with the world, a
[Gat91,92], and has since been implemented on a
sentiment often expressed by the slogan, "The world
number of other robots. (See section 4.) The main
is its own best model." Unfortunately, extracting
differences between 3T and ATLANTIS are that 1)
information from the world-as-its-own-model requires
ATLANTIS used a different representation in its
using sensors, which are unreliable and subject to
sequencing layer, one designed more for programming
occlusions. Sometimes a robot might do well to
convenience than for use as a planner representation,
remember that there was a wall in front of it a little
2) the sequencer controlled the operation of the
while ago, and to conclude that the wall is probably
planner rather than vice versa. ATLANTIS also
still there despite the fact that it seems to have
extended the then-existing RAPs action model to use
vanished according to the sonars. By eliminating
continuos real-time processes rather than atomic
internal state the reactive approach avoids the problem
operators, a feature which has since been incorporated
associated with maintaining that state, but runs
back into the de facto standard.
headlong into the problem of extracting reliable
information about the world through sensors.
2. The role of internal state
Three-layer
architectures
organize
algorithms
according to whether they contain no state, contain
At this point the question naturally arises: why do so
state reflecting memories about the past, or contain
many independently designed architectures turn out to
state reflecting predictions about the future. Stateless
have such a similar structure? Are three components
sensor-based
algorithms
inhabit
the
control
necessary and/or sufficient, or is three just an
component. Algorithms that contain memory about
aesthetically pleasing number or a coincidence? I
the past inhabit the sequencer. Algorithms that make
believe that there is a sound architectural rationale for
predictions about the future inhabit the deliberator.
Abstraction is used as a tool to isolate aspects of
Second, the algorithms in the controller should fail
reality that can be tracked or predicted reliably, and
cognizantly, that is, they should be designed to detect
ignore aspects that cannot.
(as opposed to avoid) any failure to perform the
function for which they were designed (c.f.
3. The Anatomy of the Three Layer
[Noreils90]). Rather than attempt to design
Architecture
algorithms that never fail (which is impossible on
real robots) one can instead design algorithms that
The three-layer architecture consists
of
three
never fail to detect a failure. This allows other
components: a reactive feedback control mechanism, a
components of the system (the sequencer and
reactive plan execution mechanism, and a mechanism
deliberator) to take corrective action to recover from
for
performing
time-consuming
deliberative
the failure.
computations. These components run as separate
Third, the use of internal state should be avoided
computational processes. This is most easily
whenever possible. An important exception to this
accomplished by using a multi-tasking or multi-
rule is filtering algorithms, which rely on internal
threaded operating system, but can also be done my
state, but can nevertheless be used in the controller.
carefully coding the algorithms so they can be
If internal state is used for other purposes, it should
manually interleaved within a single computational
be ephemeral, that is, it should expire after some
process.
constant-bounded time. This way, if the semantics of
In 3T the components are called the skill layer, the
the internal state do not reflect the true state of affairs
sequencing layer, and the planning layer respectively.
in the environment at least the time during which this
In ATLANTIS these layers are called the controller,
error will manifest itself will be bounded.
the sequencer, and the deliberator. The following
Finally, internal state in the controller should not
discussion uses the ATLANTIS terminology, but as
introduce discontinuities (in the mathematical sense)
much as possible the description is generic to all
in a Behavior. In other words, a Behavior (which is a
incarnations of the three-layer architecture.
transfer function) should be a continuous function
with respect to its internal state. It is the
3.1 The Controller
responsibility of the sequencer to manage transitions
between regimes of continuous operation.
The controller consists of one or more threads of
A number of special-purpose languages have been
computation that implement one or more feedback
developed for programming the controller (e.g.
control loops, tightly coupling sensors to actuators.
[Gat91b], [Brooks89]), but any language can be used
The transfer function(s) computed by the controller
as long as the architectural constraints are observed.
can be changed at run time. Usually the controller
Most
of
the
special-purpose
languages
for
contains a library of hand-crafted transfer functions
programming the controller were developed at a time
(called primitive behaviors or skills). Which ones are
when robots could only support very small processors
active at any given time is determined by an external
for which no other development tools were available.
input to the controller.
The current trend is to simply program the controller
To distinguish between the code that implements a
in C.
transfer function, and the physical behavior produced
by that transfer function when running on a robot in
3.2 The Sequencer
an environment, we shall capitalize the former.
Thus, a Behavior is a piece of code that produces a
The sequencer's job is to select which primitive
behavior when it is running. Primitive Behaviors are
Behavior (i.e. which transfer function) the controller
designed to produce simple primitive behaviors that
should use at a given time, and to supply parameters
can be composed to produce more complex task-
for the Behavior. By changing primitive Behaviors at
achieving behavior (a job done, naturally, by the
strategic moments the robot can be coaxed into
sequencer). Classic examples of primitive behaviors
performing useful tasks. The problem, of course, is
are wall-following, moving to a destination while
that the outcome of selecting a particular primitive in
avoiding collisions, and moving through doorways.
a particular situation might not be the intended one,
There are several important architectural constraints
and so a simple linear sequence of primitives is
on the algorithms that go into the controller. First,
unreliable. The sequencer must be able to respond
computing one iteration of the transfer function
conditionally to the current situation, whatever it
should be of constant-bounded time and space
might be.
complexity, and this constant should be small enough
One approach to the problem is to enumerate all the
to provide enough bandwidth to afford stable closed-
possible states the robot can be in, and precompute
loop control for the desired behavior.
the correct primitive to use in each state for a
particular task. Clever encoding can actually make
controller. Exactly how long a "long time" is
this daunting task tractable for certain constrained
depends on both the environment and the repertoire of
domains [Schoppers87]. However, this universal
Behaviors. Usually this constraint implies that the
plan approach has two serious drawbacks. First, it is
sequencer should not perform any search or temporal
often not possible for a robot to know its current
projection, but it might also constrain, for example,
state, especially when unexpected contingencies arise.
certain vision processing algorithms from appearing
Second, this approach disregards the robot's execution
in the sequencer, especially if computational resources
history, which often contains useful information.
are limited.
An alternative is to use an approach called conditional
sequencing, which is a more complex model of plan
3.3 The Deliberator
execution motivated by human instruction following.
Humans can achieve tasks based on very concise
The deliberator is the locus of time-consuming
instructions in the face of a wide variety of
computations. Usually this means such things as
contingencies
(e.g.
[Agre90],
[Suchman87]).
planning
and
other
exponential
search-based
Conditional sequencing provides a computational
algorithms, but as noted before, it could also include
framework for encoding the sort of procedural
polynomial-time algorithms with large constants
knowledge contained in instructions. It differs from
such as certain vision processing algorithms in the
traditional plan execution in that the control
face of limited computational resources. The key
constructs for composing primitives are not limited
architectural feature of the deliberator is that several
to the partial ordering, conditionals, and loops used to
Behavior transitions can occur between the time a
build SPA plans. Conditional sequencing systems
deliberative algorithm is invoked and the time it
include constructs for responding to contingencies,
produces a result. The deliberator runs as one or more
and managing multiple parallel interacting tasks.
separate threads of control. There are no architectural
constraints on algorithms in the deliberator, which are
It is possible to construct a conditional sequencing
invariably written using standard programming
system in a traditional programming language like C,
languages.
but because the control constructs are so much more
complex than those provided by such languages
The deliberator can interface to the rest of the system
conditional sequencing is much more effectively done
in two different ways. It can produce plans for the
with a special-purpose language like RAPs [Firby89],
sequencer to execute, or it can respond to specific
PRS
[Georgeff87],
the
Behavior
Language
queries from the sequencer. The RAPs-based 3T
[Brooks89],
REX/GAPPS
[Kaelbling87,
architecture takes the first approach [Bonasso et al.
Kaelbling89, Bonasso92], or ESL [Gat97].
97]. The ESL-based ATLANTIS architecture takes
the second approach. This is a natural result of the
There are two major approaches to the design of
fact that RAPs was designed specifically to serve as a
conditional sequencing languages. They can be
plan representation for an automated planning system
complete languages in their own right with their own
and ESL was not. These two approaches are not
specialized execution semantics. RAPs and PRS take
mutually exclusive. RAPs does permit deliberative
this approach. Or they can be layered on top of a
algorithms (called RAP-experts) to be invoked at
syntactically extensible programming language like
runtime to answer specific queries, and the
Lisp. This is the approach taken by the Behavior
ATLANTIS sequencer can ask the deliberator to give
Language and ESL. Furthermore, the structure of the
it a complete plan which it then executes. (This is
language can treat all possible outcomes of an action
being done in an application of ESL to autonomous
in a homogeneous fashion, or the language can be
spacecraft [Pell et al. 96].)
structured to recognize a privileged "nominal" result
of an action and treat all other outcomes as "failures."
Again, RAPs and PRS take the first approach; ESL
3.4 Discussion
takes the second.
The architectural guidelines that govern the design of
Which approach one chooses depends on what one it
the three-layer architecture are not derived from
trying to do. The RAPs/PRS approach results in a
fundamental theoretical considerations. Instead, they
more circumscribed language that is suitable for use
are derived from empirical observations of the
as a representation for an automated planner. The
properties of environments in which robots are
ESL approach, because it subsumes a traditional
expected to perform, and of the algorithms that have
programming language, is more convenient to use
proven useful in controlling them. Robot algorithms
and easier to extend, but more difficult to analyze.
tend to fall into three major equivalence classes: fast,
The sequencer should not perform computations that
mostly stateless reactive algorithms with hard real-
take a long time relative to the rate of environmental
time bounds on execution time, slow deliberative
change at the level of abstraction presented by the
algorithms like planning, and intermediate algorithms
which are fairly fast, but cannot provide hard real-time
guarantees.
"Fast" and "slow" are measured with respect to the
rate of change of the environment. In principle, if the
rate of change of the environment is sufficiently slow
(or, equivalently, if a planner were available that was
sufficiently fast) the controller could contain a
planner. (Note that this situation is essentially
equivalent to the SPA architecture.) Empirically it
turns out not to be possible to build planners that are
fast enough to operate in this manner in realistic
environments.
4. Case Study
To date at least half a dozen different robots have been
programmed using some variation of the three-layer
Figure 1: Schematic top-view of Alfred
architecture [Gat92, Gat&Dorais94, Nourbakhsh et al.
showing
sonar
directions
(radial
93,
Elsaessar&Slack94,
Connell91,
Firby96,
arrows) relative to the direction of
Bonasso et al. 97, Firby&Slack95. I will describe
one of these here in some detail.
motion (dark central arrow) and
obstacle-detection regions.
Alfred is a B12 robot built by Real World Interface
(RWI). The B12 is a cylindrical robot, twelve inches
in diameter with a synchrodrive mobility mechanism.
Alfred also used a Macintosh Powerbook Duo230
Encoders on the drive and steering motors provide
running Macintosh Common Lisp (MCL) mounted
fairly reliable odometry and dead reckoning, although
on top of the robot. The Duo was connect to the
the robot's heading tends to precess due to slight
Gespak board through an RS-232 serial port running
misalignments of its wheels. A development
at 9600 baud. Alfred's controller ran on the Gespak
enclosure houses a Gespak 68000 computer and a
board. The sequencer and deliberator were
radially symmetric ring of twelve Polaroid sonars
programmed in Lisp and ran on the Powerbook.
[Biber80]. The sonars are mounted on panels that
Alfred was programmed to compete in two events at
allow them to be easily reconfigured. The sonar
the 1993 AAAI mobile robot contest [Nourbakhsh at
configuration was rotated 15 degrees from the default
al. 93]. The first event was called "escape from the
factory configuration, resulting in one sonar pointing
office" and involved finding a route out of a room
straight forward in the direction of motion, and one
filled with furniture, and across a large open area filled
sonar pointing directly to either side. (See figure 1.)
with boxes. The second event was called "deliver the
This turned out empirically to make wall-following
coffee" and involved self-localization and path-
more reliable.
planning in a maze.
Alfred placed second in the first event, and was the
only robot to complete the second event. All the
contest-specific code for the robot was written in three
days by one person.
The following sections describe the
control,
sequencing, and deliberative layers on Alfred. These
descriptions are faithful to the actual implementation
used in the contest, and could no doubt be improved
on.
4.1 Control Layer
Alfred's control layer was implemented in ALFA (A
Language For Action) [Gat91b], a language designed
by its structure to enforce the control layer's
architectural constraints. ALFA is a dataflow
language with no looping constructs. It does,
however, have state variables, making it Turing-
constraints because it is ephemeral, and in the second
complete. It is therefore possible to implement
case it is part of a filtering algorithm. The filtering
arbitrary algorithms in ALFA, and so ALFA's
algorithm might appear to violate the prohibition on
constraint enforcement is far from perfect. The
state-dependent discontinuities, but this is not the
language does not make it impossible to violate the
case. The output of the controller is a continuous
rules, just more difficult. Unfortunately, it turns out
function of the state; it is the value of the state that
that ALFA's design also makes it more difficult to do
changes discontinuously over time. Ideally the
things that should be allowed in the control layer. I
collision response routine would have been put in the
no longer advocate the use of ALFA, preferring
sequencer, but because the robot detected collisions by
instead to use C or Lisp and a little self-discipline to
monitoring motor current, by the time a collision
enforce the architectural constraints.
was detected there was already quite a bit of
mechanical stress built up in the robot's drive
Alfred's control layer had four interesting behaviors:
mechanism. Simply stopping the robot would have
obstacle avoidance, wall-finding, wall-alignment,
caused the robot's motor servo controller to attempt
wall-following, and wandering. (It also had a number
to maintain the motor's velocity at zero, which would
of uninteresting but nonetheless useful behaviors like
have maintained this mechanical stress. Relieving
turning in place.)
the stress required backing up, and to accomplish this
Obstacle avoidance was done as follows. First, the
as quickly as possible, the response was implemented
sonar data was preprocessed to indicate the presence or
in the controller. This is a good example of how the
absence of an obstacle in each of five regions around
lines between the components of the three-layer
the front of the robot. (The five rear-facing sonars
architecture can be blurred to accommodate reality.
were not used.) There were two near-field "hard
Obstacle avoidance was done with the following
obstacle" regions close to the front of the robot
algorithm:
(figure 1, dark shading), and three "soft" obstacle
regions further away. The hard obstacle region was
IF there is an obstacle in the soft-left
divided into left and right regions that overlapped at
obstacle region and not in the soft-right
the front sonar. The robot was able to detect
region
collisions by monitoring its motor current. The
turn slowly to the right
obstacle regions were egg-shaped, extending further
ELSE IF there is an obstacle in the soft-right
obstacle region but not in the soft-left
from the robot towards the front than at the sides.
obstacle region
turn slowly to the left
At the core of the controller code was the following
ELSE
safety algorithm that was always running:
go straight, or turn towards a commanded
heading.
IF there is a collision while moving forward
BACK UP slowly for a few seconds
This algorithm only avoids obstacles when the choice
ELSE IF there is a collision while moving
backwards
of turning direction is clearly dictated by the
STOP for a few seconds
situation. When an obstacle is directly in front, the
ELSE IF there is an obstacle in one of the hard
robot does not turn. This is because the control layer
obstacle regions
has no information on which to base the choice of a
STOP
ELSE IF there is an obstacle in one of the soft
turning direction, and so this choice is deferred to the
obstacle regions
sequencer.
set the current speed to SLOW
ELSE (there are no obstacles)
Wall-finding was done by turning towards the sonar
gradually increase forward speed up to a
with the shortest range reading until the shortest
maximum value.
reading was given by the forward sonar, and
simultaneously moving forward until forced to stop
This code had the effect of slowing the robot down in
by an obstacle in a hard-obstacle region. This would
the presence of obstacles, and stopping the robot
reliably leave the robot facing the nearest object.
when it was in imminent danger of collision. By
When initiated near a wall, the robot would turn
allowing any detected obstacle to immediately slow
towards the wall.
the robot down, but only a succession of clear
Wall alignment was done by slowly turning the robot
readings to speed it back up again, the robot reliably
until a discontinuity caused by the onset of specular
slowed down in cluttered areas even if when there was
reflection was seen in the range reading returned by
a lot of specular reflection.
the forward sonar. When this procedure was begun
Note that this code uses internal state to remember
while facing a smooth wall, the angle at which the
collisions for a few seconds after they happen, and to
discontinuity occurred was reproducible to better than
keep track of the current maximum speed. This use
1 degree.
of internal state obeys the controller's architectural
Wall following was done by servoing the robot's
minimum positions along the X and Y axes.
heading to the reading on a side-facing sonar while
Wandering was done by augmenting the obstacle-
moving forward. Although conceptually simple, the
avoidance code with an algorithm for choosing a turn
actual implementation is complicated by a number of
direction when the choice was not clear from the
factors.
current situation. This must be done with some care,
or the robot can get stuck in an infinite loop. Alfred
The main problem is that a straightforward negative-
used the following algorithm: when an ambiguous
feedback servo loop off a side-facing sonar is unstable
obstacle avoidance situation was encountered the
if the robot ever turns far enough towards the wall to
robot would do an angular scan, turning first one
cause a specular reflection on the side sonar. When
way, then the other. The angle of the turn was
this happens, it appears to the robot that the wall is
gradually increased until the robot was able to move
suddenly very far away, and it will continue to turn
forward some threshold distance without triggering
towards the wall and eventually collide unless the
the scan. The scan angle was then reset to its initial
safety module stops it. A similar effect happens
value. Alfred's wander behavior was actually written
when the robot passes an open door or an intersecting
in ALFA, although its use of stored internal state to
corridor.
produce discontinuous behavior indicates that it
There are two possible solutions to this problem.
should be considered part of the sequencing layer.
The first is to servo to the shortest reading on the
Alfred then attempted to escape from the office by
side-facing sonar and its two adjacent sonars. The
trying each of the three door locations in turn. It
second is to use a model-based estimation algorithm
would move to the center of the office, point itself
such as a Kahlman filter to compute the distance to
towards one of the doors, and turn on the follow-
the wall. The solution used on Alfred was a model-
current-heading-with-obstacle-avoidance primitive. It
based estimator (though not a Kahlman filter). The
would then wait until it either escaped the office (as
estimator simply rejected any sonar reading that was
indicated by its dead-reckoning position) in which
much greater than the last known distance to the wall.
case it headed towards the finish line, or a time limit
The estimator also kept track of the robot's heading
was reached in which case it tried the next door. This
and odometer reading (i.e. the drive motor encoder
task required no planning.
reading) every time a valid sonar reading was taken.
When an invalid sonar reading occurred, the robot
The second contest was much more interesting and
turned towards the heading it was on during the last
challenging. The robot was put in a maze for which
valid reading. If the robot traveled more than two
it had been given a complete and accurate map.
meters without a valid reading the robot stopped.
However, the robot was given no information about
its initial position or orientation. The robot's task
All of these primitive behaviors were implemented in
was to search for a coffeepot hidden in the maze and
less than 200 lines of ALFA code.
deliver it to a given destination. The robot was given
partial information about the location of the
4.2 Sequencing Layer
coffeepot. Of course, Alfred had no sensors capable
of detecting a coffeepot, so it had to be told when the
Alfred's sequencing code was written in Macintosh
coffeepot was nearby, but otherwise Alfred completed
Common Lisp version 2.0 (with one exception; see
the task with no cheating.
below), using a set of macros that later evolved into
The key to Alfred's success was a combination of
ESL [Gat97]. MCL 2.0 is a single-threaded Lisp,
behaviors that allowed for reliable navigation of
which made it impossible to implement multithreaded
environments that were rich in walls, like mazes, and
task management directly. MCL version 3 is
some creative representations. In addition to
multithreaded, and all of the code and infrastructure
representing the a priori map of the maze in terms of
described in this section have been much improved
open space, the robot was also given a description of
since Alfred's code was written.
the maze in terms of the wall assemblies that
The first contest event required the robot to search an
comprised it. (With a little more time the robot
office-like environment for a door, then traverse an
could have been programmed to convert from one
obstacle-strewn area to a finish line. The door was
representation to the other automatically.)
opened between one and three minutes after the start
The robot self-localized by first locating a wall. It did
of the event, and could be in one of three different
this by invoking the wall-finding primitive, and then
locations. The robot was told its initial orientation
verifying that it had indeed found a wall rather than an
and the size of the room, but not its initial position
obstacle by attempting to follow it for some distance
nor the locations of obstacles.
(2 meters). It then began to follow the wall, turning
Alfred determined its location by wandering randomly
whenever the wall turned, and keeping track of the
for one minute and keeping track of its maximum and
sequences of turns. Whenever it made a turn, it
checked to see if the sequence of turns it had made
imposed on algorithms of the first type have been
created an unambiguous match with its a priori
largely unsuccessful.
knowledge of the shapes of the wall assemblies in the
In retrospect, in the story of the three-layer
maze. (This was done by the deliberator.) As soon
architecture there may be more to be learned about
as it had a match, the robot knew where it was. It
research methodology than about robot control
then began a systematic search of the possible
architectures. For many years the field was bogged
locations of the coffeepot, followed by a traversal to
down in the assumption that planning was sufficient
the delivery location.
for generating intelligent behavior in situated agents.
Note that the algorithms in the sequencer make
That it is not sufficient clearly does not justify the
extensive use of internal state (keeping track of which
conclusion that planning is therefore unnecessary. A
door location is being tried, maintaining records of
lot of effort has been spent defending both of these
the robot's position, etc.) but no search or temporal
extreme positions. Some of this passion may be the
projection.
result of a hidden conviction on the part of AI
researchers that at the root of intelligence lies a
4.3 Deliberative Layer
single, simple, elegant mechanism. But if, as seems
likely, there is no One True Architecture, and
intelligence relies on a hodgepodge of techniques,
The deliberative layer did the matching of Alfred's
then the three-layer architecture offers itself as a way
self-localization sequence to the a priori map, and also
to help organize the mess.
planned paths for traveling between locations. Both
algorithms were simple exhaustive searches made
The three-layer architecture is by no means the last
tractable by the fact that the search space was bounded
word in either architectures or organizational tools. It
by the size of the maze.
largely ignores, for example, issues like sensor
processing, learning, and world modeling. Such
By the standards of AI, the deliberative layer was
algorithms may turn out to fit nicely within the
trivial and uninteresting, which is precisely what
existing structure, or it may prove necessary to extend
makes the three-layer architecture non-trivial and very
the architecture to incorporate them. This promises
interesting. The use of a sequencing layer makes it
to be fertile ground for future research.
possible (in fact, easy) to use trivial and uninteresting
algorithms to control real robots performing complex
tasks.
Acknowledgments
Pete Bonasso and Robin Murphy provided extensive
5. Conclusions
and thoughtful comments on an early draft of this
chapter. David Miller and Marc Slack provided useful
The three-layer architecture arises from the empirical
comments on an early draft of section 1. This work
observation that effective algorithms for controlling
was conducted at the Jet Propulsion Lab, California
mobile robots tend to fall into three distinct
Institute of Technology, under a contract with the
categories: 1) reactive control algorithms which map
National Aeronautics and Space Administration.
sensors directly onto actuators with little or no
internal state, 2) algorithms for governing routine
sequences of activity which rely extensively on
References
internal state but perform no search, and 3) time-
consuming (relative to the rate of change of the
[Agre87] Phillip E. Agre and David Chapman.
environment) search-based algorithms
such
as
"Pengi: An Implementation of a Theory of
planners. The three-layer architecture is based on the
Activity." Proceedings of the International
premise that algorithms of the first (second) type can
Joint Conference on Artificial Intelligence
provide effective computational abstractions for
(IJCAI), 1987.
constructing interfaces to algorithms of the second
[Agre90] Phillip Agre and David Chapman. "What
(third) type. This conclusion has apparently been
are Plans For?" Robotics and Autonomous
reached independently by at least three different groups
Systems, vol. 6, pp. 17-34, 1990.
of researchers.
[Angle89] Colin Angle, "Tooth Docs: the Paper",
Algorithms of the first and third type can be
unpublished manuscript.
programmed in conventional programming languages.
Algorithms of the second type appear to benefit
[Arkin90] Ronald C. Arkin. "Integrating Behavioral,
significantly
from
specialized languages
with
Perceptual and World Knowledge in Reactive
sophisticated control constructs. Attempts to
Navigation."
Robotics
and
Autonomous
construct languages to enforce the constraints
Systems 6 (1990) 105-122.
[Balch95] Tucker Balch, et al. "Io, Ganymede and
[Firby89] R. James Firby. Adaptive Execution in
Callisto: A multiagent robot trash collecting
Dynamic
Domains.
Technical
report
team." AI Magazine, Summer 1995.
YALEU/CSD/RR#672,
Yale
University,
1989.
[Biber80] C. Biber, et al. "The Polaroid Ultrasonic
Ranging System." Proceedings of the 67th
[Firby96] R. James Firby, et al. "Programming
Conference of the Audio Engineering Society,
CHIP for the IJCAI-95 Robot Competition."
1980.
AI Magazine, Spring 1996.
[Bonasso91] R. Peter Bonasso. "Integrating
[Firby&Slack95] R. James Firby and Marc Slack.
Reaction Plans and Layered Competences
"Task execution: Interfacing to reactive skill
Through Synchronous Control." Proceedings
networks." Working notes of the 1995 AAAI
of the Internation Joint Conference on
Spring Symposium on Lessons Learned from
Artificial Intelligence (IJCAI), 1991.
Implemented
Architectures
for
Physical
Agents, 1995.
[Bonasso92] R. Peter Bonasso. "Using Parallel
Program Specifications For Reactive Control
[Gat91] Erann Gat. "Reliable Goal-directed Reactive
of Underwater Vehicles." Journal of Applied
Control for Real-world Autonomous Mobile
Intelligence, June 1992.
Robots." Ph.D. Thesis, Virginia Polytechnic
Institute and State University, Blacksburg,
[Bonasso et al. 97] R. Peter Bonasso, et al.
Virginia, 1991.
Experiences with an Architecture for Intelligent
Reactive Agents. Journal of Experimental and
[Gat91b] Erann Gat. "ALFA: A Language for
Theoretical AI, 9(2), 1997.
Programming
Reactive
Robotic
Control
Systems." Proceedings of the IEEE Conference
[Brooks86] Rodney A. Brooks. A Robust Layered
on Robotics and Automation (ICRA), 1991.
Control System for a Mobile Robot. IEEE
Journal on Robotics and Automation, vol RA-
[Gat92] Erann Gat, Integrating Planning and
2, no. 1, March 1986.
Reaction in a Heterogeneous Asynchronous
Architecture for Controlling Mobile Robots,
[Brooks89] Rodney A. Brooks. "The Behavior
Proceedings of the Tenth National Conference
Language User's Guide." MIT AI Lab internal
on Artificial Intelligence (AAAI), 1992.
publication.
[Gat93] Erann Gat. "On the Role of Stored Internal
[Brooks90] Rodney A. Brooks. "Elephants Don't
State in the Control of Autonomous Mobile
Play Chess." Robotics and Autonomous
Robots." AI Magazine, Spring 1993.
Systems 6 (1990) 3-15.
[Gat94] Erann Gat, et al. "Behavior Control for
[Brooks91] Rodney Brooks. "Intelligence without
Robotic Exploration of Planetary Surfaces."
representation." Artificial Intelligence 47
IEEE
Transactions
on
Robotics
and
(1991) 139-160.
Automation 10 (4) 1994.
[Connell89]
Jonathan Connell.
A
Colony
[Gat&Dorais94] Erann Gat and Greg Dorais. "Robot
Architecture for an
Artificial
Creature.
Navigation
by
Conditional
Sequencing."
Technical
Report
1151,
Massachusetts
Proceedings of the International Conference on
Institute of Technology Artificial Intelligence
Robotics and Automation (ICRA), 1994.
Laboratory, 1989.
[Gat97] Erann Gat. "ESL: A Language for
[Connell91] Jonathan Connell, "SSS: A Hybrid
Supporting
Robust
Plan
Execution
in
Architecture Applied to Robot Navigation,"
Embedded Autonomous Agents." Proceedings
Proceedings of the IEEE Conference on
of the IEEE Aerospace Conference, 1997.
Robotics and Automation (ICRA), 1992.
[Georgeff87] Michael Georgeff and Amy Lanskey,
[Elsaessar&Slack94] Chris E;lsaessar and Marc
"Reactive
Reasoning
and
Planning",
Slack. "Integrating deliberative planning in a
Proceedings of AAAI-87.
robot architecture." Proceedings of the AIAA
Conference on Intelligent Robots in Field,
[Hartley91] Ralph Hartley and Frank Pipitone.
Factory, Service and Space (CIRFFSS), 1994.
"Experiments
with
the
Subsumption
Architecture." Proceedings of the International
[Firby87] R. James Firby. An Investigation Into
Conference on Robotics and Automation
Reactive Planning in Complex Domains.
(ICRA), 1991.
Proceedings
of
the
International
Joint
Conference on Artificial Intelligence (IJCAI),
1987.
[Kaelbling87] Leslie Pack Kaelbling. "REX: A
[Simmons90] Reid Simmons. "An Architecture for
Symbolic Language for the Design and Parallel
Coordinating Planning, Sensing and Action."
Implementation
of
Embedded
Systems."
Proceedings of the DARPA Workshop on
Proceedings of the AIAA conference on
Innovative
Approaches
to
Planning,
Computers in Aerospace, 1987.
Scheduling, and Control, 1990.
[Kaelbling88] Leslie Pack Kaelbling. "Goals as
[Simmons95] Reid Simmons. "The 1994 AAI
Parallel Program Specifications." Proceedings
Robot Competition and Exhibition." AI
of AAAI-88.
Magazine, Summer 1995.
[Kirsh91] David Kirsh. "Today the earwig,
[Soldo90] Monnett Soldo. "Reactive and Preplanned
tomorrow man?" Artificial Intelligence 47
Control in a Mobile Robot." Proceedings of
(1991) 161-184.
the International Conference on Robotics and
Automation (ICRA), 1990.
[Latombe91] Jean-Claude Latombe. Robot Motion
Planning. Kluwer Academic Publishers, 1991.
[Suchman87] Lucy Suchman. Plans and Situated
Action. Cambridge University Press, 1987.
[Miller91] David P. Miller, et al. "Reactive
Navigation
through
Rough
Terrain:
[Wilcox87] W. H. Wilcox, et al., "A vision system
Experimental
Results."
Proceedings
of
for a mars rover," Procedings of SPIE Mobile
AAAI92.
Robots II, vol. 852, 1987.
[Nilsson80] Nils J. Nilsson. Principles of Artificial
Intelligence. Palo Alto: Tioga. 1980.
[Noreils90] Fabrice Noreils. "Integrating Error
Recovery in a Mobile Robot Control System."
IEEE International Conference on Robotics and
Automation, 1990.
[Nourbakhsh et al. 93] Illah Nourbakhsh, et al. "The
Winning Robots from the 1993 Robot
Competition." AI Magazine, Winter 1993.
[Payton86] David Payton. "An Architecture for
Reflexive Autonomous Vehicle Control."
Proceedings of the International Conference on
Robotics and Automation (ICRA), 1986.
[Payton90] David Payton, J. Kenneth Rosenblatt and
David Keirsey, "Plan-Guided Reaction," IEEE
Transactions
on
Systems,
Man
and
Cybernetics, vol. 20, pp. 1370-1382, 1990.
[Pell et al. 96] Barney Pell, et al. "A Remote Agent
Prototype for an Autonomous Spacecraft."
Proceedings of the SPIE Conference on Optical
Science, Engineering, and Instrumentation,
1996.
[Rosenblatt89] J. Kenneth Rosenblatt and David W.
Payton "A Fine-grained Alternative to the
Subsumption Architecture." Proceedings of the
AAAI Stanford Spring Symposium Series,
1989.
[Schoppers87] Marcel Schoppers,=. "Universal Plans
for
Reactive
Robots
in
Unpredictable
Domains." Proceedings of the International
Joint Conference on Artificial Intelligence
(IJCAI), 1987.
