The Internet Still Might (but Probably Won/'t) Change Everything:

U N I V E R S I T Y O F P I T T S B U R G H
The Internet Still Might
(but Probably Won’t) Change Everything:
Stakeholder Views on the Future of
Electronic Rulemaking
Dr. Stuart W. Shulman
Assistant Professor
Information Sciences and Public Administration
Senior Research Associate
University Center for Social and Urban Research

The Internet Still Might
(but Probably Won’t) Change Everything:
Stakeholder Views on the Future of
Electronic Rulemaking
Dr. Stuart W. Shulman
University of Pittsburgh
121 University Place, Suite 600
Pittsburgh, PA 15260
Phone: 412-624-3776
Fax: 412-624-4810
E-mail: Shulman@pitt.edu
Web site: http://shulman.ucsur.pitt.edu
Project Web site: http://erulemaking.ucsur.pitt.edu

Acknowledgements
This workshop report was made possible with a small grant for exploratory research (SGER) from
the National Science Foundation (NSF).1 Any opinions, ﬁndings and conclusions, or recommenda-
tions expressed in this material are those of the author and do not necessarily reﬂect those of the
National Science Foundation.
Dr. Jamie Callan graciously made six plenary presentations over the course of three
days. These talks were vitally important for guiding workshop participants to think critically and
knowledgeably about how human language technologies might help, or possibly hurt, the electronic
rulemaking process. Also, special thanks go to the dozen invited speakers who made valuable presen-
tations to help spur the small group discussions. They were: Bob Carlitz, Devin Ceartas, Neil Eisner,
Rosemary Gunn, Jay Lagarde, Andrew Langer, Bruce Lundegren, Tom Mattzie, Christine Reed,
Seth Schoen, Peter Strauss, and Jim Tozzi. This workshop also greatly beneﬁted from the wide range
of agency personnel and nongovernmental stakeholders who took the time to attend and actively
participate in the sessions.
The author also wishes to acknowledge the assistance of Dr. Lisa Thrane, who served as
the moderator for half the focus groups and oversaw the recording and transcription of the focus
group data. Mark Perry, Dmitri Shanin, and Jonah Katz served as effective recorders during the focus
group sessions. Jonah Katz, Dawn Harms, and Karla Stevens were tireless and articulate advocates for
wider stakeholder representation, as they made hundreds of phone calls during May 2004 to ensure
that there would be enough participants. Karla Stevens and Dawn Harms also provided timely and
effective transcription service.
Staff at the federal eRulemaking Initiative provided high-level guidance as the workshop was
being designed. The eRulemaking Initiative Director, Oscar Morales, and Deputy Director, John
Moses, were especially welcome as lunch speakers each day. Kevin Landy, minority counsel for the
Senate Government Affairs Committee, convened a useful stakeholder outreach meeting on Capitol
Hill prior to the workshop and delivered well-received remarks at lunch on June 3. Larry Brandt,
program manager in the NSF’s Digital Government program, has been instrumental in raising the
proﬁle of eRulemaking research for the last three years by supporting this effort and several other
workshops, and we were grateful for his lunch remarks on June 2.
Dr. Steven J. Balla, of The George Washington University School of Public Policy and
Public Administration (SPPPA), secured SPPPA sponsorship, as well as a location for the event, and
1 See EIA-00328914 “SGER COLLABORATIVE: A Testbed for eRulemaking Data,” which was one of four linked
SGER awards in 2003 from the NSF’s Digital Government Program.

he welcomed the participants at lunch on June 4. Kathryn Newcomer, director of the SPPPA, wel-
comed the participants at lunch on the ﬁrst two days. Clark Mercer and Pam Turner of the SPPPA
provided invaluable help with local arrangements.
Dr. Mack C. Shelley and Dr. Lisa Thrane made meticulous edits after the ﬁrst draft, saving
readers from the tangle of my late night prose. Dr. Charli Carpenter and Sheila Shulman contributed
skillfully to the conclusion. Andrew L. Malone provided careful editing assistance once the complete
draft was in place. Michael J. Panchura authored a timely and detailed memo about the primary
“messages” delivered by the report to the developers of the Federal Docket Management System
(FDMS), federal regulators and their overseers, and the regulated population. John Moses furnished
timely and informational FDMS responses to the ﬁnal draft report.
Finally, the author assumes all responsibility for any errors or oversights that might appear in
this workshop report.

About the Author
Dr. Stuart W. Shulman is an assistant professor in information sciences and public
administration at the University of Pittsburgh. He is a senior research associate
in the University of Pittsburgh’s Center for Social and Urban Research and in
the Université de Genève, European University Institute, and Oxford Internet
Institute-based E-Democracy Centre.
Dr. Shulman is the principal investigator and project director on related
National Science Foundation-funded research projects focusing on electronic rulemaking, digital
citizenship, and service-learning efforts in the United States. Over the last four years, he has been
the organizer and chair for federal agency-level electronic rulemaking workshops held at the Council
for Excellence in Government (2001), the National Defense University (2002), the National Science
Foundation (2003), and The George Washington University (2004). For the last two years, Dr.
Shulman has served on the program committee for the NSF’s National Conference on Digital
Government Research.
Dr. Shulman is a contributing editor for the international Journal of E-Government and the
2004–05 president of the American Political Science Association’s organized section on Information
Technology & Politics. He teaches courses on American national government, environmental policy,
digital citizenship, service-learning, and ﬁlm and politics.

Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

Ten Major Themes Addressed in the Body of the Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Workshop Outreach and Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Workshop Proceedings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Summary of the Major Workshop Themes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

1. Tools for Electronic Rulemaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

2. Usability of the New Tools and the Federal Docket Management System (FDMS) . . . . . . .21

3. The Structure of the FDMS and the Comment Process Itself . . . . . . . . . . . . . . . . . . . . . . . . .23

4. Legitimacy Gains and Losses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25

5. Issues Related to the Law and Litigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

6. Collaboration: Opportunities and Barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28

7. Innovation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

8. Educational Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

9. Level and Nature of Public Participation: Quantity vs. Quality . . . . . . . . . . . . . . . . . . . . . . . .32

10. Overall Costs and Beneﬁts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Appendix A

Focus Group Protocol: eRulemaking Workshop 2.0, June 2–4, 2004 . . . . . . . . . . . . . . . . . . . . . .37
Appendix B

Afﬁliations of Workshop Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Appendix C

Workshop Session Schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

7

r
y
a
m
m
u

S
e
t
i
v
u
c
Executive Summary
e
x
E
As part of an exploratory study, the “eRulemaking Research Group” convened a series of six work-
shop sessions at The George Washington University, in Washington, D.C., June 2–4, 2004.2 This
report covers the major themes presented and discussed at the workshop as well as the context in
which it was undertaken. Full transcripts from the focus group sessions are available online.3
The eRulemaking Research Group was formed at a January 2003 National Science
Foundation-sponsored workshop, entitled “E-Rulemaking: New Directions for Technology and
Regulation,” held at the John F. Kennedy School of Government at Harvard University.4 Following
the Harvard workshop, computer scientists Eduard Hovy (University of Southern California)
and Jamie Callan (Carnegie Mellon University) collaborated with social scientists Stuart Shulman
(University of Pittsburgh) and Stephen Zavestoski (University of San Francisco) to submit four
successful, linked small grants for exploratory research to the National Science Foundation (NSF).5
Since its formation, the eRulemaking Research Group has participated in and organized
workshops, made presentations to federal agencies, launched an eRulemaking text data testbed,6 and
collaborated with ﬁve federal agencies in the successful submission of a larger proposal to the NSF’s
Digital Government program.7 This four-year study, undertaken in the fall of 2004, will expand and
upgrade the scope and function of the eRulemaking testbed and systematically feedback information
from users in and out of government via workshops, focus groups, Web surveys, reports, publications,
and presentations.
In conjunction with the workshop, a Web survey was launched in early June 2004. This
survey instrument will remain open over the multiyear life of the project, allowing for structured
longitudinal studies of attitudes about electronic rulemaking as well as open-ended feedback to guide
both researchers and developers of electronic rulemaking applications. Results from the survey are
also posted to the Web.8
Advertised as “eRulemaking Workshop 2.0,” this set of meetings was a follow-up to a two-
day eRulemaking Research Group workshop held September 4–5, 2003, at the NSF headquarters in
Arlington, Va. In preparation for the fall 2003 meeting, agency personnel and interest group repre-
2 The GWU Workshop page can be found at http://snipurl.com/7hh5.
3 See http://erulemaking.ucsur.pitt.edu/group_report.htm.
4 The Kennedy School of Government workshop Web site can be found at: http://snipurl.com/91ic. Harvard University
also hosts the most comprehensive clearinghouse of electronic rulemaking documents at: www.ksg.harvard.edu/cbg/rpp/
erulemaking.
5 For more information on the project history, see http://erulemaking.ucsur.pitt.edu.
6 Carnegie Mellon University hosts the rulemaking text data testbed, which serves as a clearinghouse for researchers,
agency personnel, and the public. Large public comment data sets (e.g., USDA’s organic or DOT’s CAFE rulemakings)
are available to download and search using test versions of new human language technologies. See http://hartford.lti.
cs.cmu.edu/callan/Projects/ERulemaking/Data.html.
7 The ﬁve agencies are: the Department of Transportation, the Environmental Protection Agency, the Bureau of Land
Management, the U.S. Department of Agriculture’s Animal and Plant Health Inspection Service, and the U.S. Forest
Service Content Analysis Team.
8 For survey access and results, see http://erulemaking.ucsur.pitt.edu/survey.htm.

8

r
y
sentatives were invited to come on separate days to hear about and discuss the research and collabora-
a
m
tive partnerships our group was exploring. Data generated through focus groups at those sessions
m
u
became the basis for a successful March 2004 grant submission to the NSF.9 The fall 2003 session

S
e
also made clear the need to do two things differently at the June 2004 workshop. As a result, we
t
i
v
u
expanded and diversiﬁed the nongovernmental stakeholder participants, and we allowed govern-
c
e
x
mental and nongovernmental representatives the opportunity to interact during the same sessions as
E
well as over lunch.
As this report and the transcripts demonstrate, the workshop discussions were wide ranging
and in most instances did not result in a consensus. The scope and nature of public participation in
regulatory rulemaking is shifting, perhaps signiﬁcantly. A number of new opportunities exist to use
information technology and the Internet to more fully realize the intent of the “notice and comment”
process set out in the Administrative Procedure Act (5 U.S.C. Section 553). By fostering large,
dialogical, and heterogeneous public comment datasets and building appropriate tools to analyze
them, federal agency personnel can expect to complete critical regulatory actions with the best
available information.
Some call it democracy’s cutting edge: the potential for distributed, reﬂexive, transparent,
information-rich, asynchronous, widespread, low-cost, meaningful, and transformative participation
in timely decision making. Others are less sanguine. Many fear a surging wave of electronic mass
submissions will overwhelm and thus delay agencies with limited resources. Furthermore, some warn
us that electronic rulemaking may already have instantiated a sense that rulemaking decisions are akin
to a plebiscite. Thus a big question for many observers is: Does electronic rulemaking matter?
A growing electronic rulemaking (eRulemaking) research and practice community is starting
to shed light on this,10 yet the answer will, in many respects, remain subjective. How one deﬁnes an
observable measure of better procedures or improved rulemaking outcomes is highly dependent on
one’s location inside or outside the system. Nonetheless, some scholars are predicting a “revolution,”
or at least a positive deliberative evolution, as a result of electronic rulemaking.11
To understand better the actual impact, new metrics will be developed and implemented
over time. In part, these new metrics will be performance measures that capture indications of
improved agency efﬁciency, cost effectiveness, transparency, and responsiveness.12 Other baseline data
will emerge on how public users and agency ofﬁcials interact with electronic dockets and respond to
new human language technologies.13 It bears repeating, even though it has been noted before, that
much of this work remains to be done. The interdisciplinary subﬁeld of electronic rulemaking is just
now coalescing after three years of NSF-funded workshops and focus groups. There are signiﬁcant
9 See http://erulemaking.ucsur.pitt.edu/grant_proposal.htm.
10 Cary Coglianese. 2004. “E-Rulemaking: Information Technology and Regulatory Policy,” Regulatory Policy Program
Report No. RPP-05. Available at: http://snipurl.com/82pi. Coglianese summarized four broad and now widely accepted
goals for new rulemaking technologies as: (1) increase democratic legitimacy, (2) improve regulatory policy decisions, (3)
decrease administrative costs, and (4) increase regulatory compliance.
11 Beth Noveck. 2004. “The Electronic Revolution in Rulemaking,” which can be downloaded at http://snipurl.com/88jf.
Noveck asserts that “this radical overhaul of the administrative process is conducted in a closed and almost secretive
manner without public consultation.” Also see Barbara H. Brandon and Robert D. Carlitz. 2002. “Online Rulemaking
and Other Tools for Strengthening Civic Infrastructure,” Administrative Law Review 54(4), 1421–1478; Stuart W.
Shulman, David Schlosberg, Stephen Zavestoski, David Courard-Hauri. 2003. “Electronic Rulemaking: New Frontiers
in Public Participation,” Social Science Computer Review 21(2), 162–178.
12 Lee Mordecai. 2004. “E-Reporting: Strengthening Democratic Accountability,” The Business of Government Sixth
Anniversary Issue. See www.businessofgovernment.org.
13 Stuart Shulman, Jamie Callan, Eduard Hovy, and Stephen Zavestoski. 2004. “SGER: A Testbed for eRulemaking,”
forthcoming in the Journal of E-Government.

9

unanswered questions regarding the information technology to develop and data to collect. As well,
r
y
a
there are uncertainties about what to ask of the data, government partners, and other electronic
m
m
rulemaking stakeholders.14
u

S
The current Bush Administration E-Government initiatives at the federal level are predi-
e
t
i
v
cated on “citizen-centric” changes in service delivery.15 Information technology and e-commerce
u
c
e
techniques, in particular, offer interesting possibilities for tracking the development of public com-
x
E
ments and rule development. Some early hunches about why electronic rulemaking indeed matters
may well prove naïve, or at least overly optimistic.16
Rulemaking, after all, is embedded in politics that can trump all other factors.17 Scholars
have speculated electronic rulemaking may result in a more powerful democratic corrective mecha-
nism. It indeed may prove to be an enabler of better comments, higher quality deliberation, and more
durable rules. More widespread use of IT in rulemaking may bridge divides between experts and lay
persons or traditionally adversarial interests. Most likely, however, the answer to the question (“Does
electronic rulemaking matter?”) will appear slowly in several unexpected ways, as citizens, interest
groups, and government innovate and make counter-adjustments in response to the parameters of
the new digital landscape.
The overarching ﬁnding growing out of these meetings and presented here is that there
is very little agreement and much uncertainty about how rulemaking procedures or outcomes will
change as electronic rulemaking matures. Indeed, after reviewing these sessions, it was tempting to
entitle this report “The Internet Changes Nothing,” as a rejoinder to Stephen Johnson’s compelling
1998 argument in the Administrative Law Review article, “The Internet Changes Everything.”18
Both titles err on the side of oversimpliﬁcation; hence, my bets are hedged in the actual title. This
report suggests that the Internet still might (but probably won’t) change everything for reasons
set out below.
Professor Johnson and others writing at the time persuasively held out the possibility for
Internet-driven innovation, which has, in many ways, come to fruition.19 For those with reliable
14 There are signiﬁcant “Digital Government” research challenges associated with having multiple government agencies as
partners while studying and also tweaking the effect of this particularly important citizen-government interface. Apart
from the obvious conﬂict of formally studying a process that federal funding is designed to aid, there are issues posed
by the appearance to both governmental and nongovernmental organizations that the research is part of the federal
government’s overall effort. The best hope is that an honest broker role is preserved for the wider academic eRulemaking
research community.
15 See www.whitehouse.gov/omb/egov for updated information on the federal E-Government Initiatives. Congress has
been critical of limited progress while appropriating insubstantial sums for cross-agency efforts. On the progress to date,
see the GAO report “Electronic Government: Initiatives Sponsored by the Ofﬁce of Management and Budget Have
Made Mixed Progress” GAO-04-561T available at: www.gao.gov/new.items./d04561t.pdf. Dave McClure, of the Council
for Excellence in Government, notes the “E-Government Initiatives are sometimes victim to overblown expectations.”
See David Perera. 2004. “E-gov to get PR Makeover,” in FCW.Com (September 10) at www.fcw.com.
16 Stuart W. Shulman. 2003. “An Experiment in Digital Government at the United States National Organic Program,”
Agriculture and Human Values, 20: 253-265; Thomas C. Beierle. “Discussing the Rules: Electronic Rulemaking and
Democratic Deliberation,” Resources for the Future Discussion Paper, 03-22; 2003; www.rff.org/rff/Documents/RFF-
DP-03-2.pdf (accessed June 2003); Stephen Zavestoski and Stuart W. Shulman. 2002. “The Internet and Environmental
Decision-Making,” Organization and Environment, 15(3): 323–327; Robert D. Carlitz and Rosemary W. Gunn. 2002.
“Online Rulemaking: A Step Toward E-Governance,” Government Information Quarterly, 19: 389–405.
17 See Amy Goldstein and Sarah Cohen, “Bush Forces a Shift in Regulatory Thrust,” The Washington Post (August 15,
2004), A1, which was the ﬁrst of a series of three in the Post on recent regulatory politics, and which appeared about the
same time as Joel Brinkley, “Out of the Spotlight, Bush Overhauls U.S. Regulations,” The New York Times (August 14,
2004).
18 Stephen Johnson. 1998. “The Internet Changes Everything: Revolutionizing Public Participation and Access to
Government Information through the Internet,” Administrative Law Review, 50: 277–337.
19 Senator Patrick Leahy. 1998. “The Electronic FOIA Amendments of 1996: Reformatting the FOIA for Online Access,”
Administrative Law Review, 50: 339–344.

10

r
y
Internet access and the necessary skills and desire, a vast array of new information is now accessible,
a
m
searchable, and reproducible via the World Wide Web. The existing technology employed by most
m
u
agencies has delivered many new opportunities, as predicted, but the practical importance of this

S
e
innovation remains murky. Johnson argued the use of the Internet in rulemaking could increase
t
i
v
u
the transparency, efﬁciency, participation, accountability, and legitimacy in a process characterized
c
e
x
by “obvious shortcomings.” Computers and the Internet, wrote Johnson, “can dramatically expand
E
public access … broadening the inﬂuence of citizens.” Six years later, it remains uncertain whether
the technology has delivered many beneﬁts beyond the periodic spike in public participation and
education fostered by organized interest groups that have traded in their postcards for the lure of the
mass e-mail and Web site-driven awareness campaigns.
Readers of this report will come away with lessons about three key sets of actors, whose
choices will critically impact the future of electronic rulemaking.
1. Agency rule-writers and managers shared their insights and learned more about
how and why public participation in rulemaking is changing. A common set of concerns
emerged about the quantity and quality of electronic submissions, and the related issues
of usability, public education, and comment management.
2. Members of nongovernmental organizations (NGOs) were able to envision
uses of information technology that might increase, or undermine, the efﬁcacy of their
memberships’ engagement with the rulemaking process. The report concludes they
might “inadvertently petition themselves into obscurity” as a result of unleashing vast
quantities of duplicative though similar comments using information technology.
3. Developers of the Federal Docket Management System (FDMS) learned
that a signiﬁcant and diverse cross-section of stakeholders in rulemaking think that the
nature and scope of public participation will be impacted critically by current and future
design choices, and that those decisions ought to be made in an open and transparent
manner with sustained public input.
The stakeholders at these sessions highly valued ﬂexibility, transparency, and an open,
innovation-inducing architecture for the new FDMS. While much uncertainty about the impact
of electronic rulemaking remains, few people doubt there is likely to be some transformative result
driven by these technological innovations.
Ten Major Themes Addressed in the Body of the Report
1. Tools for Electronic Rulemaking
2. Usability of the New Tools and the Federal Docket Management System (FDMS)
3. The Structure of the FDMS and the Comment Process Itself
4. Legitimacy Gains and Losses
5. Issues Related to the Law and Litigation
6. Collaboration: Opportunities and Barriers
7. Innovation
8. Educational Impact
9. Level and Nature of Public Participation: Quantity vs. Quality
10. Overall Costs and Beneﬁts

11
t
s
n
a
i
p
r
t
i
c
a
Workshop Outreach and Participants

P
d
n

a
h
c
a
In April 2004, an electronic call for participants was issued via IT- and E-Government-related list-
t
r
e
u
serves and through our professional eRulemaking network. It was designed to encourage input from

O
p
a diverse set of actors with a clear stake in the development of the federal eRulemaking Initiative.20
o
h
s
Participants and session-speciﬁc speakers were invited from six broadly deﬁned stakeholder constitu-
r
k
o
encies: (1) larger businesses, (2) smaller businesses, (3) labor and environmental advocacy groups,
W
(4) good government and public participation groups, (5) state and local government, and (6) the legal
and lobbyist professions. We asked for participants who were willing to attend the speciﬁc half-day
session devoted to their constituency and that only individuals committed to contributing to a serious
dialogue register.
Initially, the response to the electronic call for participants was cause for concern. At one
point in May, reducing or even canceling the workshop was a distinct possibility. A number of more
direct measures were undertaken to generate an appropriate number of participants from the six
constituencies. Three full-time students were hired to identify and telephone directly the appropriate
representatives in nongovernmental organizations over a two-week period in May. In addition, e-mail
lists of state and local ofﬁcials (about 1,000) and D.C.-area groups (about 1,700) were purchased,
and a mass e-mail of the workshop call for participants was distributed. As a result of these efforts, a
sufﬁcient number of participants registered, and the workshop went ahead as planned.
In total, counting the two principal investigators and the four student assistants, 64 individu-
als participated over the course of the three days. A total of 10 focus groups were conducted, ranging
in size from seven to 11 participants. Not counting the researchers and research assistants, a total of
seven individuals participated in more than one session. There were 33 participants who chose to
keep their identity conﬁdential, while 31 indicated attribution of their comments in future publica-
tions was permissible. None of the comments presented in this report are attributed to their speaker.
Federal agency personnel and academics were welcome at any of the sessions. Participation was free,
and lunch was provided. Our expressed intent was to share the ﬁndings, via this report, with govern-
ment agency managers, members of Congress, IT contractors, academics, and the general public.21
20 For more on the federal eRulemaking Initiative see: www.regulations.gov/eRuleMaking.cfm.
21 See Appendix B for the afﬁliations of the workshop participants.

12
s
g
i
n
d
e
e
c
r
o

P
p
o
h
Workshop Proceedings
s
r
k
o
W
Participants heard approximately 90 minutes of presentations. At each workshop plenary session, par-
ticipants were welcomed and provided an overview of the eRulemaking Research Group’s activities, as
well as some of the trends and issues in electronic rulemaking and the history of the workshops.22 In
May 2001, the ﬁrst workshop of this kind brought together a group of about 50 academics, students,
and personnel from seven federal agencies at the Council for Excellence in Government (CEG) with
funding from the NSF and Drake University. At that time, the E-Government Act of 2002 was a
fairly obscure bill in Congress, and the idea of a consolidated Federal Docket Management System
(FDMS) would have struck many as unrealistic due to the variegated contours of the administrative
landscape and the territoriality of the major rulemaking agencies.23
At the May 2001 CEG workshop, we proposed linking government, the IT industry, and
academia to promote technology transfer that would foster interdisciplinary studies under the
auspices of Digital Government research. Our research questions ranged over the possible uses of
IT to serve democratic ideals to the impact of different IT enhancements on the analytical ability of
citizens and agency personnel. We proposed to leverage new research funding opportunities at the
NSF to set up annual end-user and technologist meetings that would build social capital around the
idea of improved rulemaking.
A great deal has changed since then. With approval of the E-Government Act in 2002 and
a far-reaching Presidential Management Agenda,24 the eRulemaking Initiative and its centerpiece,
the government-wide FDMS, became an imminent fact of regulatory life for agency personnel and
stakeholders. Concurrently, the nongovernmental organizations most attuned to rulemaking have
been developing new strategies (or at least digitizing their old strategies) to take advantage of infor-
mation and communications technologies. Indeed, a new sub-sector of the economy is taking root to
provide electronic advocacy services for trade associations and public interest groups.25 Of particular
interest to our research group are those innovations that may increase the likelihood and manageabil-
ity of massive stakeholder responses in controversial rulemakings.26 The fear of information overload
in this context is multifaceted, encompassing concerns about ossiﬁcation of the rulemaking process as
a result of what some describe as “spam-like” submissions as well as a denigration of the role of public
input in regulatory decision-making, to name just a few.
22 All of the workshop presentations are collected at: http://erulemaking.ucsur.pitt.edu/talks.htm. See the workshop
schedule on pages 39-40.
23 See http://erulemaking.ucsur.pitt.edu/workshop.htm regarding the history of the workshops. On the irregular nature
of rulemaking within and across agencies, see the GAO report “Regulatory Reform: Procedural and Analytical
Requirements in Federal Rulemaking” GAO/T-GGD/OGC-00-157 available at: www.gao.gov/new.items/g800157t.pdf.
24 See www.whitehouse.gov/omb/egov/pres_state.htm on the E-Government Act and www.whitehouse.gov/omb/budget/
fy2002/mgmt.pdf for the President’s Management Agenda.
25 Ex., see www.ctsg.com, www.getactive.com, or http://capitoladvantage.com.
26 For example, recent Environmental Protection Agency (EPA) notices about possible rulemakings in the Federal
Register concerning the deﬁnition of U.S. waters and mercury emissions each have resulted in hundreds of thousands of
e-mailed public comments. About 15 EPA EDOCKET staff members recently completed sorting through over 680,000
mercury comments, manually identifying the unique comments for inclusion in the docket, which amount to about
4,200 distinctive comments.

13
At every workshop session, Dr. Jamie Callan (Carnegie Mellon University) showed simple
s
g
text analysis techniques that can be used to organize and summarize the contents of large public
i
n
d
e
comment databases. He asked that attendees share their responses about what tools might or might
e
c
not be useful in the future. Public comments, Dr. Callan noted, are expressed in human language.
r
o

P
Although computers cannot understand human language the way people do, they can still be useful in
p
o
helping people make sense of large public comment databases.27
h
s
r
k
Text analysis software can identify letters that are exact duplicates (e.g., form letters from
o
W
a letter-writing campaign) and near-duplicates (e.g., form letters that people have modiﬁed to
represent their opinions better). Simple phrase recognition techniques can identify concepts that
people mention frequently, which can serve as a starting point for “drill down” activities that examine
comments addressing particular topics or points of view. People often identify their roles with respect
to a particular regulation—for example, “As a mother, I believe …,” or “I have been a truck driver
for 25 years and …” Relatively simple techniques can ﬁnd and organize such references, enabling
policy makers, rule-writers, and other interested parties to understand better who commented on a
particular aspect of the rule.
These and a wide variety of similar techniques are possible in the near future. Today regula-
tory agencies are struggling with basic IT issues related to capturing public comments electronically.
Soon these will be mastered, and attention will turn to better use of language analysis and text mining
software. At present there is an opportunity to provide better tools for rapidly analyzing large public
comment databases, and consequently for increasing transparency and efﬁcacy in the comment
submission and analysis process.
At lunch each day, the morning and afternoon groups together heard from and questioned
Oscar Morales, director of the federal eRulemaking Initiative. Mr. Morales stressed the importance
of feedback from agencies and stakeholders in the development of requirements for the FDMS. He
described a three-module organization of the initiative. Module 1, which is complete and operational,
created a Web portal that allows users to search for and comment on any open federal rulemaking.28
He provided an overview of the commercial off-the-shelf (COTS)-based architecture (Oracle and
Documentum) for the FDMS, which will be Module 2. User testing of Module 2 was projected for
fall 2004 (ﬁrst with agencies followed by stakeholders), with the ﬁrst agencies migrating to the system
early in 2005. Mr. Morales then provided a broad outline of Module 3 plans for an online regulatory
tool-kit which will blend IT tools with improvements in institutional rulemaking practices to create
Internet and desktop tools that help gather, process, analyze, and communicate information in the
rule-writing process. He noted the output from the workshop would directly inform the development
process of Modules 2 and 3.
In addition, invited speakers made session-speciﬁc presentations focusing on the speciﬁc
needs of the stakeholder community. At the ﬁrst session, targeting representatives of larger busi-
nesses, Bruce Lundegren, director of regulatory affairs at the U.S. Chamber of Commerce,29 stated
that the success of the FDMS will be measured by how well the new software ensures public partici-
pation in the regulatory process and facilitates meaningful input from interested stakeholders. From
the U.S. Chamber’s perspective, transparency and data quality are the key goals that must underpin
all aspects of the eRulemaking Initiative. Mr. Lundegren discussed E-Government as a revolutionary
27 See Dr. Callan’s presentation on tailored text analysis tools online at: http://snipurl.com/99q4.
28 See www.regulations.gov.
29 See www.uschamber.com for the U.S. Chamber of Commerce.

14
s
g
concept that will be hugely beneﬁcial if properly implemented. He argued that IT should help ensure
i
n
d
that the voice of the business community is heard, and that it will allow the business community to
e
e
c
respond to issues raised by other stakeholders. This open debate should help improve the quality of
r
o
agency rules.

P
p
o
Mr. Lundegren stated that all rules should be based on clear statutory authority, sound
h
s
science, and rigorous technical analysis. He also said that regulatory dockets should include all back-
r
k
o
ground materials, public comments, and legal/technical analyses outlining how the agency reached
W
its decision. Finally, Mr. Lundegren said it would be interesting to make historic dockets available to
reassess what agencies did in the past, but acknowledged that the huge costs associated with such an
effort will probably require agencies to prioritize which historic dockets would be made accessible ﬁrst.
At the second session, Andrew Langer, of the National Federation of Independent
Business,30 spoke to a group assembled to consider electronic rulemaking issues for smaller busi-
nesses. Mr. Langer emphasized the unique challenges small business owners face in terms of regula-
tory compliance, with a particular emphasis on paperwork burdened hours. He noted regulatory
costs are higher per employee for smaller businesses that often lack the specialized personnel and
state-of-the-art tools for understanding and complying with federal regulations. Mr. Langer provided
an example of a ﬁrst-time ﬁler spending up to 60 hours completing paperwork for the EPA’s Toxic
Release Inventory. He called for innovations that ease the burden related to regulatory compliance
and which facilitate better partnerships between stakeholders and regulators.
The invited speakers for the third session, targeting labor and environmental groups, were
Tom Mattzie, an organizer with the AFL-CIO, and Devin Ceartas, a private sector Web program-
mer with NacreData who has worked with many organizations in the sustainable agriculture and
forest protection movements.31 Tom Mattzie spoke about the need to consider the different roles
that NGOs play as intermediaries relaying word of important decisions and the opportunities to
comment. His online advocacy resulted in 50 million e-mails to members and ﬁve million comments
or letters to decision makers over the last two years. For Mr. Mattzie, the e-advocacy model is one
that relies on using an e-mail list to grow and mobilize the union membership around issues that are
important to working people.
Devin Ceartas identiﬁed two factors that framed his comments. First, he expressed concern
about the digital divide. For Mr. Ceartas, while many low-income and/or rural individuals and groups
would welcome the arrival of broadband as well as the skills and equipment to make the most of it,
access is far from universal and some communities are likely to remain behind the curve for the fore-
seeable future. In the case of Mennonite and Amish communities affected by federal farm policies, as
well as those choosing to live “off the grid,” the divide is intentional and will never go away. In many
cases, those with lower levels of Internet access correspond directly to the groups of people most in
need of the government programs for which rules are being developed. These groups have always had
less access to government. Such people have the same rights as any other group of citizens.
Second, Mr. Ceartas suggested his relationship with agency rule-makers was adversarial,
rather than cooperative. In part, this stemmed from dissatisfaction with agency decisions. In other
cases, however, the antagonism involved difﬁculties with comment submission in the past. He noted
some of the agency “Spam” ﬁlters encountered have been crude and overly broad. Mr. Ceartas argued
systems that block IP numbers or rely on reverse DNS mapping run the risk of throwing away many
30 See www.nﬁb.com/page/home.html for the National Federation of Independent Business.
31 See www.aﬂcio.org for the AFL-CIO and www.nacredata.com for NacreData.

15
legitimate e-mails and create a particular disadvantage for smaller organizations, which often use
s
g
shared name-based servers. He concluded that organizations will strongly resist redirecting visitors
i
n
d
e
from their Web sites to an agency or government-wide Web portal to make comments, since the
e
c
chance to donate or volunteer via the NGO Web site is a primary reason for its existence.
r
o

P
For the fourth session, targeting good government groups, Robert Carlitz and Rosemary
p
o
h
Gunn, of Information Renaissance, made a joint presentation. They were followed by Seth Schoen,
s
r
k
of the Electronic Frontier Foundation (EFF).32 The objective of Information Renaissance, said
o
W
Carlitz and Gunn, is an e-rulemaking system that facilitates broader public involvement and deeper
discussions of the issues behind the rules. They presented an overview of features of value to indi-
viduals and organizations. The basics include a system that encourages participation by meeting high
usability standards, providing active notiﬁcation, and presenting background information about rules
and rulemaking; and promotes transparency, accountability, use of public input—and in the end
“better” rules. More advanced features include structured input forms (to help assure meaningful
comments), rebuttal comments (and eventually online dialogue), open modeling (to help in under-
standing technical data), a continuous electronic record (to link rulemaking documents to what comes
before and after a notice and comment process) and Web services. Web services are of special interest
to organizations that work with rules. They could allow, for example, analysis with local tools, sub-
mission of complex sets of comments, bulk data transfers, and development of alternative front ends.33
Seth Schoen spoke about the lack of public awareness of rulemaking. Citizens often don’t
know it exists, how it works, that the public can participate, what subjects it addresses, and that
particular rulemakings are taking place. He argued it is the role of advocacy organizations to increase
public awareness of and participation in rulemaking proceedings, since relatively few individuals will
learn about them on their own from agencies, as opposed to through advocacy organizations.
Mr. Schoen asserted agencies should never reject, discount, or ignore duplicative comments.
Agencies, he noted, are required by law to consider comments and make them a part of the public
record, although they are certainly welcome to use software to help them organize comments and
identify which are duplicative. Preserving the full record is important to many entities for their own
reasons. The ability to send form letters helps make people aware of issues, familiarize them with
the comment process, and causes them to think of themselves as people who can and do comment
on proposed regulations. As a result, he suggested they will be more willing and able to participate
in other ways in the future. In short, the ability to send form letters has value to people outside of
agencies, for example, political advocacy value to advocacy organizations and their constituents.
Mr. Schoen stated that advocacy organizations would like programmatic input and output
from the docket management system (for example, by means of a Web services interface). This would
facilitate integration with “action centers” that help people submit comments through an advocacy
organization’s Web site, which is and probably will continue to be overwhelmingly the way that non-
specialists learn about proceedings and comment. He advocated for ﬂexible searches and notiﬁca-
32 See www.info-ren.org for Information Renaissance and www.eff.org for the EFF.
33 For a more detailed version of the Information Renaissance position, quick search for the public comment with the
Docket ID No. OEI-2004-0002-0010 at http://docket.epa.gov/edkpub/index.jsp, where the EPA opened a docket to host
a public forum on the eRulemaking Initiative and the development of the FDMS.

16
s
g
tions, whether via e-mail or Rich Site Summary (RSS),34 and the ability to design custom notiﬁcations
i
n
d
with high speciﬁcity, in addition to bulk data retrieval from dockets.35
e
e
c
State and local government needs were the focus at the ﬁfth session. Christine Reed of the
r
o
University of Nebraska, Omaha, representing the National Association of Schools of Public Affairs

P
p
o
and Administration (NASPAA), was followed by Jay Lagarde, of the Virginia Department of Planning
h
s
and Budget.36 Dr. Reed is the principal investigator in Nebraska for the most recent round of the
r
k
o
Small Communities Outreach Project for Environmental Issues (SCOPe), a project funded through
W
a Cooperative Agreement between NASPAA and the U.S. EPA Ofﬁce of Policy Economics and
Innovation. The purpose of SCOPe is to engage small, local government elected ofﬁcials and admin-
istrators more effectively in the rulemaking process. Since 1998, NASPAA member schools and their
Public Administration faculty have met with over 350 small communities in 20 states to discuss clean
air, safe drinking water, and other EPA rules as they were being developed by agency rule-writers.
Dr. Reed discussed the initial results of the most recent round of SCOPe, which focused
on the eRulemaking Initiative, rather than a speciﬁc rule. Although two other states were involved
(Pennsylvania and Indiana), Dr. Reed reported only on her own study in Nebraska. She visited 16
different sites across the state. Local participants included city administrators, municipal clerks, and
county clerks, and all had access to high-speed Internet. None of the participants were aware of the
eRulemaking Initiative, but once Dr. Reed demonstrated use of the new portal all agreed it would
be easy to access for purposes of submitting comments. All stressed the continued need for member-
ship associations (e.g. League of Nebraska Municipalities and Nebraska Association of County
Organizations) to “ﬁlter” the large amount of technical and legal information contained in each
Notice of Proposed Rule Making (NPRM) and to alert members about particular rules and salient
issues. They stressed the institutional constraints affecting their participation as small local govern-
ments: limited human and ﬁnancial resources, as well as limited expertise needed to analyze complex
rules and associated impact analyses. All stressed the fact that despite access to high-speed Internet,
small local governments were still at a disadvantage compared to larger entities in making an impact
on the development of federal rules.
Jay Lagarde noted that Virginia has been a leader in the e-rulemaking revolution since 1999,
when it ﬁrst launched the Virginia Regulatory Town Hall Web site. The Town Hall is a Web-based
system for managing, accessing, and participating in state rulemaking. In addition to managing rules
review and other internal aspects of the rulemaking process, the Town Hall also provides public
access to current and historical rulemaking documents; regulatory meetings, meeting agenda, and
minutes; guidance documents; legislative mandates requiring new regulations; custom tracking
reports; and other information useful for public participation. Users of the Town Hall can receive
free, customized e-mail notiﬁcations about regulatory meetings and events. Users also can view and
enter comments on regulations through the Town Hall’s public comment forums.
At the sixth session, Friday afternoon, it was decided that 10 focus groups provided suf-
ﬁcient data to draw substantive conclusions. As a result, the workshop concluded with a three-hour
34 “Short for RDF Site Summary or Rich Site Summary; an XML format for syndicating Web content,” a Web site that
wants to allow other sites to publish some of its content creates an RSS document and registers the document with an
RSS publisher. A user that can read RSS-distributed content, can use the content on a different site. Syndicated content
includes such data as news feeds, events listings, news stories, headlines, project updates, excerpts from discussion forums,
or even corporate information (Source: http://snipurl.com/89bl).
35 For a more detailed statement of the EFF position, quick search for the public comment with the Docket ID No. OEI-
2004-0002-0012 at the EPA’s public forum at http://docket.epa.gov/edkpub/index.jsp.
36 See www.naspaa.org for NASPAA and www.dpb.state.va.us for the Virginia Department of Planning and Budget.

17
plenary session targeting lawyers and lobbyists. Neil Eisner, an assistant general counsel at the U.S.
s
g
Department of Transportation (USDOT), followed Peter L. Strauss, of the Columbia Law School.37
i
n
d
e
Peter Strauss presented ﬁndings from a recent survey of members of the American Bar
e
c
Association’s Section on Administrative Law and Regulatory Practice.
r
o
38 He framed his presentation

P
with a set of questions about the impact of electronic rulemaking: At what point in the process should
p
o
h
agency deliberations become part of the docket? How will the character of public participation
s
r
k
change? Will a centralized, electronic docket further transform the role of the White House, creating
o
W
new issues related to information control and transparency?39 In sum, Professor Strauss noted the
move from expertise to a form of plebiscite, machine-generated comments, and the possible shift
from one-way to dialogical commenting, were among the issues requiring greater attention.
Neil Eisner spoke about six legal issues involved in Internet-accessible dockets. He stressed
that some are not new issues, but are exacerbated by placing the dockets on the Internet. The ﬁrst
issue is whether electronic records can be a legal record or whether the agency would also have
to keep a hard-copy record. Mr. Eisner believes this issue already has been resolved and that the
electronic record can be the legal record. Another issue is whether agencies can accept anonymous,
or “Jane Doe,” comments. The third issue is whether agencies must censor documents they receive to
delete obscene material, especially because children would have easier access to the material online.
There is a similar issue concerning copyrighted material, since it can be copied more easily via the
Internet. Due to the ease with which individuals can search electronic, Internet-accessible dockets,
there is also the question of whether privacy rights would be violated. Finally, Mr. Eisner noted the
issue of whether agencies must prevent information they would have reason to believe has been
obtained illegally from being placed in the docket.
After a break in all but the ﬁnal half-day session, the participants split into two focus groups
for the remainder of the time. The smaller groups engaged in a facilitated and recorded discussion of
a series of open-ended questions about the practical and political impact of information technology
on the “notice and comment” process (see Appendix A for the focus group protocol). The discussion
questions sought to elicit feedback that could help shape the development of requirements for future
eRulemaking research and practice. Participants signed informed consent forms allowing them to
choose anonymity or disclosure regarding their identity. What follows is a summary of the discus-
sions drawn from all ten focus groups. Ten major themes emerged as salient to the participants in
these sessions.
37 See www.dot.gov for the USDOT and www.law.columbia.edu for the Columbia Law School.
38 Peter Strauss. 2004. “The ABA’s Ad Law Section’s E Rulemaking Survey,” Administrative Law and Regulatory News,
29(3): 8-10.
39 The role of the Ofﬁce of Information and Regulatory Affairs (OIRA) within the Ofﬁce of Management and Budget
(OMB), described in a recent General Accounting Ofﬁce report as “not well documented or well understood,” has come
under greater scrutiny in this regard. See the GAO report “Rulemaking: OMB’s Role in Reviews of Agencies’ Draft
Rules and the Transparency of Those Reviews” GAO-03-929 available at: www.gao.gov/new.items/d03929.pdf.

18
s
e
m
e
h

T
p
o
h
s
r
k
o
Summary of the Major Workshop Themes
r

W
j
o
a

M
1. Tools for Electronic Rulemaking
e
Many of the discussions focused on the role of new tools and technology that are, or soon will
f

t
h
be, part of the digital landscape of rulemaking. A wide range of opinions were expressed on the

o
r
y
impact of the Internet, Web portals (e.g., Regulations.Gov), electronic dockets (e.g., DOT’s Docket
a
m
Management System or EPA’s EDOCKET), government and nongovernmental organization (NGO)
m
u
S
Web sites, e-advocacy techniques, and human language technologies (e.g., Google or duplicate
detection algorithms). While a number of possible impacts and outcomes were raised, many of the
participants qualiﬁed their own remarks, or the comments of others, by suggesting the true impact of
these developments was not yet known, or perhaps even knowable.
Thus, at the very least, there was a convergence of viewpoints around the notion that
questions about the impact of these new tools were rife with uncertainty. Some suspected that the
unintended consequences of technological innovation might overshadow the hoped-for gains. One
participant pointed out that any improvements would be “illusory” if they were not designed carefully
and measured accurately. Readers of the transcripts and this report will note the frequent juxtaposi-
tion of technological impacts that constitute, in the words of one speaker, a “double-edged sword” in
the e-rulemaking environment.
As a result, more ubiquitous and accessible information in rulemaking was seen as potentially
a beneﬁt and cost—both in and out of government—that might either help or hurt the process and
its outcomes. For example, amid the general sense that more accessible information generally is a
positive development, lay a solid undercurrent of skepticism that the information “ﬂood” could be
managed. It was suggested this was true not only for individual citizens and overburdened agency
ofﬁcials, but also for many organizations with staff and resources spread thin. Unless there were
user-friendly technological and procedural innovations, electronic rulemaking would result in de facto
information overload. One participant stated:
… it shouldn’t be that, agencies publishing more and more paper inherently means,
that actually ﬁnding the part you care about has to get harder. You can argue that
if it does get harder, that’s a failing of the way the agencies have laid it out or the
way that the technologists have failed to present useful information display tools
or information navigation tools; not with the volume of data. We have phonebooks
and the phonebooks get longer and longer and longer but it doesn’t actually get
harder … to search the phonebook.
A number of participants nonetheless mentioned the likelihood of time and cost savings
derived from better online navigation tools and automated categorizing of more complete and well-

19
indexed dockets.40 In a nutshell, electronic rulemaking would be more efﬁcient and effective, over-
s
e
coming many of the shortcomings associated with paper dockets, while pushing the beneﬁts out to a
m
e
h
more geographically and demographically diverse set of stakeholders.41 Automation of mundane tasks

T
p
was called a “great time saver.” One person stated: “I want to see what my industry is saying about
o
h
s
this rule or other comments, and be able to click and say, sort by industry. Sort comments by industry:
r
k
o
printing industry. This is what this rule does for the printing industry. And that may be a huge hurdle
down the road of actually being able to sort who says what.” Other comments, however, predicted an
r

W
j
o
increase in the number of hours or dollars devoted to developing and using information technology
a

M
during both the preparation of comments and the agencies’ analytical process. Some of the top quality
e
e-advocacy services available in the current market, as one person noted, are not cheap.
f

t
h

o
The paradox for many observers is that the same technologies that make it easier to submit
r
y
a
or read public comments and aggregate supporting information also can make it both easier and
m
m
more difﬁcult to make sense of the comments and the agency’s underlying rationale for its decisions.
u
S
Several participants noted the presence and associated implications of these new tools would mean
unmanageable costs and time burdens for NGOs with limited resources, or for citizens with limited
time to master the details of navigating through a sea of documents. One asked that designers of
these tools attend to the unique needs of small business owners.42 Another thought that for some
constituencies already not used to conducting mass-mail campaigns that electronic rulemaking was
another opportunity simply beyond their range of knowledge. One person was concerned electronic
rulemaking would formalize an existing “two-tier system.”
A set of participants focused on the idea that electronic rulemaking tools would result in
better comments. Rulemaking submissions during “notice and comment” could be better informed
and expertly guided in a hyperlink-intensive, interactive, or well-structured commenting environ-
ment. The guidance could come from experts inside the agencies as well as interest groups with
specialized knowledge.43 One participant noted:
… business models are changing and the roles of intermediaries are changing. I
think the intermediaries in the past have either said, we’ll represent our members
and we’ll do the commenting or in some cases they said, we’ll organize our
members and we’ll have a mass submission; most of it kind of all the same. It’s now
possible to do things which perhaps have more impact in the process because it
generates more reasoned, more detailed comment from the members and it’s not
all identical.
40 In separate focus groups with EPA ofﬁcials in late July 2004, participants remarked that the EPA’s current EDOCKET
system lacked many of the rudimentary document categorization and indexing principles that were routine in the
paper docket era. Several mentioned the irony that the move to a rigid electronic repository without categories or
adequate navigation tools had been a step backward. In September 2004, long-planned enhancements to EDOCKET
were deployed for public and agency use, including improved categorization features and additional navigational tools.
Personal communications with the managers of the eRulemaking Initiative indicate that the FDMS will include features
such as enhanced categorization, navigation, and reporting that should beneﬁt rule-writers and the public.
41 It is possible that IT will succeed where legislative reforms have fallen short, such as in the effort to reduce the paper-
work burden. See the GAO report “Federal Rulemaking: Procedural and Analytical Requirements at OSHA and Other
Agencies” GAO-01-852T available at: www.gao.gov/new.items/d01852t.pdf.
42 The recent ﬁnal “Report of the Small Business Paperwork Relief Act Task Force,” echoed this concern when it
outlined a number of information access needs. Among these is the ability to search the full volume of Federal
information by subject or task, rather than through older structures based on agency organization or structure. See
www.whitehouse.gov/omb/inforeg/sbpr2004.pdf.
43 For more on improving the quality of public comments, see Barbara H. Brandon and Robert D. Carlitz. 2002. “Online
Rulemaking and Other Tools for Strengthening Civic Infrastructure,” Administrative Law Review, 54(4), 1421–1478.

20
s
e
The idea that information technology necessarily will result in better comments was not
m
e
universally accepted. For example, to the extent IT is employed to mobilize one-click “me too”
h

T
comments, the fear for some is that far from improving the quality of public comments, the current
p
o
h
e-advocacy model will result in a dramatic increase in comments that have little or no value to the
s
r
k
administrative process. The proliferation of these electronic postcards might give the impression that
o
the administrative process is actually a forum for direct democracy via nose-counting plebiscites.44
r

W
j
o
Dissenters spoke of the right for anyone to submit any quality of comment, with some arguing
a
effective engagement with the rulemaking process legitimately could be a secondary concern. Some

M
e
nongovernmental representatives described the notice and comment process as a forum to propel
f

t
h
political and legal strategies into the spotlight or the courts.

o
r
y
a
The gap in the ability to mobilize resources to perform these functions was noted as an
m
m
issue for participants both in and out of government. Resource-poor groups and IT-dollar-strapped
u
S
agencies cannot necessarily be counted on to provide the needed content and delivery mechanisms
required to produce better comments. One participant questioned whether it is the proper role of
government to concern itself with improving the quality of the public comments.
A number of participants also raised the notion that new tools for online collaboration might
result in innovative deliberation possibilities. The most commonly mentioned tools were listserves,
online chats, and moderated discussions. Here again, there was a recurrent tendency to temper
enthusiasm for new communications technologies with considerations that were much more social,
political, and legal in nature than they were technical. One person remarked that commenting on
comments was “just something we are going to have to live with … it has the potential to fundamen-
tally change the regulatory process.”
A key debate concerned who, or what entity, was able to make appropriate choices about the
nature and scope of new deliberative mechanisms. Many nongovernmental stakeholders questioned
whether even a well-meaning federal government can get the critical design choices right. Others
wondered if a single design choice for the FDMS would ill-serve the divergent business processes at
the 180 or so rule-writing entities in the federal government, or their many constituencies.45
Signiﬁcant concerns were expressed by some attendees that the ongoing development of
the FDMS might constrain innovation in the marketplace of ideas about electronic rulemaking. An
inﬂexible architecture for citizen government interaction might result in a vast, monolithic system
that functions poorly and draws in few users. Instead of fostering innovation in the manner of delib-
eration at the agency or sub-agency level, in the words of one participant, “you have the potential to
slow down some of the thousand ﬂowers blooming,” if the new centralized system forestalls constitu-
ency- or agency-speciﬁc creativity. The effort to build a FDMS that is “all things to all people,” noted
another participant, likely would produce a “lowest common denominator” system that rulemaking
agencies and stakeholders alike would reject or ignore. Similarly, if the FDMS was perceived as
incomplete by agencies with existing docket systems, over time duplicative systems might emerge and
44 A leading rulemaking scholar asserted the “overall trend has been away from the expertise model and toward the politics
model,” as the comments submitted increase by orders of magnitude. See Jeffrey S. Lubbers (ed.) Developments in
Administrative Law and Regulatory Practice 2002-2003 (Chicago, IL: American Bar Association), 149–150.
45 Managers of the eRulemaking Initiative indicate that both Module 1 (Regulations.gov) and Module 2 (FDMS) have been
developed as ﬂexible applications which are conﬁgurable to meet varying agency needs and business practices, while
offering the beneﬁts of a common look and feel for public access and cost savings from a consolidated system.

21
compete. Furthermore, one person stated that if the FDMS was developed in a secretive, top-down
s
e
manner, the likelihood of failure would increase.
m
e
h
Dissenting views on this point stressed that the rollout of the FDMS would solve a signiﬁ-

T
p
o
cant digital divide that exists within the federal government, between those agencies (about 30) that
h
s
have electronic docketing systems and the over 150 rule-writing entities that do not. One participant
r
k
o
remarked that it actually required a process of forcing rule-writers to use a new technology to make
r

W
them think creatively and feed back critical information about how to improve it over time. Another
j
o
a
stated that no amount of planning could overcome the complexity involved, and that trial and error

M
e
was unavoidable. Agency representatives noted they tend to worry that technical error in a rulemak-
f

t
h
ing procedure could endanger years of work on an important rule.

o
r
y
Perhaps most important, many stakeholders identiﬁed a corrective for any present or future
a
m
ﬂaws in the design of the FDMS—an open application programming interface, or API. An API is
m
u
a set of procedures and protocols that enable one software process to communicate with or control
S
another software process. One computer is providing a service (the one that publishes the API), and
the other is using the service (the one using the API). One participant explained “you publish techni-
cal information to help third parties write programs that do things on your Web site; that’s an API. So
for example if I wanted to write my own program that will search your Web site or retrieve particular
documents from your Web site or something like that; if you published a suitable API than I could
use that to do that thing.”
A clear consensus was expressed that an open API would relieve the developers of the FDMS
from the burden of making their comment submission portal “all things to all people.” For non-
governmental actors and electronic advocacy specialists, this single FDMS design choice—whether
or not to publish an open API—seemed to hold the promise that the traditional intermediaries
(the interest groups) would use their own electronic interfaces to inform their members, dissect the
docket, prepare meaningful comments, and manage high-value client lists. “This is a catch to the API
thing,” remarked one participant, “how can you do it in a way that doesn’t put a cost of mobilization
or a cost of participation on the members of speciﬁc constituency groups?” Another remarked:
It feels like the Web services API takes some of the pressure off the development
team from the government side. They can get some functionality up and running
and groups outside the government may be able to complete some of the details
and get better interfaces. I think it is this spirit of collaboration so that we have a
system we can all use. It is going to inhale a cost for groups outside the government;
however, you could have a consortium of 100 nonproﬁts pooling resources to
keep costs low.
2. Usability of the New Tools and the Federal Docket Management System (FDMS)
Workshop participants were in general agreement that the usability of electronic rulemaking systems
was a high priority.46 If future systems, such as the FDMS, were to make usability studies a central
part of an iterative development process, the perception was widely shared that electronic rulemak-
46 Managers of the eRulemaking Initiative plan to conduct agency and public usability sessions of FDMS starting in late
2004 and continuing into 2005.

22
s
e
ing would become a “great mobilizer” and a net beneﬁt to society.47 In the absence of such studies,
m
e
many participants could imagine the new FDMS being underutilized by agencies and the public.
h

T
One participant worried about “exclusivity” and the “need to acknowledge that the people who are
p
o
h
excluded are not a random selection of people.” Echoing this digital divide theme was an expressed
s
r
k
concern that new functionality will be tied to broadband access. Also, if the system is slow (with or
o
without broadband), lacks adequate categorization, navigational options, or is poorly indexed, it will
r

W
j
o
inspire “work-arounds” and off-system solutions to conduct routine business for which the system
a
was supposed to be designed.

M
e
One participant noted that future regulatory reforms, such as those promulgated by
f

t
h

o
Congress or the Ofﬁce of Management and Budget (OMB), ought to target those aspects of rulemak-
r
y
a
ing that have the effect of confusing or overwhelming non-expert participants. Some of the speciﬁc
m
m
sources of confusion noted were themselves past regulatory reforms. While it was widely thought that
u
S
IT can help in this regard, if done right, there also was a sentiment that no amount of technology can
trump the numbing effect of the Paperwork Reduction or Data Quality Acts on the ability of average
citizens to engage the process. “I think there is a hazard,” noted one person, “that there is a move
on the part of OMB which may give the agencies a blanket excuse to disregard public comment.”
For others, these measures represent a healthy “corrective mechanism” allowing the public better
opportunities to monitor agency practices.
It was noted that e-commerce ﬁrms, such as eBay and Amazon.com, pay particular attention
to every mouse click, looking for signs in their Web logs that their systems are losing or confusing
users at speciﬁc points in a transaction.
Those kinds of things would make it really usable. So I think usability testing
is going to be really critical because some of these things are so drab and so
unfriendly. You come up on a screen; there’s a bunch of words at the top and a
bunch of blank ﬁelds. A lot of people sort of stop there but catalogues have ﬁgured
out how to do it in a way that sort of keep you going. What are you looking for?
How can we help you?
One participant thought it would be desirable to have a “scorecard” that showed the nature
of the visits to electronic dockets. The most rudimentary Web log analysis would tell researchers and
practitioners a great deal about how electronic dockets are utilized.48
The hope was expressed that the federal government would improve a poor record in this
area (what one person called “the challenge of writing a reg in eBay style”) by commissioning regu-
lar and extensive user studies and borrowing the best practices from e-commerce where applicable.
The Web-based interface allows unique opportunities for anonymous tracking via session cookies;
thus the e-commerce model ought to be employed to compile baseline data on how the system is
used. Referral systems, for example, seamlessly could facilitate easier and broader dissemination of
the most commonly-viewed documents and comments, making navigation inside a complex docket
47 In August 2004, the eRulemaking Initiative convened three public meetings in Boston, San Francisco, and Washington
to solicit questions and feedback from the public on the usability of Regulations.gov and FDMS, as well as preferred
features and other suggestions for the program. The public can review and comment on the presentations given and
meeting transcripts at www.epa.gov/edocket, docket number “OEI-2004-0002.”
48 See www.deepmetrix.com, www.coremetrics.com, or www.statcounter.com for examples of commercial products
designed for precisely these evaluative functions.

23
more manageable for non-experts.49 One participant stated that a user-friendly format will help
s
e
bring about better comments.
m
e
h
Some concern was expressed about how future systems might be sophisticated technologi-

T
p
o
cally, and therefore difﬁcult to use. For example, one person noted a system that automatically chops
h
s
up complete comments and compartmentalizes them for analysis can be cumbersome analytically in
r
k
o
the absence of the full context of the original comment. Similarly, the widely discussed hope that IT-
r

W
enhanced systems will increase transparency in the process, noted some participants, can be realized
j
o
a
only if you can search for and ﬁnd what you need with the efﬁcacy and ease that many now attribute

M
e
to Google searches. A related issue was “the disappearance of information products off agency Web
f

t
h
sites,” which has been of particular concern to the library community over the last decade.

o
r
y
On the issue of the usability of the application programming interface (API), one participant
a
m
made it clear that for advocacy organizations, this simply means a lower cost to access and manage
m
u
data during both the data submission and export processes. If the government publishes a standard set
S
of API speciﬁcations that apply across all rulemakings and agencies, most of the participants seemed
to agree that the usability of the system will go up signiﬁcantly, while the pressure on the federal
government to structure its FDMS interface to suit all future users’ needs and abilities, in and out of
government, would drop considerably. One person noted:
My concern is that if you look at the folks in the government, they are just going to
set it up so that it works for them. They don’t want to get the 5 million comments,
so they are not going to be particularly helpful in setting up a system that accom-
modates getting the public comments. Unless the orientation of the people doing
the various portals and all that is really looking for public input, then you are going
to get things set up in a way that will basically frustrate a whole lot of people.
3. The Structure of the FDMS and the Comment Process Itself
There were two predominant lines of discussion about the role of structure in electronic rulemaking.
At the micro level, participants talked about how to structure the comment intake to improve the
ability of people submitting comments to provide usable information and the analysts, in and out of
government, to sort through the comments. At the macro level, there was concern (and some hope)
expressed about the notion that all agencies and constituencies would be adapting to a single, central-
ized structure for electronic rulemaking.
On the issue of comment intake, a central question was: Can it be structured usefully, and,
if so, by whom? If such a structured commenting platform were to be developed, would anyone want
to use it? One person thought a structured form would be more reliable, but cautioned “people can
make mistakes and then can sue and say, the form is confusing, there were too many boxes to check.”
Another person remarked:
You don’t want to have systems to be more complicated and not get you any beneﬁt.
Until we actually see what comes into place, the question is out. I’ve had to submit
comments where you have to give the line number, the page, your comment, your
name, title, and e-mail with each one, because that is going to be chopped up later
on. It is cumbersome, and harder than writing just a uniﬁed letter. If you actually
49 Respondents to the Web survey also identiﬁed value-added features that would cluster and rank frequently viewed
comments or supporting documents.

24
s
e
have to write them in a speciﬁc format, and then paste them up individually in
m
e
different places, that could be cumbersome.
h

T
p
There also was a fear expressed of forms that ask “stupid” questions.
o
h
s
That is to say, the agency asks the questions that it wants answered. But, I am not
r
k
o
ready to defer to the agency as being the arbiter as to what the relevant questions
r

W
are. So, while we frequently answer the agency’s questions, the remaining ¾ of what
j
o
a
we have written have to do with posing what we think are the relevant questions,

M
e
why they are relevant, and why they mean that we are right.
f

t
h
Another person, however, thought just the opposite:

o
r
y
I think there is a better opportunity on the beneﬁts side to get some speciﬁc ques-
a
m
tions answered. In every rulemaking that is done well, they have speciﬁc questions
m
u
S
that they ask. There may be a good way for the system to be set up, so that for
every rule there is almost like a form question: “Are you being regulated?” “Are you
being impacted?” So you could have a system of that Q&A. They could bypass that
system and just submit their letter. But, it would give the agency an opportunity to
get speciﬁc questions answered.
In another session, a participant argued:
… it seems that an opportunity exists to use the technology to encourage, certainly
not force, but encourage better structured, better thought out comments … it’s
helpful when a person cites the section of the regulation; that frequently and
certainly in my state experience does not happen. But certainly by encouraging the
comment writer to include that information, preferably in some systematic way
that you could tag it or stick it in the database and connect it to that part of the
comment, would perhaps encourage a little bit more discipline on the part of the
commenter in thinking about what sections it is they are commenting on, perhaps
encourage them to structure their comments around speciﬁc areas in the regulation;
speciﬁc sections. That could be a very signiﬁcant beneﬁt right there in making good
use of tags.
One participant warned about the apparent limited thinking about the process itself, and
the excessive focus on technology. It was stated repeatedly that the needs of the rulemaking process,
and not the availability of the technology, should be the driver of change. Another remarked that the
people in and out of government who understand the business process and the nature of the available
IT are rare, but critical for transformational activities to take place.
The development to date of a centralized FDMS was criticized by some participants
and praised by others. One speaker noted, “it is very, very hard on day one to say this is what the
system should look like and have something which is really right.” The different agency cultures
and constituencies were pictured as a barrier to effective centralization that in the worst case might
thwart innovation at the agency level and foist an unwanted system on thousands of rule analysts and
millions of commenting citizens.50 Another participant noted:
50 There has been evidence of agency resistance to cross-agency standardization for some time. See the GAO letter in
response to Congressman Henry Waxman and Senator Joseph Lieberman on the subject “Federal Rulemaking:
Agencies’ Use of Information Technology to Facilitate Public Participation” GAO/GGD-00-135R available at:
www.gao.gov/archive/2000/gg00135r.pdf. More recent signs have emerged in the House appropriations process. Some
individual agencies balk at the idea of joining a centralized docket management system. See Jason Miller. 2004. “House
Plans E-Gov Cuts,” Government Computing News (June 28).

25
… an important consideration as you move to federal government-wide eRulemak-
s
e
ing, is to somehow have this process for engaging not only career employees but
m
e
h
also political appointees in what the objectives and goals of eRulemaking are, other

T
p
than to put a system in place. There were a number of persons with whom I’ve
o
h
s
worked whose emphasis and bottom line with eRulemaking was “Get the system
r
k
o
up.” It wasn’t a matter of “Do we have kinds of participation in the process that
we need?” So, I don’t have a solution for that. I think it is an issue that has to be
r

W
j
o
continually impressed upon, not only the groups themselves that may not have
a

M
effective participation now, but also on the managers, political and career, in a
e
number of the agencies.
f

t
h

o
r
y
a
m
4. Legitimacy Gains and Losses
m
u
S
On the issue of increasing rulemaking legitimacy, participants remarked that hopes were high, but the
ultimate impact was uncertain. “The risk,” stated one participant, “is that we develop an eRulemaking
process, and then the agencies and departments say, ‘We have democracy now, because anyone can
get on the Internet and submit comments.’” The beneﬁts of enhanced legitimacy, one participant
noted, were difﬁcult to gauge due to the number of intangible factors and the lack of a reliable metric.
One speaker identiﬁed less controversial rulemaking outcomes as a pathway to rules that are more
likely to be implemented effectively and observed. Another stated categorically that enforceable rules
are not by deﬁnition good, because the ability to overturn a bad rule is fundamental to the mission of
their organization.
Most of the participants seemed to agree that to the extent the transparency of the process,
data, models, and workﬂow in rulemaking increased, so, too, would procedural and substantive
legitimacy. Hyperlinked navigation to the statutory authority for a rule, or OMB guidance on rule-
making itself, also was mentioned as innovation likely to enhance the legitimacy of the process. One
participant noted electronic rulemaking would result in more supportive comments and with a visible
rationale, thereby enhancing agency legitimacy. Better access to a more complete set of documents
might translate into fewer Freedom of Information Act (FOIA) requests. Conversely, increased access
to FOIA procedures could produce a spike in requests for agency documentation. Either way, the
hoped-for effect is that rulemaking legitimacy may increase as the public comes to better understand
and appreciate the regulatory process and their rights and role in it.
A dissenting view, however, argued that super-transparency, revealing perhaps long-obscured
aspects of agency discretion in an unﬂattering light, could inadvertently harm the legitimacy of the
process. Awareness of a shift from black box to glass box in sensitive aspects of rulemaking might
compel rule-writers and regulatory managers to move substantive decision-making and deliberations
into forums that in effect ﬂy below the information highway’s Super-Doppler version of the transpar-
ency radar. One participant noted legitimacy derives from serving the public, not the government’s
own interests.
It was suggested that electronic rulemaking could enable more open peer review of scientiﬁc
studies. Some participants recoiled at the suggestion that an e-commerce-style rating system might
allow the public to vote on the validity of a particular study. Nonetheless, most participants seemed
to agree that getting models used in the process into the public domain, available for public comment
and subject to a rebuttal, would help to “de-bug” error-prone assumptions. A particular problem
mentioned more than once was the reliance on proprietary models that are not transparent. For some
participants, increasing the transparency level would make the agencies more vulnerable to techni-

26
s
e
cal and procedural challenges. One person remarked that “retroactive re-analysis of the costs and
m
e
beneﬁts and the impact of regulations would be very beneﬁcial.”
h

T
p
Several participants remarked that legitimacy might be enhanced if new tools rendered the
o
h
s
assumptions involved in cost-beneﬁt analysis more amenable to inspection and analysis by experts
r
k
and lay persons alike. This innovation also might expose any failures to understand fully the nature of
o
stakeholder needs. Whether the stakeholder is a small business owner or an organic farmers’ associa-
r

W
j
o
tion, the ability to see, submit, and comment on the use of cost-beneﬁt data is likely to improve the
a

M
opportunity to shape the outcome of rulemaking.
e
f

t
h
Some participants worried that human language technologies might be used by agen-

o
cies “automatically” to discount form letters. One participant voiced concern about “this kind of
r
y
a
automated taking of language to see what people are concerned about without actually reading the
m
m
comments. That is how the agency is going to see the results of what they are getting from what is
u
S
being submitted, as opposed to what the person is actually presenting, and how they are presenting
it.” Another participant thought it “would be really helpful if people were able to review how the
agency is using the technology to analyze things.”
It was noted that current accepted administrative procedure demands that duplicative
comments be treated as a single substantive comment and that technology simply would render it a
less burdensome and more accurate process. One person remarked that recent regulatory reforms
and Executive Orders put rulemaking on a trajectory to diminish the signiﬁcance of all non-expert
public commentary, regardless of the media used to deliver it. For some, the implementation of the
Data Quality Act and OMB peer review guidelines represent an overt threat to all public comment;
to others, there are clear and fair options to eliminate non-substantive input from consideration in
the decision process. Several participants linked concerns about the aims of regulatory reform to the
expanding power of information technologies.
One participant claimed the ability to point to and view one million public comments,
duplicative or not, instills a sense of legitimacy about the transparency of the process. Another noted
that some courts will occasionally cite the agency’s tally of the yeas and nays in their decision. There
was general agreement that transforming mass e-mail campaigns into distributed data collection
enterprises had potential to remake the model of e-advocacy and public participation in rulemaking.51
Nonetheless, advocacy groups are likely to cling to the mass e-mail campaign as long as there is
potential political, legal, or organizational utility in doing so. One person noted:
There are positive potentials in accountability and transparency, in having com-
ments visible. It is one thing to have a million comments on mercury submitted; it
is another thing to be able to point to them. To have your Web site of your NGO
point to those million comments and then a link to the e-mail of your representa-
tive to ask what happened in regards to those comments. Those million comments
may be identical and not have much weight with the agency, but I think we increase
the democracy and the accountability by having them accessible for that political
use. To me, that is a positive.
51 In the case of the January 15, 2003 Advanced Notice of Proposed Rulemaking (ANPR) the EPA issued related to issues
associated with the scope of waters that are subject to the Clean Water Act in light of the U.S. Supreme Court decision in
Solid Waste Agency of Northern Cook County (SWANCC) v. U.S. Army Corps of Engineers, the total volume of public
comment (over 130,000 individual comments) was discussed in a White House West Wing meeting on whether to go
ahead with the rulemaking (Source: EPA focus group, July 26, 2004). A decision was ultimately made not to proceed with
a new rulemaking.

27
A related discussion concerned uncertainty about how different comments, submitted
s
e
via different media, would be valued.52 Whereas some fear sorting technologies will cause paper
m
e
h
(especially handwritten comments) to lose relevance versus electronic comments, others predict a

T
p
diminished fate for the e-comment and the continued supremacy of the paper comment, particularly
o
h
s
on organizational letterhead. One participant stated you “might have 30 handwritten pages, and
r
k
o
they are sometimes called rants, because they really do not have a coherent presentation. So, while
the courts are not turned upside down by rants, they handle them differently than they do a skilled
r

W
j
o
analysis of an issue.”
a

M
e
One person noted it was not so important to be concerned about how your medium of
f

t
h
choice is viewed. Rather, what matters is the ability to see how your input is used. The generation of

o
an automatic acknowledgement of a received comment and a link to it in the docket were considered
r
y
a
easy places to realize legitimacy gains. Another participant, however, spoke of concerns that federal
m
m
agencies will foster and avail themselves of a citizen management industry—something more like an
u
S
automated public relations ﬁrm designed to “manage people’s anger and frustration” rather than an
administrative law enterprise.
5. Issues Related to the Law and Litigation
There was strong sentiment expressed by some that radical change indeed is underway at the conﬂu-
ence of IT, presidential Executive Orders, and the regulatory reform efforts in Congress over the
past two decades. There was consensus that the regulated community is better informed and more
strategic than ever before. The proliferation of litigation opportunities, whether to delay or derail
unpopular rules, has increased the perception, rightly or wrongly, that there is a general up-tick in
interest group opportunism. One participant worried agencies “may have algorithms that statistically
work very, very well, but if a company with plenty of lawyers submits something and their submission
is misinterpreted than the agencies could get sued.” Another participant predicted we will see “groups
who are opposed to rules using the IT accessibility, the information and the comments on the
comments and so forth, as a mechanism to litigate wherever they see an opportunity to beneﬁt their
posture or position.”
The optimists see IT used as a mechanism for enhanced early participation by stakehold-
ers to stave off time- and resource-consuming litigation. Especially in an era of declining agency
resources and expanding responsibilities, some participants thought the use of IT in this manner
might free up agency resources. Others were less sanguine. One person remarked that “many people
do not ﬁle comments because they are attempting to inﬂuence the regulation, or because they think
we are going to reach some kind of community consensus and sing Kumbayah. Many people are
ﬁling comments primarily for the purpose of preparing a record for litigation.”
Whereas the agency personnel seem to view legal challenges as roadblocks, the interest
group representatives were more likely to characterize them as needed checks against arbitrary
administrative powers. When asked if the rate of litigation in the era of electronic rulemaking was
a proxy for better or worse rulemaking, many agreed subject matter, level of political involvement,
and the supporting regulatory analyses (ex., cost-beneﬁt, risk, and environmental analyses), not the
availability of IT, determine whether there will be litigation. Whereas IT can help members of a
52 Both the FDMS and EPA’s EDOCKET have been designed to accept electronic renditions of paper comments.

28
s
e
group shape, understand, and comply with a rule, it equally well provides the tools to undermine an
m
e
undesirable outcome.
h

T
p
o
h
s
6. Collaboration: Opportunities and Barriers
r
k
o
Participants generally were optimistic about the impact of IT on collaborative elements of the
r

W
process, although one dissenter noted collaboration was an “overblown” goal. Another stated that
j
o
a
interagency meetings help to sort out unresolved issues and spread best practices by talking about

M
e
how different agencies experience the same process. The possible need for a federal advisory com-
f

t
h
mittee to formalize the collaborative development of the FDMS was raised. A concern was expressed

o
r
y
that, although some interagency agreement on data collection and other business process questions
a
m
was possible, the tendency for agencies to covet their own way of doing business was not necessarily
m
u
going to be reduced by collaboration on the development of the FDMS. The complexity is manage-
S
able, noted one participant, but the minutiae represented a considerable burden.
One person noted the way their NGO used IT to identify and coordinate with allies in a
rulemaking. Another thought the new technology might “socialize” government agencies in a manner
that made collaboration within agencies and with the public more meaningful.
I think one of the promises of online rulemaking is socialization—that people outside
government and people inside government can actually work together on these
things. And, I don’t think it is totally pie in the sky. My own experience with this is
that it is so unusual for a rule-writer to see anybody but a lobbyist. If there is a way
to use the technology, to make personal contacts with people who are signiﬁcantly
impacted by the rules, I think it is possible to get rules that are serving the public
better as well as make life a little less miserable for the people in government. I think
socialization is a burden for people outside government and people inside govern-
ment. It is a burden of interagency collaboration, which is a difﬁcult topic.
It was suggested that electronic rulemaking also will ease, and thereby increase the level of,
state and local government interactions with federal rulemaking. “There is a lot of interaction already
between local and state jurisdictions,” noted one participant, “it wouldn’t be too much of a jump to
bring them in a more systematic way into the eRulemaking initiative so that they can let folks know
that this is happening at the federal level.”
Following on the idea that mass mail campaigns might be converted to more useful data
gathering exercises, one participant noted it would take substantial, possibly IT-enabled, collabora-
tion between agencies and stakeholders to make this happen. One participant stated that some form
letters are the product of signiﬁcant collaboration, between, for example, the experts in the various
divisions of an NGO or advocacy coalition and the supporting membership. In such cases, some
contextual knowledge about the source of the form letter and the collaborative process by which it
was derived would be needed to realize the extent to which the “form letter” is in fact the end-
product of a lengthy, deliberative process. One participant referred to a mass mail campaign as a
“constituency” whose “signatures on form letters are actually endorsements of that entire set of
priorities” rather than an insubstantial rant based on ideology or a summer project of an isolated
intern. No consensus emerged about the role of value-based outrage expressed via public comments
in rulemaking, however, the right to participate in whatever form one chooses, effectively or not, was
repeatedly afﬁrmed by various workshop participants.

29
7. Innovation
s
e
m
There was wide agreement about the nature of IT as a driver of innovation. One participant noted
e
h
that IT speeds the process up, creating more pressure on those who prepare comments, and leaving

T
p
o
those who prepare public comments with the sense of rising expectations about their work. One
h
s
participant explained: “When you are submitting electronically, you could be working on changing
r
k
o
the comments on a 3:00 conference call or an electronic bulletin board discussion for submittal by
r

W
5 p.m. that same day.”
j
o
a
As with other elements of the discussion, the participants did not agree whether the innova-

M
e
tion would help or hurt the process. One concern was that the FDMS would be slow to respond to
f

t
h
the needs of its users. There were discussions about how best to ensure ﬂexibility, agility, privacy,

o
r
y
security, and reliability. Participants noted that rigid architectural choices could produce stagnation
a
m
in the evolution of electronic rulemaking. One predicted a ﬁght for control of the structure of the
m
u
FDMS, making it a site for battles over the hard-wired rules for public participation in rulemaking.
S
Discussions of procedural innovations facilitated by IT focused on the ability to see and
comment on other comments or documents in the docket, as well as the possibility of making rebuttal
comment periods (which currently are rare) a standard part of the rulemaking process. While IT
could greatly facilitate such a change, it would take Congress or an Executive Order to get all the
agencies on board for such a major innovation. The rebuttal comments on the comment process,
most agreed, would address the gaming of the system by those participants who ﬁle comments at the
last minute to avoid being subjected to a rebuttal comment.
Some participants were ambivalent about the beneﬁts of a point-counterpoint dialogue. A
proponent stated that a visible and open debate would sharpen both sides, and another saw potential
for more scientists pointing out ﬂaws in assumptions. One person questioned whether such a dialogue
would feature statements such as “there’s no science” by people who have never read any scientiﬁc
studies themselves. There were several discussions of the use of rebuttal periods as a way to get
in-depth responses to speciﬁc assertions. One person called the ability to see and respond to the
comments of others a source of checks and balances in the system. Another predicted “electronic
comments and replies to those comments will become more of a discussion or dialogue online than
traditional rulemaking has been … agencies may want to encourage that, because it allows alternative
views to be laid out and debated. It takes the agency off the hook to some extent.”
There were a number of remarks about the way that online tools might transform the nature
of advocacy, activism, and the analytical process itself. Perhaps the most salient advocacy transforma-
tion was the idea that the increasing trend toward mass campaigns of identical (or nearly identical)
public comments might be replaced over time with more sophisticated methods for gathering and
synthesizing data, or novel ideas, that are applicable to the decision-making process under the tradi-
tions, laws, and rules of administrative practice. The hope expressed by some was for a steady move
away from duplicative form letters toward more interactive electronic forms and forums that cull
actual data and truly innovative ideas from stakeholders.53 “I’m hoping the eRulemaking will allow
53 USDOT ofﬁcials have explained in other interviews that an unmistakable beneﬁt derives from public comments that call
attention to overlooked data, studies, or novel concepts. In the case of the Corporate Average Fuel Economy (CAFE)
standards, a well-thought-out public comment resulted in the addition of a missing beneﬁt in the critically important cost-
beneﬁt analysis. This is perhaps suggestive of a future direction for e-advocacy. Instead of 60,000 form letters, a campaign
might use IT to foster a massive brainstorming session that raises, debates, and distills the best ideas a constituency can
offer collectively.

30
s
e
advocacy groups to become more sophisticated,” stated an agency ofﬁcial, “because if we don’t get
m
e
information from advocacy groups that is speciﬁc, we can’t do anything.”
h

T
p
The dissenting view, generally held by NGO representatives, was that these mass mailing
o
h
s
campaigns have a meaningful role that is both educational and political, even if 500,000 nearly identi-
r
k
cal, opinion-based, vote-like submissions count only once in the decision process. Many participants
o
had a hard time imagining that the campaigns would go away or that form letter submitters would
r

W
j
o
ﬁnd the time for more demanding forms of engagement. Others insisted the preservation of this form
a

M
of comment was critical to ensuring democratic legitimacy, even if the comments ultimately had no
e
bearing on the decision.
f

t
h

o
The perception of a tool-driven transformation of activism was two-fold. First, some partici-
r
y
a
pants viewed the new technologies as enablers of more litigation that would increase the vulnerability
m
m
of agencies to legal challenges from “losers” in the most vigorously contested rulemakings. For those
u
S
activists who thrive on playing the game of defeating, delaying, or substantially altering undesirable
rules, some said the new IT would be a boon. The fear of electronic monkey-wrench activities was
palpable for the agency participants, while some nongovernmental speakers saw it instead as the
check and balance of last resort when politics or some other factor precludes winning on the pre-
sumed merits.
The second activist transformation suggested was that electronic rulemaking would either
expose, or else proliferate, “ﬂawed” models that sometimes are the basis for a rulemaking decision.
Better access to industry, academic, or government data would increase the ability to replicate and
test models. Others noted the same tools might just as well explain and demonstrate the legitimate
models on which many expert decisions rest. Many seemed to agree that user-friendly, non-expert
visualization and manipulation applications (akin to UrbanSim54), would make the operative assump-
tions and rationale more transparent. One person speculated:
You can imagine real strange things in the future. For example, someone might
make a game that reﬂects that person’s worldview and if you’d play the game you
might experience that person’s worldview. I think of things like SimCity, where it
embodies certain assumptions about how urban planning works and how people live
and things like that. And if you play that game you experience certain consequences
of your decisions and it gives you more tangible experience of that theory of the
world. I could imagine people saying, if you want to see how we think ecological
systems work, just experiment with this game for a little while and then you’ll see
how this model actually plays out.
In terms of the transformation of the analytical process, there were concerns that new
technologies invariably will bias the process. One participant thought de-contextualizing comments
using advanced human language technologies would diminish their signiﬁcance. Another asked
whether the tools themselves would be transparent. If agencies are sorting and categorizing public
comments using advanced technologies, participants wondered about the extent to which that fact
and the technical details behind it need to be made public. Others questioned whether the expertise
in agencies might shift from knowledge of rules and data to mastery of large-scale language analysis
tasks. The dissent here was to note that good tools would take an already-bogged-down process and
54 See www.urbansim.org.

31
shift the analyst workload from sorting similar and identical documents to actually doing analysis on
s
e
all the unique and valuable submissions.55
m
e
h

T
p
o
8. Educational Impact
h
s
r
k
Many participants spoke positively about the educational potential for electronic rulemaking. The
o
hope was expressed that IT-enhanced rulemaking would contribute to making better citizens and
r

W
j
o
comments, while increasing compliance with and understanding of federal rules. Citizens would
a

M
beneﬁt not only from the ubiquity of accessible information, but also the ability to personalize
e
government and interest group information services. While some predicted better education for
f

t
h
traditionally underrepresented groups, dissenters warned that no amount of online information or

o
r
y
tools will solve education issues for the many digitally and otherwise illiterate citizens who dispropor-
a
m
tionately make up underrepresented groups.
m
u
S
One thread of discussion focused on the proper role of government as an educator of its
citizens. One person remarked it was insufﬁcient simply to post information on the Web, since the
rulemaking process and its imperatives were unfamiliar to many citizens. It was suggested that the
FDMS could have a prominent link to a “This is Rulemaking” page, a tutorial that explained the
notice and comment process in straightforward terms. This page could explain what constitutes a
good public comment, and that page could link to agency-speciﬁc deﬁnitions of the type of informa-
tion that is credible to rule-writers (e.g., scientiﬁc studies, reports, data, replicable models, etc.).
A recurrent theme expressed by agency personnel in these and other focus groups stressed
the need for those facilitating and participating in mass mail campaigns to understand better the
limited impact of such efforts. Agency personnel have stated consistently that whether it is 50 or
500,000 identical or similar comments, the value added to the rulemaking process is considerably less
than one careful, speciﬁc, substantive comment. One participant, however, stated that for many of the
one-click submitters participating in a rulemaking, the sending of a form letter as part of an e-mail
campaign is often a ﬁrst step into the political process, one that can be a gateway to more signiﬁcant
engagement and participation.
Others expressed interest in having the government use the Web to get an explanation of
the agency mandate into the hands of the public. It was noted that most citizens remain ill-informed
not only about the rulemaking process and its statutory authority, but also the rules themselves, once
they are part of the Code of Federal Regulations. One participant was concerned that government
use of IT to produce more informed comments or better-informed citizens might exceed the proper
scope of its role and generate a whole new set of issues. There was general agreement, however, that
the government bore some responsibility to try and reach the affected stakeholders in a rulemaking,
regardless of whether they spoke English. To that end, many agreed that a Federal Register notice
was insufﬁcient and that IT presented a new toolbox for educational outreach.
It was noted that there exists a possibility for traditional intermediaries (interest groups) to
educate their membership more carefully, with the goal of generating more detailed, high-impact
public comments. One participant stated that thousands of associations count educating their
members among their top priorities. In return, many of those groups rely on having their members
55 In truth, most agencies farm out to the private sector or special internal branches of government, such as the USFS
Content Analysis Team, the task of sorting and doing content analysis on the large batches of public comments. New
tools for accessing and analyzing the docket are already in use. An interesting question is whether the public is aware of
the extent to which contractors are involved in the process.

32
s
e
go through their system for preparing and submitting comments on rules. The ability to harvest
m
e
membership data via these systems in turn educates the groups about the needs and composition of
h

T
their constituency. For one participant, this data became the basis for more carefully targeted action
p
o
h
alerts and other campaigns. Another person suggested that there ought to be annual conferences on
s
r
k
the “ABCs” of rulemaking, which might ensure that the groups’ leaders and mobilization specialists
o
themselves better understand the function of public comment in agency decision-making.
r

W
j
o
a

M
e
9. Level and Nature of Public Participation: Quantity vs. Quality
f

t
h
Many of the groups wrestled with the sense that increased public participation might degrade the

o
r
y
overall quality and efﬁcacy of public comment. The implicit assumption seemed to be that a quan-
a
m
tity/quality tradeoff exists. According to this view, the increasing number of comments necessarily
m
u
means more duplicative, non-substantive form letters and possibly less attention to those comments
S
that might contribute meaningfully to a better rule. One participant commented that “the beneﬁts
are engaging more people in that process … and, I think that is possibly one of the costs. You create
a process whereby it really does become sort of no more value than people sending mass e-mails or
spam … you need to design the process so that people actually feel they do have a voice and role in all
of this, and that their voice is being heard.”
An “explosion” of anecdotal comments also was thought, by some, to be a net loss for the
overall quality of the comment process. One person saw the move to rhetoric-based comments as
unlikely to solve any of the problems rule-writers face. Another remarked that some of the Web
applications currently running are automated dumps without any practical meaning. Yet another was
worried about the ossiﬁcation of the process as the volume of comments increases:
I fear eRulemaking will bring us into what now happens with C-Span, where
you get the “crazies” on the phone calling in. I think you are going to have the
potential for some “crazies” to overwhelm the process, whereas, in a more formal
rulemaking, they have to be willing to show up to a hearing and willing to get a
paper document to you. We’ve had some rulemakings where people have been so
vehement that they have threatened bombing us if a rule went through. We’ve had
to have federal marshals at our hearings. Now, they will have anonymity, and I see
the potential for that to bog down the rulemaking process.
Some dissenters noted that this view of a clear trade-off misses the fact that amongst the
ﬂood of form letters there also might be signiﬁcantly larger numbers of useful comments. Consider
this remark:
I represent 8,000 people who mostly are in academics. We think electronic rule-
making has made life quite a bit easier. As a society, we are able to weigh in more
and more often. We can provide public comment on different proposed acts. Our
members are reluctant, because they are scientists, but I think it is easier for them,
we are seeing more participation from them too. That is a good thing.
Another participant noted a trade-off for groups that might take on the task of synthesizing
all their members’ input into one coherent, high-impact comment, or could instead seek a newswor-
thy large count of forms and what the U.S. Forest Service Content Analysis Team calls “form-plus”

33
comments.56 One person noted any turn to technology that limits or marginalizes participation would
s
e
constitute a democratic loss. Conversely, any technology that ensures every comment submitted
m
e
h
gets considered, even in a torrent produced by well-organized campaigns, would increase the public

T
p
legitimacy and legal defensibility of the process.57
o
h
s
There was some debate about what actually counts as participation. Most seemed to agree
r
k
o
that the submitter who never reads the proposed rule or even the preamble before clicking on an
r

W
e-mail or Web site link to comment barely passes the lowest possible threshold. When that person
j
o
a
goes to an automatically populated Web action form (the two-click, cost-free comment), the tempta-

M
e
tion for some is to dismiss this act for failing to clear the threshold of meaningful participation. While
f

t
h
some participants identiﬁed this as a less than fully legitimate form of engagement in rulemaking,

o
others saw it as a proper compliment to more thorough, costly comments prepared by organized
r
y
a
interests. One person stated the EDOCKET system was designed to limit participation until it was
m
m
too late to make a difference. Another suggested the focus ought to be on the entire process, and not
u
S
simply improving notice and comment.
Some were concerned that the outgrowth of these actions would be the full and formal
disenfranchisement of mass submissions.58 Another spoke to the fear that the people are being written
out of the process. One person remarked that the goal is not to ensure everyone can participate in
every rule; rather, it is to get real stakeholders interested in rules that impact them, and to do so early
in the process. Many agreed with the notion that letter counts were inferior by deﬁnition to solid data
and facts, yet one person recognized there is a “legitimate tension” between anecdotes grounded in
values and facts derived via the scientiﬁc method.
10. Overall Costs and Beneﬁts
The groups highlighted the many unknowns with regard to the costs and beneﬁts of electronic
rulemaking. Will the FDMS work? Will anyone use it? Will it reduce or increase the costs and
burden hours devoted to sorting through information, or the comments on comments? Will it lead
to quicker promulgation of rules, and should it? How much will it actually cost? Will it facilitate the
proliferation of “me, too” votes, or bring the wisdom of more diverse opinion to bear? As was the
case for so many of the issues raised in this report, there was no overarching consensus, except to say,
as one person did, that “we just don’t know yet.”
It was suggested that cost savings ought to be less of a factor if the money spent resulted in
much better information services. The reduction of manual labor and content analysis subcontract-
ing was identiﬁed as an avenue of real savings. One nongovernmental participant, however, thought
it would be a signiﬁcant cost to monitor discussion threads and rebuttals to be able to get the “last
word” in. Another spoke of the privacy cost if companies used the new tools to gather data on citi-
zens. A consistent concern was expressed about the misuse of tools designed to enhance the process.
One person worried: “For us government analysts, the thing I think of as a cost is how many people
56 Dr. Jamie Callan has developed a rudimentary near-duplicate detection tool that, if incorporated into the FDMS, will
simplify and improve the accuracy of efforts to cull the distinctive additions to form letters that count as “form-plus” in
the USFS lexicon. See the recent example at: http://snipurl.com/858i. Since the Administrative Procedure Act requires
that the agency address every substantive comment, this increased ability to highlight and review near-duplicates may
lessen the likelihood that meaningful additions to form letters are sorted out of consideration by accident.
57 In other venues, agency personnel have expressed concern about losing an otherwise carefully prepared rule in the courts
based on failure to address substantive comments that are lost amongst thousands of modiﬁed form letters.
58 Depending on who you ask, the USFS either ﬂoated or had a free-wheeling staff member let slip the idea that electronic
mass submission could be eliminated. See: http://snipurl.com/858y.

34
s
e
you will need and how will we sort through all that information. I have visions of people commenting
m
e
on the comments on the comments, and that there will be millions of comments. How will we get all
h

T
this done?”
p
o
h
s
The reduction of paper and potential ease of access were noted regularly. An ability to
r
k
produce and share a better legal record was considered a plus. One person stated that the focus on
o
E-Government afforded too little attention to dealing with the digital divide.59 There was a potential
r

W
j
o
beneﬁt noted in tracking visitors to the site and how they use the materials and tools that were made
a

M
available. Electronic rulemaking, remarked one person, would allow the agencies to communicate
e
better with and understand the regulated community. To fulﬁll this promise, however, the agencies
f

t
h

o
would need to commit to substantial outreach to stakeholders and the general public.
r
y
a
Finally, it was suggested that there is a need to go well beyond “notice and comment”
m
m
rulemaking, and that the Administrative Procedure Act of 1946 was “out of date.” One person stated:
u
S
I would suggest that the model of the regulatory process that we have right now,
which is the Administrative Procedure Act, was promulgated in the 1940s, a very
different world. The agency would publish something in the Federal Register,
people would ﬁle comments, and they’d issue a ﬁnal rule. The level of complexity
of the issues that we are dealing with now is so far beyond what they were dealing
with then. We are talking about genetically modiﬁed foods and other organisms,
pharmaceuticals that do things that people couldn’t even conceive of, information
technologies, all of these things. We need to really start thinking about the entire
rulemaking process in different way. This could really revolutionize the process.
Over the last few years, you’ve had Executive Order 12866, the Regulatory
Flexibility Act, the Data Quality Act, the Peer Review Act, the Negotiated
Rulemaking Act, all these other things that are being piled on to APA. Out of this
we are going to have a radically different regulatory process in 20 years. I don’t
think we can yet even conceptualize what that might look like.
59 A recent report from the American Political Science Association Task Force on Inequality and American Democracy
stated “the Internet may ‘activate the active’ and widen disparities between participants and the politically disengaged by
making it easier for the already politically engaged to gain political information.” See “American Democracy in an Age of
Rising Inequality” at http://snipurl.com/7egy.

35
n
i
o
s
l
u
c
n
o

C
Conclusion
Much of what has been written and said about electronic rulemaking over the last three years has
been forward-looking and highly speculative. The dearth of empirical studies and the debate over
metrics were characteristic of rulemaking scholarship prior to the emergence of electronic govern-
ment.60 We continue to sort out exactly what to measure and how best to measure it. Simultaneously,
developers of the FDMS are determining what to build and how to fund it. This report represents
another step in the maturation of both processes. While many questions remain unresolved, several
likely scenarios are unfolding. At ﬁrst glance, some of these seem mutually exclusive, but in fact they
are more often complimentary and may be sustainable over time.
Electronic rulemaking may transform the process fundamentally or it may simply digitize
established paper-based processes. Evidence gathered at this series of focus groups supports the latter
conclusion. The pace of procedural and technological change is slow due to considerable inertia
and territoriality among both governmental and nongovernmental actors. Despite idealism among
advocates of a highly engaged digital democracy, people and institutions tend to do what they’ve
always done. If there is a stamp-free method to register a practically effortless preference on a favorite
issue, such as the roadless conservation areas or hours of service for truckers, many citizens will take
the chance to cast what they many times perceive as a vote for a favorable decision, often as deﬁned
and scripted by their interest group of choice.
Moreover, citizens are wise to the fact that both the prevailing architecture of ﬁrst-genera-
tion electronic government and practices in administrative rulemaking impede truly deliberative
opportunities. In addition, many submitters of comments prefer not to have their comments rebutted
and therefore submit on the last day or, with the advent of the electronic mode, even at the last
minute. The anti-deliberative effect of this practice might be mitigated through various technological
or procedural innovations, but how would this then affect citizens’ willingness to commit the extra
time to provide input and consider feedback?
We know very little about how the behavior of affected groups will change when (and if)
duplicate detection algorithms or moderated online dialogues emerge as regular features of elec-
tronic rulemaking. It is safe to surmise that some individuals and groups will ﬁnd the value-added
of advanced information and communications technologies useful to submit more effective public
comments. Others will look for new ways to beat the system and a cat-and-mouse game may move
from the paper-based to the digital playing ﬁeld. One participant stated the “whole mass form mail is
sort of a game like radar detectors. I’ve begun using scripts to take massive lengthy well-documented
comments, carve them up into paragraphs, randomly combine them, and feed them to my users, so
that they can be signed and submitted to get past your ﬁlter. I’m wise to the game. It allows us to then
have that input.”
60 For more on the nature of rulemaking and the trends in its scholarship, see Cornelius M. Kerwin. 2003. Rulemaking: How
Government Agencies Write Law and Make Policy, 3rd Ed. (CQ Press: Washington, D.C.).

36
n
Administrative law scholars worry about a perceived shift away from agency discretion and
i
o
s
expert decisions toward the politics and the psychology of plebiscites. They are not alone. At a recent
l
u
c
n
agency focus group, one participant stressed: “Rulemaking is not a democracy.” While some NGOs
o
C
seem committed to mass mailings, many agency personnel and other observers believe these efforts
aim to undermine the role of expertise in public administration. Mass mailers may inadvertently peti-
tion themselves into obscurity. The use of mass-mailing campaigns during public comments periods,
while occasionally effective, engenders a degree of cynicism and resentment among those intended
to be inﬂuenced by such comments. As a result, the general level of respect of agency personnel for
citizen comments and values may decline over time.
Despite the skepticism linked to mass submissions, electronic rule-making will nonetheless
result in at least some better quality comments and perhaps more durable rules, alongside a prolifera-
tion of quasi-spam in the form of automated junk mail, that one-click cacophony striving for a virtual
direct democracy. Promising opportunities exist for IT to facilitate, structure, and guide informed,
meaningful public input. Rule-writers agree that public comments can be more on point, substantial,
and manageable than they are currently. At stake is whether the guiding or interactive structure
will reside inside or outside the federal government. Focus group participants made it clear that the
best thing developers of the FDMS can do to ensure innovation is to publish an open application
programming interface. The result will be a vibrant marketplace for ideas, data, studies, and other
comments relevant to the rulemaking enterprise.
Two further outcomes are possible. One likely scenario is that IT will increase the transpar-
ency and legitimacy associated with federal rulemaking. Another is that less transparency will result as
agencies fail, for whatever reason, to document the interagency or ex parte communications on which
some critical decisions are based. Consider the current debate over the role of OIRA review and early
participation in decisions about the scope of regulation. The Internet, as potentially a 24-7 spotlight
on agency activities, may result in further blurring the boundary between informal and formal
interagency communications. With formal communications subject to Freedom of Information Act
(FOIA) requests, government personnel may shift their communicative routine to compensate for the
enhanced transparency radar.
It should be obvious by now that the impact of information technology and the Internet is
limited. It cannot solve problems that are legal, political, social, and economic in nature. However,
it can provide more choices and tools than ever before to those with the knowledge of electronic
opportunities to engage the rulemaking process. The playing ﬁeld is shifting and much is unsettled.
Signiﬁcant choices lie ahead for the developers and users of the Federal Docket Management System,
Congress, and the courts. As Lawrence Lessig might say, how you build it matters. Balancing the
interests of regular users and other stakeholders against the imperatives of governing a complex
democratic society is no easy task. To the extent we talk openly about the challenges and opportuni-
ties, the democratic potential for innovative and equitable solutions remains intact.

37

A
i
x
d
n
e
p
p
A
Appendix A
Focus Group Protocol: eRulemaking Workshop 2.0
June 2–4, 2004
{For the purpose of this discussion, we will use IT to denote the wide range of information and communications
technologies that are used in eRulemaking.}
1. What are some of the costs and beneﬁts of applying IT to the rulemaking process?
2. How can information technology improve the rulemaking process?

a. How do you deﬁne “improved” in the context of rulemaking?
3. How do stakeholder groups currently use IT to realize the objectives of their organization?

a. How will this use likely change in light of the tools and capacities discussed during

the presentations?
4. What kinds of cultural changes do you think IT may facilitate or necessitate in stakeholder
organizations?

a. Will IT result in a more collaborative rulemaking process?
5. What speciﬁc things would you suggest to make electronic rulemaking more transparent,
accessible, and effective?
6. Will eRulemaking help “level the playing ﬁeld” between large/small organizations and/or
between urban/rural entities?
7. Would government decision-making be improved if eRulemaking created the opportunity
for a dialogue among stakeholders?

a. If so, how would you structure it?
8. What role should IT play in improving regulatory compliance?
9. Is there anything else we should discuss that we have not addressed?

38

B
i
x
d
n
e
p
p
A
Appendix B
Afﬁliations of Workshop Participants
AFL-CIO
National Campaign for Sustainable Agriculture
Alliance for Healthy Homes
National Environmental Trust
American Association of Law Libraries
National Federation of Independent Businesses
American Petroleum Institute
National Science Foundation
American University
National Security Archive
ASRC Aerospace Corp.
National Small Business Association
Associated Builders and Contractors
Physicians for Social Responsibility
Association of State and Territorial Solid Waste
Public Citizen

Management Ofﬁcials
Small Business Administration, Ofﬁce of
Carnegie Mellon University

Advocacy
Center for Regulatory Effectiveness
Social Security Administration
Committee on Small Business
Specialty Graphic Imaging Association
Congressional Research Service
Transportation Security Administration
Defenders of Wildlife
U.S. Coast Guard
Drake University
U.S. Department of Agriculture
Ecological Society of America
University of Michigan
Electronic Frontier Foundation
University of Nebraska at Omaha
George Washington University
University of Pennsylvania
Government Printing Ofﬁce
University of Pittsburgh
Information Renaissance
U.S. Chamber of Commerce
Insero
U.S. Department of Labor
Iowa State University
U.S. Department of Transportation
National Academy of Public Administration
U.S. Environmental Protection Agency
National Association of Home Builders
Virginia Department of Planning and Budget
National Association of Schools of Public Affairs

and Administration

39

C
i
x
d
n
e
p
p
A
Appendix C
Workshop Session Schedule
Wednesday, June 2
8 a.m.–8:30 a.m.

Registration and Coffee (Room 309)
8:30 a.m.–noon

Session #1 Larger Businesses (Room 309)

Stuart Shulman, Drake University

Jamie Callan, Carnegie Mellon University

Bruce Lundegren, U.S. Chamber of Commerce

Jim Tozzi, Center for Regulatory Effectiveness
noon–1 p.m.

Lunch for all June 2 Participants & Afternoon Registration (Room 310)

Kathryn Newcomer, director of the GWU School of Public Policy and

Public Administration

Lawrence Brandt, program manager, National Science Foundation,

Digital Government Program

Oscar Morales, program manager, Federal eRulemaking Initiative with Q&A
1 p.m.–4:30 p.m.

Session #2 Smaller Businesses (Room 310)

Stuart Shulman, Drake University

Jamie Callan, Carnegie Mellon University

Andrew Langer, National Federation of Independent Businesses
Thursday, June 3
8 a.m.–8:30 a.m.

Registration and Coffee (Amphitheatre, third ﬂoor)
8:30 a.m.–noon

Session #3 Labor & Environmental Advocacy Groups

(Amphitheatre, third ﬂoor)

Stuart Shulman, Drake University

Jamie Callan, Carnegie Mellon University

Tom Mattzie, AFL-CIO

Devin Ceartas, NacreData and the National Campaign for Sustainable Agriculture
continued

40

C
i
x
d
n
e
p
p
A
Appendix C
Workshop Session Schedule continued
noon–1 p.m.

Lunch for all June 3 Participants & Afternoon Registration (Amphitheatre, third ﬂoor)

Dr. Kathryn Newcomer, director of the GWU School of Public Policy and

Public Administration

Kevin Landy, minority counsel, Senate Government Affairs Committee with Q&A

Oscar Morales, program manager, Federal eRulemaking Initiative with Q&A
1 p.m.–4:30 p.m.

Session #4 Good Government and Public Participation Groups

(Amphitheatre, third ﬂoor)

Stuart Shulman, Drake University

Jamie Callan, Carnegie Mellon University

Robert Carlitz & Rosemary Gunn, Information Renaissance

Seth Schoen, Electronic Frontier Foundation
Friday, June 4
8 a.m.–8:30 a.m.

Registration and Coffee (Room 310)
8:30 a.m.–noon

Session #5 State & Local Government (Room 310)

Stuart Shulman, Drake University

Jamie Callan, Carnegie Mellon University

Dr. Christine Reed, National Association of Schools of Public Affairs and Administration

Jay Lagarde, Virginia Department of Planning and Budget
noon–1 p.m.

Lunch for all June 4 participants & Afternoon Registration (Room 310)

Dr. Kathryn Newcomer, director of the GWU School of Public Policy and

Public Administration

Oscar Morales, program manager, Federal eRulemaking Initiative with Q&A
1 p.m.–4:30 p.m.

Session #6 Legal and Lobbyist Professions (Room 310)

Stuart Shulman, Drake University

Jamie Callan, Carnegie Mellon University

Neil Eisner, U.S. Department of Transportation

Peter Strauss, Columbia University

The University of Pittsburgh is an afﬁrmative action, equal opportunity institution. Published in cooperation with the
Department of University Marketing Communications. UMC 45683.

University Center for Social and Urban Research
121 University Place, Suite 600
Pittsburgh, PA 15260
412-624-3776
www.ucsur.pitt.edu