Performing A Literature Review
PERFORMING A LITERATURE REVIEW
Lois E. Reed
Mathematician
Systems Engineering Division
Naval Air Warfare Center Weapons Division
China Lake CA 93555-6100
Abstract – This report is intended as a guide for teachers
complex inter-relationships, including those of the literature
and engineering students when conducting research is part
search and documentation efforts.
of course-work requirements. Discussion includes a
description of a literature search, the purpose of a
literature review, finding sources (especially for
engineering), and a general strategy to help conduct an
efficient and productive literature search. Using tools such
LITERATURE SEARCH
as this report, students can become more pro-active about
their research projects. Teachers can use this report,
among other tools, to begin dialog with their students
about expectations for research assignments.
IDENTIFY
GENERATE
STATE
FIND OR
TOPIC OF
RELATED
UNSOLVED
DEVELOP
Two key steps in a literature search are i) finding
INTEREST
QUESTIONS
PROBLEM
SOLUTION
sources and ii) synthesizing information. Each of these is
addressed in two of the major sections in this report, as
well as how the literature search relates to the entire
research process. Then pertinent information is repeated
in the summary section for your convenience. An annotated
reference list is included for ease in finding other useful
DOCUMENT RESULTS
guidance.
FIGURE 1. Flow Diagram of Research Process.
INTRODUCTION
What Is A Literature Search
Purpose Of A Literature Search
The literature search is a very significant step in the research
An introduction chapter in a research paper generally
process. The basic stages in a typical research project are: i)
includes information gleaned from a thorough literature
identify your topic of interest, ii) perform a literature review,
search. A literature search serves three important functions
iii) generate related questions, iv) state your unsolved
[2]. Reference 2 states that the literature review gives your
problem or hypothesis, v) find or develop a solution, and
reader background information regarding your own research,
vi) document your results. We are often trained to think,
demonstrates your familiarity with research in your field,
approach, and describe tasks sequentially, but anyone who
and shows how your work contributes one more piece in the
has earned an advanced degree in engineering (or any other
puzzle of expanding the knowledge base in your field.
field) can tell you that a sequential checklist of the process is
The important idea to convey is that you really
too simplistic. Reference 1 explains “... how researchers
understand what others in your field have accomplished and
must work at different stages of their project simultaneously
how your work differs from the works of others [3]. In a
[1].” The authors describe how overlapping the stages can
literature review, you demonstrate your understanding of the
help you understand the problem, manage complexity, and
relevant works of others and your ability to summarize this
motivate changes in both earlier and later stages.
information for the convenience of your readers. This sets
The important concept to grasp is that the literature
the stage for you to describe what your research contribution
search is not just one distinct step in a research task. It is
is going to be.
both a step and an iterative feedback loop. Defining an
unsolved problem determines what kind of literature search
FINDING SOURCES
is appropriate, and performing a literature search helps define
an unsolved problem. Because of this complexity, it is not
surprising that many beginning researchers have difficulty.
Libraries are one obvious resource for a student performing a
Figure 1 depicts the research process and some of the
literature search, but there are many others as well,
including the internet. Regardless of the sources you use,
cover topics such as tables, formulas, engineering
keep a bibliographic trail [1]. Track titles, authors,
fundamentals, measures and units of conversion,
publication information, page numbers, and possibly library
mathematics, statistics, and numerical calculations; these
call numbers (LCN), International Standard Book Numbers
sources are especially useful during the writing phase of your
(ISBN), or International Standard Serial Numbers (ISSN).
research [4].
When searching through cyberspace, also note on-line
addresses and other pertinent information so that a search
can be duplicated if necessary. Also be aware that some on-
A STRATEGIC APPROACH
line information changes daily – one day it is there, the next
day it is gone. You should print it out immediately, if
As a beginning researcher, two of the most common
possible, and note the source, the “navigation” route, and
problems you might encounter are: i) not knowing where to
the search engine you used to get to that source.
find sources, and ii) once sources are located, not knowing
These tracking habits can help you avoid duplication of
how to sift or sort through an excess of information to
effort and speed the process of obtaining permission (to use
determine what is useful to you. The following paragraphs
the work of others) when needed. The time required to
elaborate on these two problems and describe a general plan
relocate and reaquire a source is also reduced. You will also
that can help you overcome these difficulties during your
be constructing your reference chapter as you work.
literature review process.
Retracing these efforts by memory is very risky, so it is
better if you take notes at every step.
Getting Started
Kinds Of Sources
A lot of time and frustration can be saved by understanding
the process and the system. “Doing research consists of
Sources are generally described as primary, secondary, or
two basic tasks: brainstorming and researching itself. You
tertiary [1].
Primar y. Primary sources are “materials that
need to know what information you will need, and you need
you are directly writing about, the raw materials of your
to know where to get your hands on it [5].” “Developing
own research [1].”
Secondary . Secondary sources are
the working bibliography requires knowledge of library
“books and articles in which other researchers report the
resources, both printed and electronic, and the use of
results of their research based on (their) primary data or
reference systems to locate sources. During early phases of
sources [1].”
Tertiary . Tertiary sources are “books and
research, developing a tracking method to maintain a
articles based on secondary sources, on the research of others
complete record of all of the bibliographic information from
[1].” Tertiary sources synthesize and explain the work of
your sources is advised, since you do not yet know whether
others and might be useful early in your research, but they
it will be cited in your thesis [6].”
are generally weak support for your own arguments [1].
It is also advisable for you to construct a list of experts
Primary.
Examples of primary sources are data sets,
in your field. Such a list can be invaluable. An oral
computer runs, computer programs, scale models, drawings,
interview with an expert can produce valuable insight and
and engineering notebooks. A well-kept engineering
possibly save you a lot of time and effort. You might try
notebook can provide valuable information for later
sending e-mail messages to well-known authors or lecturers.
documentation of test conditions and assumptions, materials
If you notice that several recent dissertations are written by
used, observations as well as measurements, and unusual
students at a particular university, you might contact the
occurrences that prompted further testing.
authors or the authors’ advisors. Given the opportunity to
Secondary. Examples of secondary sources include
ask, one important question is: “If I were to pick up where
conferences, proceedings, journals, and books. Journal
you (or your student) left off, what advice would you give?”
articles are often the most current source of information on a
topic of study that is new or subject to rapid change. Lists
A Search Plan
of references at the end of each journal article can provide
leads to further sources. Engineering journals are typically
A recommended search strategy is to consider how a topic
field-specific. For a selected list of current journals in
progresses through the research documentation life-cycle.
agricultural, chemical, civil,
computer,
electrical,
One approach is to look for books that are currently accepted
environmental, industrial, and mechanical engineering, ask
reference texts in your research area and find out who has
at your facility or university library for specific guides.
cited them in recent years. Another approach is to look for
Tertiary.
Examples of tertiary sources include
original (seminal) reports, papers, or theses written by
dictionaries, encyclopedias, guides, and handbooks.
known experts and see who has cited them in recent years.
“Dictionaries and encyclopedias are excellent starting points
For these two approaches, Science Citations Index (often
for research. They can provide general background
available on CD-ROM) becomes a very important source.
information to help narrow or broaden the focus of a topic,
Implementing this strategy can help you develop a tree-like
define unfamiliar terms, and offer bibliographies of other
diagram or network showing not only relevant research, but
sources. Some works include an index, which will provide
possibly even how the works of others are interrelated.
excellent access to a subject [4].” Guides and handbooks
By locating and reading at least the abstracts of these
counterproductive to try to read everything in detail. At
papers, you can begin to categorize the available papers by
first, it is more important to categorize and understand what
topic and by the authors’ technical interests in a topic.
sources you have and what might still be missing.
Then look at the reference lists for the newer papers. This
approach is especially useful because some databases might
Summarizing and Note-Taking
categorize differently or use different descriptors as key
words. This approach should at least get you started on a
Both references [7] and [1] have good advice about note-
good literature search.
taking. Reference [7] states that the challenge is to condense
The above advice sounds easier than it really is. Even
others’ work without distorting it. Their seven-step process
experienced researchers often feel overwhelmed by
includes: i) do not write everything down, ii) create your
“information overload.” As technological electronic storage
own shortcuts and shorthand, iii) use numbers for
and retrieval capabilities continue to advance, we will all
numerical terms, iv) leave out vowels when you can, v)
struggle with this problem. That is why it becomes very
record all vital names, dates, and definitions, vi) mark
important to have a plan in mind when conducting a
items that need further examination, and vii) check accuracy
literature review, and to have a conceptual ”blueprint”
before returning or filing the source.
understanding of the resources available. Otherwise, you
Reference [1] recommends writing and summarizing as
can easily become lost in your search or feel as if you are
you find sources and has a three-page section called “Quick
going around in circles.
Tip: Speedy Reading.” It recommends a five-step process:
i) become familiar with the geography of the source,
What To Do If You Have Too Few Sources
ii) locate the point of the argument, iii) identify key
subpoints, iv) identify key themes, and v) skim paragraphs.
If you find little or no information, your topic might be too
The book suggests that not all five steps are needed all the
narrow. If this is the case, it is a good idea to see what
time. The main point is speed and efficiency, and focusing
topic headings or terms any newly found information was
on material relevant to your own research without spending
cataloged under, so that you can use those key words in
time on material that is at best only marginally related.
other database searches. If your search still yields little
While taking notes and preparing condensed summaries
information, get help from library personnel and experts in
of the work of others, you must be ever mindful of the
your field of research. In particular, ask for advice about
requirement to eventually cite all “borrowed” work in your
ways to expand your topic so that searches will produce
final paper. All of the sources mentioned contain
some useful information.
discussions about direct quotations, summarizing, and
plagiarism. Readers and researchers are advised to locate
What To Do If You Have Too Many Sources
and carefully read about these topics from any available
source in order to avoid trouble while writing.
If, on the other hand, you find volumes of information, then
you need a plan to scale your search down to a manageable
amount. Specific aspects of a field of study are often listed
SUMMARY
in annotated bibliographies. Journals specific to a field are
good sources. Experts in your field of interest can suggest
A recommended beginning search plan has been described,
where many of your important sources can be found. It also
especially for engineering researchers. Understanding how
helps to understand and categorize your sources so that you
the literature search dovetails with other steps in the research
know what kind of information you have. Then you can
process helps form a good plan. Guidance has been given
draw on what you need without being overwhelmed by
for finding sources, for determining if your topic is too
material that is interesting and related, but not necessarily
narrow, and for scaling your search to a manageable amount.
critical to your own research.
Useful tips have been given to help you track and
summarize information so that it becomes useful for your
Synthesizing Information
research purposes. Hopefully, this report will help you get a
“jump-start” on a good literature review and contribute to
Once information is located, the next step is to summarize it
the successful conclusion of your research project.
into a coherent literature review section for your document.
You must analyze accurately and critically. It takes a lot of
skimming of books and articles to identify which sources are
ANNOTATED REFERENCES
useful to you and in what context they are useful.
Your goal at this point is to “present an overview of
[1] Booth, Wayne C. and Gregory G. Colomb and Joseph
what your source offers; its topic, research problem,
M. Williams, 1995. The Craft of Research. Chicago: The
resolution, and the outlines of its argument [1].” The
University of Chicago Press. From the Chicago Manual of
abstract, introduction, and conclusion sections of each
Style publishers, this book is a concise, practical guide to
source document should contain the necessary information
mastering the art of research. The guidance helps you
for you to write a good summary paragraph. It can be very
strategize the entire process, from understanding the uses of
research in general to editing and rewriting the final draft.
student. Skills learned during thesis writing easily overlap
Especially useful are the sections on posing questions,
and transition into the workplace setting.
making and supporting claims, and writing meaningful
introductions. Chapters 5 and 6 are especially helpful for
[6] Slade, Carole, 1997. Form and Style, Research
the literature review.
Papers, Reports, Theses, Tenth Edition. Boston: Houghton
Mifflin Co. This work contains comparisons of Chicago,
[2] Weissberg, Robert and Suzanne Buker, 1990. Writing
MLA, and APA documentation systems and is oriented
Up Research, Experimental Research Report Writing for
toward academic writing in general. There are many specific
Students of English. Englewood Cliffs, NJ: Prentice Hall
examples for format and style, as well as discussion about
Regents. This book is especially written for students of
writing research papers in general and presentation of data
English as a second language (ESL). It describes the
and graphics in particular. The book also contains a
various sections of a research report and gives examples of
categorized list of general and specialized indexes useful for
convention, format, and style used in writing research
the literature-search phase of research.
reports. There are practice exercises for students.
[7] Brusaw, Charles T. and Gerald J. Alred and Walter E.
[3] Michaelson, Herbert B., 1982. How to Write and
Oliu, 1993. Handbook of Technical Writing, Fourth
Publish Engineering Papers and Reports. Philadelphia: ISI
Edition. New York: St. Martins Press. This reference book
Press. This author specifically describes how important it
is both a comprehensive and a practical source of
is for you to understand how your work complements the
information on language usage, writing procedures, and
work of others.
technical communication. It is alphabetically organized
with
extensive
indexing,
cross-referencing,
and
[4] Delzell, Barbara, Compiler, 1995. How to do Library
categorization of entries. There are sections at the beginning
Research in Engineering – General,” NMSU Library. Las
of the book that offer a sequential look at the writing
Cruces NM: New Mexico State University Library. This
process.
short instructional report is an excellent guide to the
resources and help available to students at New Mexico
State University. However, your own library or facility
ABOUT THE AUTHOR
should have similar materials, including instructions for
accessing internal and external electronic databases.
Lois E. Reed is a mathematician at the Naval Air Warfare
Center, Weapons Division at China Lake, California since
[5] Markel, Michael H., 1988. Technical Writing
1990. She received B.S. and M.S. degrees from Northern
Essentials. New York: St. Martins Press. Oriented to
Arizona University in 1988 and 1989 respectively, an M.S.
writing effectively on the job, topics include writing roles,
degree from New Mexico State University in 1998, and is
processes, and styles. Chapters also cover elements of
currently a doctoral student in Industrial Engineering at
reports, memos, letters, resumes, and applications. The
NMSU. Ms. Reed’s interests include optimization,
author discusses presentation of graphic data and oral
reliability, Bayesian statistics, large-scale system dynamics,
presentations. Since research and thesis writing follow
and simulation.
similar patterns, this is a useful reference for a graduate
