Drifting Toward A (systematic) Review Of Qualitative Studies ...
The Qualitative Report Volume 9 Number 1 March 2004 95-112
http://www.nova.edu/ssss/QR/QR9-1/jones.pdf
Mission Drift in Qualitative Research, or Moving Toward a
Systematic Review of Qualitative Studies,
Moving Back to a More Systematic Narrative Review
Kip Jones
De Monfort University, Leicester, UK
The paper argues that the systematic review of qualitative research is
best served by reliance upon qualitative methods themselves. A case is
made for strengthening the narrative literature review and using
narrative itself as a method of review. A technique is proposed that
builds upon recent developments in qualitative systematic review by
the use of a narrative inductive method of analysis. The essence of
qualitative work is described. The natural ability for issues of ethnicity
and diversity to be investigated through a qualitative approach is
elaborated. Recent developments in systematic review are delineated,
including the Delphi and Signal and Noise techniques, inclusion of
grey literature, scoping studies and meta-ethnography. A narrative
inductive interpretive method to review qualitative research is
proposed, using reflective teams to analyse documents. Narrative is
suggested as a knowledge-generating method and its underlying
hermeneutic approach is defended as providing validity and
theoretical structure. Finally, qualities that distinguish qualitative
research from more quantitative investigations are delineated.
Starting points for reflecting on qualitative studies and their usefulness
are listed. Key words: Qualitative Systematic Review, Evidence-Based
Policy, Grey Literature, Scoping Studies, Delphi, ‘Signal and Noise’,
Meta-ethnography, Narrative Review, Narrative Method, and
Reflective Teams
Introduction
Qualitative research is no longer the poor stepchild of quantitative inquiries.
Over the past ten years, qualitative research has come into its own, particularly in
terms of wider acceptance in academic and policy communities. At the same time,
evidence-based medicine (or EBM—‘a politically correct term’) has given birth to the
systematic review with all of its complexities and conundrums. Some, however, have
begun to question the systematic review approach and its appropriateness, particularly
in the generation of evidence-based policy (Packwood, 2002, p. 268), and its
usefulness in reviewing qualitative studies continues to be debated. Nonetheless,
because qualitative research is now ignored at peril within the systematic review
camp, such reviews have become the proving ground for qualitative work as well as
the quantitative study. A mistake is often made, however, in transposing methods best
suited to systematic review of quantitative studies into qualitative ones. Check-lists,
‘standards’, matrices, ‘hierarchies of evidence’ and other terminology borrowed from
the arsenal of the quantitative camp pepper qualitative ground like so many cluster
bombs; therein lies the danger of the loss of much of the ground that qualitative
Kip Jones
96
research has won over the past decade or so. It is my belief that this rush to imitate
quantitative procedures is producing a kind of ‘mission drift’ in many qualitative
‘systematic’ research reviews.
‘Systematic’ is, by definition, ‘a system, plan, or organized method’ (Oxford
English Dictionary [OED]), ‘methodical in procedure or plan’ (Merriam-Webster). If
a systematic review is then, at least in part, about method itself, then qualitative
methods include valid principles and procedures with which to progress in reviewing
literature ‘systematically’. A wide range of qualitative ‘methodical’ approaches are,
therefore, available to perform or carry out a review in a particular field of interest—
‘systematically and meticulously’ (OED). The time has come when it may be best to
reacquaint ourselves with the adage that ‘the hallmark of good qualitative
methodology is its flexibility rather than its standardisation’ (Popay, Rogers &
Williams, 1998, p. 346).
It is widely assumed that the ‘systematic review’ is the ‘gold standard’ of the
Evidence Based Policy Movement (Young, Ashby, Boaz, & Grayson, 2002, p. 216).
Systematic review protocol now routinely states that the narrative approach (one of
many qualitative methods) to literature review is passé, dead, simplistic and, above
all, unsystematic. On the other hand, the use of narrative in the social sciences is
nowadays heralded as a fait accompli (Denzin, 2001, p. 23). This paper will make the
case that narrative is the bread and butter of qualitative work, that qualitative research
is always about story reporting and story making and that narrative is a democratising
factor in social science research as it should be in evidence review as well.
Supporting this, Dixon-Woods, Fitzpatrick and Roberts point to the fact that
‘qualitative data take the form of narrative, with themes and concepts as the analytical
device’ (Dixon-Woods et al., 2001, p. 126). By supporting a narrative review that
incorporates some of the more revitalizing methodological procedures emerging from
the systematic review arena, the toolkit of the evidence-based policy movement is
expanded, enhanced and enriched.
A system of narrative review and analysis of qualitative research will be
proposed in this paper that utilises and builds upon some of the more democratising
techniques that are emerging within qualitative systematic review (Delphi and
Nominal group techniques of iterative consensus-building, inclusion of grey literature
in reviews, scoping studies, meta-ethnography and ‘signal and noise’ method—
revised as a qualitative tool) and combines them with a rigorous, inductive narrative
analysis method. The technique involves the use of reflective teams to analyse
qualitative data and is based upon a narrative interpretive method (Jones, 2003),
combined with classic concepts of the utility of case study (Yin, 1989). This
interpretive/analytic procedure includes moving well beyond the widespread tendency
in qualitative research to fragment data by using code and retrieve methods.
By such a return to and emphasis on the very nature of qualitative methods,
the systematic review of qualitative studies gains credence and applicability in the
systematic review arena. In fact, Dixon-Woods et al remind us, ‘Most of the ways in
which qualitative research could be used in systematic reviews are logical extensions
to existing uses of qualitative methods in primary research’ (Dixon-Woods et al.,
2001, p. 128). In the end, research synthesis encompasses a process of theory
development and creation of a holistic interpretation (Kirkevold [1997] cited in
Raholm, Lindholm, & Eriksson, 2002, p. 5), a stronghold of qualitative research.
My own primary research takes a biographic narrative approach to social
health issues, using in-depth, unstructured participant interviews (for example, Jones,
2002c). Through such an approach, I am able to uncover the possibilities of
97
The
Qualitative
Report
March 2004
meanings behind health events at the individual and family levels, thus illuminating
the social contexts of health, ill health and care giving. The method’s reflective team
analyses are conducted within the framework of the psychosocial, but not reducible to
simple psychology; rather, case studies are considered as “complex responses to
events and people in the social world, both past and present” (Hollway & Jefferson,
2000, p. 24). Through this team approach to analysis, the investigator and reflective
teams remain transparent and active participants in the story (re)telling. The stories
(re)created in the analyses echo cultural resources at every level reflected through the
narrator, the interviewer-writer and the analyses team members (and later, the reader).
Of particular interest to me in my current endeavours and my colleagues at the
Centre for Evidence in Ethnicity, Health & Diversity (CEEHD
[http://users.wbs.ac.uk/group/ceehd/home/ceehd_home] —based at Warwick and De
Montfort Universities in the UK), one of nine nodes connected through the UK ESRC
Centre for Evidence Network (http://www.evidencenetwork.org/), is the potential for
qualitative reviews to unearth and report minority experience and viewpoints in
healthcare studies. The philosophical underpinnings of our work, therefore, support
our efforts to first deconstruct, then expand and enrich the concepts of ‘systematic’
and ‘evidence-based review’ as well as the term ‘diversity’. It is our purpose to
champion diversity itself as an approach to systematic and/or evidence-based review
of qualitative studies. Through this process, our aim is to make meaningful the voices
of a wide range of often-overlooked and undervalued minority healthcare consumers
in healthcare policy and practice circles in Britain.
One of the virtues of qualitative research is its inclusionary nature and ability
to give service-users a voice, both through the research process itself (for example,
through a wide range of qualitative social science practices that include participatory
action research, in-depth interviewing, ethnographic studies, visual anthropology,
biographic narrative studies and so forth) and in reports, documents and
presentations. The importance of this kind of research cannot be overemphasised,
particularly when dealing with the disadvantaged and/or the unheard voice. Calasanti
elaborates: ‘Incorporating diversity involves more than content or comparison. … it
also involves theorising about underlying relations. …Merely positing that the
experiences of a racial/ethnic group are “different” places them in the category of
“other”—a special “deviant” case—with the dominant group serving as the often
unacknowledged “norm’’’ (Calasanti, 1996, p. 148).
By use of a variety of qualitative methods, both in research, analysis and
review, the richness of human experience and the voice of the ‘other’ is finally given
a full hearing. For example, the move to a user-led approach to health studies where
participants or their representatives are involved in the design, conduct, analysis and
reporting of research (as in Participatory Action Research) has beneficial effects on
research itself, making it more sensitive to the interests of service-users (Johnson,
2003, p. 13). Qualitative research has the potential to contribute to such paradigm
changes dramatically, particularly in the field of health, healthcare policy and service
provision.
Indeed, we find that the storied nature of health and health information can be
harnessed to benefit community health awareness programmes. The interface of
cultures within our contemporary society includes the interface between specific
population groups and an additional culture, the medico-social establishment and its
subsets. By its nature, therefore, health and social service use is always comprised of
an interface of cultures.
Kip Jones
98
A social model of health care begins by listening to the stories of healthcare
consumers. Such stories attempt to integrate the narrator’s health event within his/her
individual and unique past. A reflective team’s analysis of such a story interweaves
“the psychological with the social and historical, by analysing not just pathology but
also strengths and adaptive capacities, and by studying formative influences not just
in childhood but throughout the life span” (Runyan, 1982, p. 209). A social model of
healthcare is, therefore, necessary in order to begin to comprehend the diversity and
pluralities of understanding around social health events and potential service usage.
By embracing a social model of healthcare with its inherent narrativity, rather than a
rigid and regimented medical model, opportunities can be presented for
understanding of ‘others’ through narrated self-knowledge, reflective practise and
acknowledgement of the shared habitus (Jones, 2002a).
What is the Essence of Qualitative Research?
In qualitative research, the tyranny of numbers is abandoned for the enigma of
words. It is often seen as rooted in a non-tangible domain, fundamentally experiential
and intuitive.
Qualitative work is in constant, dynamic flux, but moving toward some
end-point in an evolutionary way. There are efforts by the mind to
concretise meaning and the qualitative dimension has an integrative
function for the researcher. Unity provides context and meaning and it
is toward such unity that the researcher is striving. Qualitative efforts
make use of that part of the person concerned with meaning, truth,
purpose or reality—the ultimate significance of things. (Hiatt, 1986, p.
737)
Not mere exercises in truth or falsehood, however, these investigations are polyvocal
attempts at interfacing with cultural/relational/linguistic accounts of the real. They
are, therefore, interpretations and not truths in the positivistic sense. The potential of
intuition is ultimately a great advantage to this very process (Scheff, 1997, p. 33-36).
Recent Developments in the Systematic Review of Qualitative Research
The following are but several examples of recent developments (and
expansions upon older ideas) that offer promise for more inclusionary and diverse
ways of observing, organising and classifying qualitative evidence in the social
sciences. Some are chosen because they directly relate to techniques such as
participatory action research (Delphi, Nominal Group, Stakeholder consultation)
while others are singled out for their innovative manipulation of data within a
philosophy of inclusiveness (Signal & Noise) or the interpretive (hermeneutic)
potential of the technique for combining studies that is akin to methods of analysing
primary qualitative data (Meta-ethnography).
The Delphi technique is a consensus-building method of evidence review and
considers evidence from a wider range of study types than is the case in statistical
reviews (Jones & Hunter in Pope & Mays, 2002).
The Delphi process is a way of structuring communication among a
group of people in order to get their opinions, offer feedback, and offer
99
The
Qualitative
Report
March 2004
insights about a course of action. It is not an opinion poll, because it
involves multiple rounds of communication where the results of the first
survey are fed back to the participants, who can then change their minds
(or not) in the next round. Also, the items in a Delphi survey can go
beyond questions about preferences to ask about such issues as the
importance, desirability, feasibility, or potential impact of choosing a
particular course of action. The technique is usually used to obtain
consensus from a group of experts or to assist in forecasting. However,
the group may not reach a consensus, but rather delineate different
courses of action. (Freeman, Strong, Barker, & Haight-Liotta, 1996)
The process, developed by the Rand Corporation, involves assembling panels of
experts (who never physically meet thus supposedly avoiding personality conflicts),
ranking and re-ranking statements submitted to them by questionnaire. Respondents
also are asked to rate the confidence of certainly with which they express opinions.
The ‘Nominal Group’ technique (a physical gathering of participants—unlike
the communication process in Delphi technique by questionnaire), on the other hand,
works more closely with a brain-storming technique, but also includes private ranking
of ideas and tabulation (Pope & Mays, 1996). Pope and Mays caution, ‘Consensus
methods, in particular Delphi, have been described as methods of “last resort”. Even
their advocates have warned against overselling them and suggest that they should be
regarded more as methods for structuring group communication on a question, than as
a means for providing definitive answers’ (Pope & Mays, 1996).
‘Signal and Noise’ technique involves systematic literature review
inclusion/exclusion criteria employing qualitative meta-synthesis and quantitative
meta-analysis. The qualitative meta-synthesis is based on themes, interventions and
results, but without attempting to combine the data into one variable. A ‘signal score’
is used to assess the relevance of publications. What is useful in this technique is the
fact that the process does not eliminate research simply because it is not at a certain
level of evidence or if it has certain methodological weaknesses. ‘There may be some
articles in which the design is suspect (high “noise” level) but the findings appear
important (strong “signal”)’ (Higginson, Finlay, Goodwin, Cook, Hood, Edwards,
Douglas, & Norman, 2002, p. 99). The author’s base signal score on such areas as
study relevance, study implementation and study value to a UK audience, using a
quasi Likert-type scale. In this way, there is more of a reliance on the intuitive input
and knowledge of the investigators. One of the subtleties allowed for using such a
scoring system is in its value to a UK audience, for example. Many reviews will
automatically eliminate US studies, for instance, as irrelevant to a UK study. Using
signal and noise, criteria can be considered on several levels simultaneously. Studies
that ‘automatically’ would be eliminated in other reviews can be included in the
signal and noise system.
Grey
literature and its inclusion in systematic review is the singularly most
important contribution to the democratisation of the evidence-based movement. Grey
literature is non-conventional, fugitive, and sometimes ephemeral but, by its nature,
often more inclusionary than standard, peer-reviewed and commercially published
work. For example, reports published by small issue-centric, voluntary organisations
can often make the case for service-users more directly and convincingly than larger
academic studies of the same issues. Reviews and assessments by government
departments and agencies also contribute to the utility of grey literature. This
literature becomes particularly crucial to investigations such as the ones that we are
Kip Jones
100
conducting at the Centre for Evidence in Ethnicity, Health & Diversity. ‘Social
science publishing is considerably more fragmented than is some scientific
disciplines, where peer-reviewed journal literature is the norm’ (Young et al., 2002, p.
222). Grey literature is comprised of the literature that is not found in peer reviewed
journals and is made up of practitioner journal literature, conference papers, books,
literature from a range of public, private and voluntary sector bodies, and government
publications. A major effort in democratising the evidence-based movement is the
inclusion of grey literature in assessing the state-of-the-art in social science fields of
inquiry. Increasingly, this information is internet-based, confounding the more
traditional search approach of peer reviewed literature. Nonetheless, developments
such as the SIGLE (System for Information on Grey Literature) database, produced
by the European Association for Grey Literature, available on CD through
SilverPlatter Information Ltd. (http://www.ovid.com/site/index.jsp), are becoming
more common.
Scoping Studies are devices by which the ‘scope’ and aim of a proposed study
are investigated. Traditionally, the scoping study uses a mix of literature review
(with a particular purpose of uncovering previous systematic reviews in the field
under study) and stakeholder consultation. In a typical quantitative systematic review,
the preliminary searching (scoping) is usually carried out to validate these initial
questions:
•
What is the size of the candidate literature review?
•
What is the likely quality of the literature (e.g., are RCTs
common or rare?)?
•
What are the specific issues with regard to terminology (i.e.,
definitions or indexing)?
•
Which databases are likely to provide the highest yield of
relevant items?
•
What is the review likely to cost? (Booth & Fry-Smith, 2003)
Booth and Fry-Smith (2003) follow up in this approach with ‘refinement in
discussion with researchers’, but do not mention other consultation, for instance, with
service-users or their representatives.
Stakeholder consultation is a process that typically emanates from
governmental policy consultations (Scottish Executive 2001, ‘Scoping study for a
national survey of Scotland’s minority ethnic populations,’ available at:
http://www.scotland.gov.uk/cru/kd01/red/minethnic-03.asp), but a method that is now
being employed more regularly in researcher-led investigations. In ‘Eliciting the
user’s views of the processes of health care’ (Nicholson, 2000) a call is made for user
consultation, development of consensus, an inclusionary research process and efforts
made to reach and elicit the views of hard-to-reach or marginalized groups.
Nicholson calls for doing this through qualitative methods sensitive to needs and
experiences of these groups and incorporating the marginalized into the mainstream,
rather than treating them as special interest groups (Nicholson, 2000, p. 9).
In the Scottish Executive’s scoping study, emergent issues that impact on the
methodological and practical issues of study design are fore fronted. These are too
numerous to raise here, except to comment that ‘during the consultation process, a
number of research questions were raised which people thought were important in
enabling the understanding of ethnic minority experiences but which they felt were
essentially qualitative issues and would be explored most effectively through focus
101
The
Qualitative
Report
March 2004
group research or in-depth interviews’ (Scottish Executive, 2001). Particular mention
was made, on several occasions throughout the document, of ‘the need for engaging
with the community under study to ensure that research represents the views of
minority ethnic communities and involvement in the research process throughout the
research process’ (Scottish Executive, 2001).
Meta-ethnography was developed as a systematic review process by US
educators, George Noblit and Dwight Hare (Noblit & Hare, 1988). Meta-ethnography
is a comparative textural analysis of field studies, using three ways to order them: in
terms of one another, set against one another or tied to one another. The premise is
based in the assumption that there is always a social and theoretical context in which
substantive findings emerge; the recovery of this context is the aim of meta-
ethnography (Noblit & Hare, 1988, p. 5-6). Meta-ethnography is driven by
interpretation, not analysis, and is seen as an alternative to the positivist paradigm.
Using such tools as key metaphors, analogy, reflexivity and ritual, this inductive and
interpretive form of knowledge synthesis has much in common with anthropology
and sociology. Importantly, an interpretation enables the reader to translate the case
studied into her/his own social understanding (1988, p. 18). It consists of a process of
‘like-with-like’ comparison, in that it translates qualitative studies into one another,
while remembering that the translator is always translating studies into his own world
view (1988, p. 25). Importantly, Noblit and Hare support case study criteria: ‘Unless
there is some substantive reason for an exhaustive search, generalizing from all
studies of a particular setting yields trite conclusions’ (1988, p. 28).
By updating and expanding upon Noblit and Hare’s foundational work, meta-
ethnography is now beginning to gain ground in the field of systematic review in the
UK (Britten, Campbell, Pope, Donovan, Morgan & Pill 2002; Campbell, Pound,
Pope, Britten, Pill, Morgan, & Donovan, 2003). The authors champion the
interpretive framework of meta-ethnography as a technique for combining studies that
is akin to the methods of analysing primary qualitative data (Britten et al., 2002, p.
214). They see the approach ‘as perhaps the best developed method for synthesising
qualitative data, and one which clearly had its origins in the interpretivist paradigm
from which most methods of primary qualitative research evolved’ (Campbell et al.,
2003, p. 673).
The EPPI (Evidence for Policy and Practice Information) Centre in the Social
Science Research Unit at the Institute of Education, University of London, has
developed streams of work both in reviews on health promotion as well as education
(Gough & Elbourne, 2002, p. 229). In contrast to more traditional reviews (such as
those conducted by the Cochrane Collaboration and the Campbell Collaboration), the
EPPI Centre also addresses wider questions using what it calls ‘systematic narrative
reviews’ (Gough & Elbourne, 2002, p. 229). Central to this approach is user
involvement, including involvement of service-users in the setting of the research
question of what needs to be known.
Like Britten et al. (2002), Gough and Elbourne build upon the early
pioneering work of American educationalists, Noblitt and Hare (1988) and their
concept of meta-ethnography. Gough and Elbourne propose a narrative review that
includes defining and developing the discourse to be addressed, a synthesis that needs
to be in the form of new interpretative constructions rather than generalisations, and a
qualitative case study approach where the primary qualitative studies are the case
studies. The authors explain that ‘a meta-ethnography must be driven by the need to
construct adequate interpretive explanations,’ that ‘each interpretive qualitative
research review will be likely to develop its own specific interpretation’ and that the
Kip Jones
102
‘success of each review is the contribution to human discourse rather than the review
itself’ (Gough & Elbourne, 2002, p. 233).
The promise of an ethnographic approach to the systematic review of
qualitative studies is found in its foundational concepts that privilege
alternative ways of thinking, knowing and viewing the world. These concepts
become especially poignant in work where concepts of culture, population,
identity, the study site and researcher stance move to the forefront, as they do
in any consideration of ethnicity and diversity. ‘Traditional postures used by
researchers must be revised to conform to realities of contemporary
technological, global and multicultural, racial and linguistic existence’.
(LeCompte, 2002, p. 283)
Table 1. Examples of Qualitative Review Techniques.
Method
Data Collection
Analysis
Benefits
Delphi
Consensus building
Inductive
Reflective; avoids
personality
conflicts
Nominal
Brainstorming &
Inductive
Physical gathering
ranking
encourages
contributions
Signal & Noise
Documents and
Intuitive; value-
Simultaneous
literature
based
consideration of
criteria
Grey literature
Alternative
Can be analysed
Uncovers under-
documents; web
inductively or
used resources and
based literature, etc.
deductively
voices
Scoping
Literature &
Consultative,
Produces overview
stakeholders
intuitive consensus
in short timeframe
building approach
Meta-ethnography Like-with-like
Comparative,
Alternative to
comparisons; case
textural, interpretive
positivist paradigm;
study method
allows for further
interpretation
A Proposed Way Forward for (Systematic) Narrative Reviews of
Qualitative Studies
A narrative interpretive method of systematic review of qualitative literature is
proposed that both extends and consolidates these developments (see Table 1) in
evidence based reviews. The method builds upon the data gathering qualities of these
techniques (consensus building, brainstorming, anonymity) and the benefits of
inductive analysis (consultative, intuitive, comparative, textural and interpretive).
The narrative method’s foundation is based in the use of analytic induction through
the technique of reflective teams to interpret and respond to qualitative data in a case
study setting (for an elaboration on the reflective team approach, see Jones, 2003).
What is involved in this approach is a kind of synthesis involving a more inductive
approach in which the reviewer(s) may reformulate the focus of the review in the
course of doing it (Hammersley, 2003, p. 4). By employing inductive analysis and
using a ‘reflecting team’ approach to interpretation, the introduction of multiple
103
The
Qualitative
Report
March 2004
voices is facilitated, unsettling and creating a mix of meaning and encouraging
communication and collective means of deliberation (Gergen, 2000, p. 4). What is
sought in using this procedure is an opening up of the possibilities in interpretation,
rather than relying solely upon one researcher’s (or, in fact, one homogeneous team
of researcher’s) interpretation of the data at hand.
This method is envisaged as an approach to the systematic review of
qualitative documents that will counteract the current tendency to ‘vet’ papers for
inclusion in reviews through a checklist of criteria and open up the dialogue on
synthesis to include not only researchers, service providers and policymakers, but
service users as well. In fact, the abilities required of this method’s analysis group
participants are openness and creativity/imagination rather than knowledge of specific
research methods or fields of practice. Diversity of approach to the material
(facilitated greatly by the actual diversity of participants) should be solicited and
encouraged. In using this method in my primary research on informal care (Jones,
2002c), for example, a mix of participant backgrounds and a social health approach
encouraged even the most reluctant academic participants to engage in dialogue on
the level of the personal and the social, thus contributing greatly to the analytical
process.
The team-building process begins by recruiting participants (two, three or
more per team) from varying backgrounds (professionally as well as
demographically) to be immersed in the selected literature, at times ‘line by line’ and
hypothesise at each new revelation of dialogic material. Each bit of material can then
be included or discarded for further hypothesis testing later in the process. What is
sought in using this procedure is an opening up of the possibilities in interpretation,
rather than relying solely upon the primary researcher’s interpretation of the
materials.
The following is not a checklist of criteria for the teams, but rather, starting
points for reflecting on qualitative studies and their usefulness (Jones, 2002b).
1. How are the studies transparent and dialogical? Where is (are) the study’s
researcher(s) coming from? What are the backgrounds, prejudices and beliefs?
(Professional as well as personal; e.g., gender, ethnicity, age, etc.) How are
these limitations? In what way are these revealed?
2. How thoroughly is the research process described? Does the narrative include
roadblocks, false steps, etc. described in a helpful way?
3. How are groups and individuals studied included in the larger research
process? How are they involved in the research design? How are direct
benefits to research participants included?
4. In what way is attention paid to the wider cultural context? For example, is
the study of a minority group (age, ethnicity, class, etc) placed in context with
the larger culture (‘dominant’ culture, majority, other age groups [e.g.,
youth/older people])?
5. How is the study presented as a narrative? How does the writing take the
reader on a journey? Is it dialogical?
6. Is dissemination, more generally, part of the research design? How is
feedback from both participants and the wider community included in the
design and dissemination?
7. What are the main metaphors? How are they productive, engaging and
explanatory? What specific attention is given to language and the construction
of realities by use of language?
Kip Jones
104
8. What are the studies bases in philosophical and theoretical ground? Are they
transparent from the start?
9. How are ethical issues clearly delineated?
Many of the proposed and emerging standards for quality in interpretive social
science are also standards for ethics (Lincoln, 1995, p. 286). Lincoln discusses
several issues in qualitative research that enlarge the debate about standards:
• Problems of the face to face encounter
• The virtual impossibility of maintaining anonymity under
some circumstances
• Selecting and excluding material to be included in case study
• Open and honest negotiations around data collection, analysis
and presentation. (1995, p. 287)
10. Is it useful?
In what imaginative ways might the study contribute to both a
body of knowledge and society?
The abilities required of group participants are openness and
creativity/imagination rather than knowledge of specific research methods or even
fields under review. In fact, diversity of approach to the material should be solicited
and encouraged. "The value of the panel of analysts and of peer review lies in part in
the capacity of different researchers to have anxieties that are different form those of
each other ..." (Wengraf in Chamberlayne, Bornat & Wengraf, 2000, p. 144) and from
that of the original author.
What is Narrative?
Freeman proposes narrative as a way of ordering the ‘landscape of events’
(Freeman, 1984, p. 7). ‘The fact that particular forms of knowledge derive from
retrospection, from an essentially backward look over the terrain of experience, is an
irreducible “peculiarity” of the human being’ (1984, p. 9). Freeman views the
structure of the narrative as representing the imposition of a continuous account upon
discontinuous data (1984, p. 10). For example, Hunter (in Mishler, 1995) reminds us
that medicine is filled with stories and is, in fact, dependent on narrative, is
essentially case-based knowledge and practice and that clinical judgement is
‘fundamentally interpretative’ (1995, p. 112-113). Greenhalgh proposes:
‘Appreciating the narrative nature of illness experience and the intuitive and
subjective aspects of clinical method does not require us to reject the principles of
evidence based medicine’ (Greenhalgh, 1999, p. 325). Harré concludes:
It is an essential characteristic of narrative to be a highly sensitive guide
to the variable and fleeting nature of human reality because it is, in part,
constitutive of it. This makes it such an important subject of inquiry for
the human sciences in general. . . Narratives are both models of
threshold and models of the self. It is through our stories that we
construct ourselves as part of our world. (Harré, 1997, p. 278-79)
By adopting a narrative rather than an empirical mode of inquiry, we allow
reviewers to get closer to the phenomena studied in several ways. First, the narrative
provides access to the specific rather than the abstract; secondly, narratives allows
105
The
Qualitative
Report
March 2004
experience to unfold in a temporal way; thirdly, everyday language and its nuances
are encouraged; finally, narrative allows personal dynamics to reveal themselves in
the actions and relationships presented as well as the reviewers response to them
(Chesla, 1995, p. 73). It is important to remember that even the most quantitative of
us still approach work with the ‘hidden agenda’, if you will, of our background,
culture, experience, preferences and prejudices. Part of being post modern in our
approaches includes acknowledging as much of these things as possible and being
vigilant in discovering the more hidden ones. By clearing the air in this way, we not
only can attempt to produce more transparent data, but also can often find keys to
understanding that we may have otherwise overlooked.
In considering a narrative approach, we arrive at the hermeneutic. Gergen
(1997) proposes, ‘Hermeneutical deliberations serve the valuable function of
thwarting the modes of depersonalisation so common to the empirical research
tradition’ (1997, p. 6). ‘Essentially, the task of interpretation necessitates critical self-
reflection for there to be validity, a respect for the autonomy of the text’ (Freeman,
1984, p. 12). The data itself, therefore, becomes the ultimate reference with its
‘seeming refusal to fit into tightly sealed categories or schematisms’ (1984, p. 14), as
is the nature of so many quantitative approaches to systematic review. ‘Misguided
efforts to verify findings (for example, the use of test-retest and inter-rater reliability
kinds of measures) suggest a misplaced preoccupation with empirical rather than
narrative standards of truth and a profound lack of understanding of the temporal and
liminal nature and vital meaning-making functions of storytelling’ (Sandelowski,
1991, p. 165).
What is Analytic Induction?
Analytic induction represents the bedrock ‘from which Glaser and Strauss’s
work on grounded theory derives’ (Chalip in White, Chalip, & Marshall, 1998, p. 3)
and was first described by the sociologist Florian Znaniecki in 1934 (Ratcliff, 2001,
p. 1; Robinson, 1951, p. 812). In the late teens of the last century, F. Znaniecki
developed the research-technique known as the analysis of human documents (letters,
memoirs, life histories and so forth) with the seminal work, The Polish Peasant in
Europe and America (1918-1920; with W. I. Thomas, 1958). This approach to life
and lived experience was later defined as the autobiographical method in sociology
and located in the theory of symbolic interactionism (Plummer, 1983, p. 40).
Znaniecki was a member of faculty at the University of Chicago at the time when its
Department of Sociology –the first of its kind in the U.S.—was known as ‘the
Chicago School.’ ‘It was the first American university to establish an original
collective school of thought: pragmatism’ (Plummer, 1983, p. 51).
Znaniecki held that analytic induction is the true method of the physical and
biological sciences, and that it ought to be the method of the social sciences too
(Znaniecki cited in Robinson, 1951, p. 812). Inductive rather than deductive
reasoning is involved, allowing for modification of concepts and relationships
between concepts. The process occurs throughout the action of doing research with
the goal of most accurately representing the reality of the situation. No analysis is
considered final, since reality is constantly changing. The emphasis in analytic
induction is on the whole, even though elements and the relationships between
elements are analysed. A specific case need not necessarily be ‘average’ or
representative of the general phenomena studied. It is crucial, nonetheless, that a case
Kip Jones
106
has essential characteristics and that it function as a pattern by which future cases can
be defined (1951, p. 1).
In 1950, Cressey summarised Znaniecki’s analytic induction as six steps:
1. A phenomenon is defined in a tentative manner.
2. A hypothesis is developed about it.
3. A single instance is considered to determine if the hypothesis is
confirmed.
4. If the hypothesis fails to be confirmed, either the phenomenon is
redefined or the hypothesis is revised to include the instance
examined.
5. Additional cases are examined and, if the new hypothesis is
repeatedly confirmed, some degree of certainty about the hypothesis
results.
6. Each negative case requires that the hypothesis be reformulated
until there are no exceptions.
(Cressey cited in Ratcliff, 2001, p. 1)
Analytic induction, however, contrasts to the now more widely used and
invoked grounded theory (Glaser & Strauss, 1967) in several ways. Analytic
induction tests as well as generates theory and all data available must by used to test
hypotheses (Ratcliff, 2000, p. 2). Additionally, ‘in interpretive (hermeneutic)
research, unlike in grounded theory, the goal is to discover meaning and to achieve
understanding’ (Benner, 1994, p. 10). Inductive data analysis, as an alternative to
grounded theory’s ‘constant comparison method’ (Thomas in White et al., 1998, p. 1)
‘is typically qualitative; it makes use of comparisons (typically of cases); it often
makes use of techniques which share some affinity with phenomenology and
hermeneutics’ (Chalip in White et al., 1998, p. 3). By using analytic induction within
a phenomenological or hermeneutic approach, a philosophical statement is made
about the underpinnings of the analysis (White in White et al., 1998, p. 5).
The concept of this analytic process is not an easy one to envisage. It can be
explained as similar to ‘brain-storming’ techniques. The principles of brainstorming
embrace the ideas that all suggestions are considered valuable and that no idea or
thought is negated or not included for possible hypothesis testing at a later stage. It is
a process of freeing up participants from censoring their thoughts (and each others)
and reaching deeper levels of creative reflection and participation in a group process.
By eliminating negativity, brainstorming encourages full member participation and
contribution to the process.
By concentrating on analytic induction as a process of hypothesis raising and
testing in a group setting, a clear cut agenda is set for the reflective team meetings. It
is based in the process of revealing data piece by piece with no ancillary information
available and then hypothesising on projected future possibilities and/or
developments. The principle of this analytic tool is based upon that of ‘abduction,’
developed by Charles Sanders Peirce. It involves ‘generating hypotheses contained in
a given unit of empirical data, progressing to hypotheses as to further developments
and then testing these with the empirical outcome’ (Chamberlayne & King, 1996, p.
213).
107
The
Qualitative
Report
March 2004
Although this narrative approach to evidence review using analytical
induction in a reflective team setting is yet to be tested, it is put forth now as one way
forward in considerations of systematically reviewing qualitative studies. By
returning to the theoretical foundations of both narrative and analytic induction, a
case is made for further explorations of systematic review of qualitative work within
a framework that is sympathetic to qualitative work itself. Efforts such as these will
return reviews of qualitative efforts to a reliance upon the diversity and creativity that
has emerged from the development and expansion of qualitative methods over the
past decade or so.
Relational Qualities to look for in Reviewing Qualitative Research
Signposts for reviewing research in a qualitative, narrative way:
• Dialogical—in what way is dialogue produced: between the researched and
the researcher, the researcher and the reader, and the public at large?
‘A performance, such as an interview, is a bounded, theatrical social act,
a dialogical production’ (Denzin, 2001, p. 44).
When I finish reading a particular paper, do I feel that the author has just had
a conversation with me personally? Have I been drawn in, enticed, has my
world been included?
• Experiential –how does it evoke a sense of the shared habitus?
‘Habitus –our second nature, the mass of conventions, beliefs and
attitudes which each member of a society shares with every other
member’ (Scheff, 1997, p. 219).
For example, we all share certain life events in common, no matter what the
culture to which we pay allegiance: birth, death, love, hate, hunger, war,
poverty, joy and so forth.
• Reflective—in what way is the researcher transparent?
‘No longer does the writer-as-interviewer hide behind the question—
answer format, the apparatuses of the interview machine’ (Denzin,
2001, p. 30).
Do we know something about the person who is talking to us and is this why
we respond/do not respond to what we are reading? Has this been the case
with the participants in the study? Is there a direct line of communication from
the study participant to the reader, thus keeping the writer transparent?
• Narrative—how is it a good story? In what way does it take the form of a
familiar plot?
In the best narrative work, descriptive/interpretative analysis is a story
about stories. When it veers from this basic concept, it goes off course.
Kip Jones
108
When I, as a narrative researcher, look for stories to tell there is another
overarching story to tell in how I came to be in this particular landscape
in the first place. What is it about me and my peculiar interface with
society, policy, trends and conventions that led me on the particular path
I take? If I disclose this half of the circle then the second half makes
sense. It is within the fullness of this circle that the hermeneutic process
becomes complete. Only when I can find myself in an ‘other’ can I
began to understand what is unique and individual about an ‘other’ and
ultimately what is distinctive about myself.’ (Jones, 2000, p. 2)
Conclusions
The paper puts forth the argument that the systematic review of qualitative
research is best served by reliance upon qualitative methods themselves. Although
perhaps misunderstood in more quantitative camps, the narrative review of literature
and narrative as a concept and method in itself are natural allies in doing qualitative
research. Rather than dismiss the narrative review, a case is made for strengthening it,
both by using recent developments in qualitative systematic review and developing a
consensus-building tool by using reflective teams to interpret qualitative studies
through a system of narrative review.
The essence of qualitative work is described, including it natural concerns
with issues such as meaning, truth, purpose and the significance of things. The
poignancy and natural ability for issues of ethnicity and diversity to be investigated
through a qualitative approach is elaborated. Its inclusionary nature (akin to
Participatory Action Research) and ability to give service-users a voice, both through
the research process itself and in reports and documents, is reported. The storied
nature of health and health information is noted. A case is made for the polyvocal
nature of qualitative research and its dialogical, performative qualities. Recent
developments in systematic review are discussed, including the Delphi and Signal and
Noise techniques. A case is made for the inclusion of grey literature in the review
process and its importance as sources of the views of the disadvantaged and
disenfranchised. Scoping studies are observed as particularly valuable in reaching
consensus and involving service-users and/or their representatives. The earlier
development of meta-ethnography as a systematic review process is explained.
Recent developments building upon meta-ethnography, including work at the EPPI
Centre at the University of London and the work of Campbell, Britten and others, are
noted.
A method is proposed that builds on these methodological developments in
systematic review by adopting a narrative inductive method to order the landscape of
qualitative research reviews, using reflective teams to analyse and interpret aggregate
data. Narrative is argued as a particular knowledge-generating method, particularly
useful to the interpretation of language-based research. The reflective nature of the
hermeneutical is proposed as providing keys to concepts of validity and grounding the
method philosophically.
Analytic induction is explained as the predecessor and bedrock of the now
more frequently invoked grounded theory. Its history, based in the analysis of human
documents, is noted. Inductive reasoning is clarified and the utility of reflective teams
as a natural expression of the method is described. Finally, relational qualities that
distinguish qualitative research from more quantitative investigation are delineated.
109
The
Qualitative
Report
March 2004
A list of starting points for reflecting on qualitative studies and their usefulness is
offered.
The proposed method builds upon developments in systematic review of
qualitative documents which are inclusionary and bring service-users and their
representatives into the mainstream of the health and social care research agenda. By
the use of inductive analysis in reflective team settings to analyse qualitative research
healthcare studies, a polyvocal approach to systematic review comes full circle.
References
Benner, P. (Ed.). (1994). Interpretive Phenomenology: embodiment, caring, and
ethics in health and illness. Thousand Oaks, CA: Sage Publications.
Booth, A., & Fry-Smith, A. (2003). Developing the research question. Extext on
Health Technology Assessment (HTA) Information Resources (chap. 2).
Retrieved from National Library of Medicine online at:
http://www.nlm.nih.gov/nichsr/ehta/chapter2.html
Britten, N., Campbell, R., Pope, C., Donovan, J., Morgan, M., & Pill, R. (2002).
Using meta ethnography to synthesise qualitative research: A worked
example. Journal of Health Services Research and Policy, 74, 209-215.
Calasanti, T. M. (1996). Incorporating diversity: Meaning, levels of research, and
implications for theory. The Gerontologist, 36(2), 147-156.
Campbell, R., Pound, P., Pope, C., Britten, N., Pill, R., Morgan, M., & Donovan, J.
(2003). Evaluating meta-ethnography: A synthesis of qualitative research on
lay experiences of diabetes and diabetes care, Social Science & Medicine, 56
(4), 671-684.
Chamberlayne, P., & King, A. (1996). Biographical approaches in comparative work:
The ‘cultures of care’ project. In L. Hantrais & S. Mangen (Eds.), Cross-
national research methods in the social sciences (pp. 95-104). New York:
Pinter.
Chamberlayne, P., Bornat, J., & Wengraf, T. (Eds.). (2000). The turn to biographical
methods in social science: Comparative issues and examples. New York:
Routledge.
Chesla, C. (1995). Hermeneutic phenomenology: An approach to understanding
families. Journal of Family Nursing, 1(1), 68-78.
Denzin, N. K. (2001). The reflexive interview and a performative social science.
Qualitative Research, 1(1), 23-46.
Dixon-Woods, M., Fitzpatrick, R., & Roberts, K. (2001). Including qualitative
research in systematic reviews: Opportunities and problems. Journal of
Evaluative Clinical Practice, 7(2), 125-133.
Freeman, M. (1984). History, narrative, and life-span developmental knowledge.
Human Development, 27, 1-19
Freeman, A., Strong, M. F., Barker, L. T., & Haight-Liotta, S. (1996). Priorities for
future research: Results of BPA's delphi survey of disabled women. Retrieved
January 1, 2004, from http://www.ncddr.org/rr/women/priorities.html
Gergen, K. J. (1997). Social theory in context: Relational humanism. Retrieved
January 1, 2004, from
http://www.swarthmore.edu/SocSci/kgergen1/web/printer-
friendly.phtml?id=manu9
Gergen, K. J. (2000). The poetic dimension: Therapeutic potentials, draft for K.
Deissler and S. McNamee (Eds.), Phil und Sophie auf der Couch, Die soziale
Kip Jones
110
Poesie Therapeutischer Gesprache,. Carl-Auer, Heidelberg. Retrieved June 2,
2004, from Swarthmore College, Department of Psychology Web site:
http://www.swarthmore.edu/SocSci/kgergen1/web/printer-
friendly.phtml?id=manu19
Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory. New
York: Aldine.
Gough, D., & Elbourne, D. (2002). Systematic research synthesis to inform policy,
practice and democratic debate. Social Policy and Society, 1(3), 225-236.
Greenhalgh, T. (1999). Narrative based medicine: Narrative based medicine in an
evidence based world. British Medical Journal, 318(7179), 323-325.
Hammersley, M. (2003, February). Systematic or unsystematic? Is that the
Question?: Some reflections on the science, art, and politics of reviewing
research evidence. Paper presented at seminar series, Department of
Epidemiology and Public Health, University of Leicester, UK.
Harré, R. (1997). He lived to tell the tale. Journal of Narrative and Life History, 7(1-
4), 331-334.
Hiatt, J. F. (1986). Spirituality, medicine, and healing. Southern Medical Journal,
79(6), 736-743.
Higginson, I. J., Finlay, I., Goodwin, D. M., Cook, A., M., Hood, K., Edwards, A. G.
K., et al. (2002). Do hospital-based palliative teams improve care for patients
or families at the end of life? Journal of Pain and Symptom Management,
23(2), 96-106.
Hollway, W., & Jefferson, T. (2000). Doing qualitative research differently: Free
association, narrative and the interview method. London: Sage Publications.
Johnson, M. R. D. (2003). Research governance and diversity: Quality standards for a
multi-ethnic NHS. Nursing Times Research, 8(1), 27-35.
Jones, K. (2000). Big science or the bride stripped bare by her bachelors, even.
Qualitative Social Research, 1(3). Retrieved January 1, 2004, from
http://www.qualitative-research.net/fqs-texte/3-00/3-00review-jones-e.htm
Jones, K. (2002a, April). Participatory mapping of community resources at a local
level to reflect service-users’ needs and involvement. Paper presented at
Caring with Diversity: Ethnicity, Nursing & Multi-agency Working, Mary
Seacole Research Centre Annual Conference, De Montfort University, UK.
Jones, K. (2002b, October). Some initial reflections on systematic reviews of
qualitative research in health, ageing and diversity. Paper presented at ESRC
Evidence Network Annual Meeting, Glasgow University, UK.
Jones, K. (2002c). Narratives of Identity and the Informal Care Role. Unpublished
doctoral dissertation, De Montfort University, UK.
Jones, K. (2003). The turn to a narrative knowing of persons: One method explored.
Nursing Times Research, 8(1), 60-71.
LeCompte, M. D. (2002). The transformation of ethnographic practice: past and
current challenges. Qualitative Research, 2(3), 283-299.
Lincoln, Y. S. (1995). Emerging criteria for quality in qualitative and interpretive
research. Qualitative Inquiry, 1(3), 275-289.
Merriam-Webster Dictionary online. Available at:
http://www.m-w.com/dictionary.htm
Mishler, E. G. (1995). Models of narrative analysis: A typology. Journal of Narrative
and Life History, 5(2), 87-123.
111
The
Qualitative
Report
March 2004
Nicholson, P. (Ed.). (2000). Eliciting users’ views of the processes of health care: A
scoping exercise. The School of Health and Related Research (ScHARR) and
Consumer Health Psychology at ScHARR (CHePAS).
Retrieved January 1, 2004, from
http://www.sdo.lshtm.ac.uk/pdf/elicitingusersviewssc
opingexercise_execsumm.pdf
Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative
studies. Newbury Park, CA: Sage.
Oxford English Dictionary (OED) online. Available at:
http://dictionary.oed.com/entrance.dtl
Packwood, A. (2002). Evidence-based policy: Rhetoric and reality. Social Policy &
Society, 1(3), 267-272.
Plummer, K. (1983). Documents of life: An introduction to the problems and
literature of a humanistic method. London: George Allen & Unwin.
Popay, J., Rogers, A., & Williams, G. (1998). Rationale and standards for the
systematic review of qualitative literature in health services research.
Qualitative Health Research, 8(3), 341-351.
Pope, C., & Mays, N. (Eds.). (2000). Qualitative Research in Health Care. Retrieved
January 1, 2004, from http://www.bmjpg.com/qrhc/
Raholm, M. B., Lindholm, L., & Eriksson, K. (2002). Grasping the essence of the
spiritual dimension reflected through the horizon of suffering: An
interpretative research synthesis. Australian Journal of Holistic Nursing, 9(1),
4-13.
Ratcliff, D. E. (2001). Analytic induction as a qualitative research method of analysis.
Retrieved January 1, 2004, from http://don.ratcliff.net/qual/analytic.html
Robinson, W. S. (1951). The logical structure of analytic induction. American
Sociological Review, 16, 12-18.
Runyan, W. McK. (1982). Life histories and psychobiography explorations in theory
and method. Oxford: Oxford University Press.
Sandelowski, M. (1991). Telling stories: Narrative approaches in qualitative research.
IMAGE: Journal of Nursing Scholarship, 23(3), 161-166.
Scheff, T. J. (1997). Emotions, the social bond, and human reality part/whole
analysis. Cambridge, MA: University Press.
Scottish Executive Central Research Unit. (2001). Scoping study for a national survey
of Scotland’s minority ethnic populations: Other issues arising from the
consultation (appendix four). Retrieved January 1, 2004, from
http://www.scotland.gov.uk/cru/kd01/red/minethnic-18.asp
Thomas, W. I., & Znaniecki, F. (1958). The Polish peasant in Europe and America.
New York, Dover Publications.
White, G., Chalip, L., & Marshall, S. (1998, August 26). Grounded theory and
qualitative data analysis. Electronic discussion message posted to Health
Research Methods Service (HRMAS) Newsletters (9, 10, 11). Accessed
December, 2001 at: http://www.auckland.ac.nz/mch/hrmas/qual2d.htm
Yin Robert, K. (1989). Case study research: Design and methods. Newbury Park,
London: Sage Publications.
Young, K., Ashby, D., Boaz, A., & Grayson, L. (2002). Social science and the
evidence-based movement. Social Policy & Society, 1(3), 215-224.
Kip Jones
112
Author Note
Kip Jones, Ph.D. is an ESRC Research Fellow at the Centre for Evidence in
Ethnicity, Health and Diversity (http://users.wbs.ac.uk/group/ceehd) based at the
Mary Seacole Research Centre (http://www.dmu.ac.uk/msrc/) at De Montfort
University, Leicester and at Warwick University, Coventry, in the UK. He is
developing qualitative approaches to systematic review, including using meta-
ethnography and narrative techniques to ‘synthesise’ qualitative research. His specific
area of interest is in health and ageing issues and their interface with ethnicity,
diversity, and race. Dr. Jones has written on and given workshops in narrative
biographic interpretive method. His chapter, 'Minimalist Passive Interviewing
Technique and Team Analysis of Narrative Qualitative Data' has recently been
published by Routledge in New Qualitative Methodologies in Health and Social Care,
F. Rapport (Ed.). He is Associate Book Review Editor of the online journal, Forum:
Qualitative Social Research (http://www.qualitative-research.net/fqs/fqs-eng.htm).
Correspondence regarding this article should be addressed to Dr. Jones at Mary
Seacole Research Centre, De Montfort University, Charles Frears Campus, 266
London Road, Leicester LE2 1RQ, United Kingdom; Telephone: +44 (0)116 201
3906; Fax Telephone: +44 (0)116 201 3805; E-mail: kjones@dmu.ac.uk.
Copyright 2004: Kip Jones and Nova Southeastern University
Author’s Citation
Jones, K. (2004). Mission drift in qualitative research, or moving toward a systematic
review of qualitative studies, moving back to a more systematic narrative
review. The Qualitative Report, 9(1), 95-112. Retrieved [Insert date], from
http://www.nova.edu/ssss/QR/QR9-1/jones.pdf
