Validity and action research: An online
conversation
© 1999 Judith M. Newman
with Shankar Sankaran, Brian
Murphy, Jack Whitehead, Bob Dick, Pam Swepson, John O'Brien, Dina
Boogaard and Pat D'Arcy
An online conference on "The Reflective Practitioner" 1 was held during
March of 1998. The conference lasted for three weeks, followed by a one week
roundup. The objective of the conference was to focus on the late Donald
Schon's work on The Reflective Practitioner and the issues his work raises for
action researchers.
Following the conference several participants continued the on-line
discussion. One issue which particularly interested me was our discussion
about "validity" and the significance of this construct within an action
research context. The prompt which triggered the discussion had been posted a
month earlier; it was my response to Shankar's message which set the
conversation in motion.
I happened across this conversation while browsing my saved email. I was
surprised at the coherence of the contributions and the specific interplay
between the other correspondents and myself. The various responses to my
writing prompted me to flesh out my theoretical views. I had never before
written about "validity" -- it was in the course of this conversation that I
was able to make explicit for myself what I considered to be some of the
critical issues for action researchers to consider.
Without further ado, let me share the written conversation which took place
in just a single week.
Date: Mon, 11 May 1998
From: Shankar
Sankaran<elogue@mbox2.singnet.com.sg> [ NOTE ]
Dear all,
I am about to put finishing touches to the final chapter
of my action research thesis and am trying to defend it for rigour in terms
of
1. Validity of my findings
I have tried to do this using criteria suggested by both from the
qualitative research literature and action research/ action inquiry
literature....
Shankar
Date: Tue, 09 Jun 1998
From: Judith Newman
<jnewman@hfx.andara.com>
Shankar,
The issue of "validity" is, I think, a "red herring" in our line of work.
The point of a piece of action research isn't to "prove" anything -- the most
any research account can really do (no matter what the flavour) is to allow
the reader to take a fresh look at his or her own work. More traditional forms
of research wave a "validity" flag because rarely do readers see any real
connection to their work so the researcher attempts to persuade us readers
that this work matters with a discussion of validity. But if "truth" isn't
what I care about, if what I care about is whether I can see the problematic
in my own work with new eyes, if your inquiry raises questions that I think I
might find it useful to ask of myself, then that's what I think this
enterprise is about. How do I find out whether my discussion resonates for my
readers? Well, I have to give it to some folks to discover what their response
is.
Judith
Date: Wed, 10 Jun 1998
From: Brian Murphy
<bmurphy@iol.ie>
Judith,
You write:
-
> the most any research account can really do (no matter what the
flavour)
> is to allow the reader to take a fresh look at his or her
own work.
Do you mean by "his or her own work" the work of the reader, as opposed to
the work of the individual reported in the research account?
If the former, then it seems to be suggesting that the only purpose for
reading research is to understand better what we are doing ourselves -- we are
not really interested in the "account in itself" only in the leverage it gives
us. This seems unduly self-centred.
Sure, when we read research accounts we try to relate them to our own work
and interests but we also sometimes try to "judge" them or come to some
opinion about them. In particular, where we have doubts about the plausibility
and credibility of the research we need to be convinced by the nature of the
evidence presented. (Though I accept that some people will accept an account
on the basis of a researcher's claims to "intensive, personal involvement").
So, to me, this implies that validity is an important consideration and should
be taken seriously.
Regards,
Brian
Date: Wed, 10 Jun 1998
From: Jack Whitehead
<edsajw@bath.ac.uk>
I've enjoyed so much of Judith's work and agreed with her ideas. So, this
morning I was startled by her points about validity. I'm smiling as I write
because I know they challenge a fundamental commitment of mine to validity! As
a university researcher and teacher I have a commitment to help my Ph.D.
students to make original contributions to educational knowledge. All my
students are educational action researchers who are asking questions of the
kind: How do I improve what I am doing? As they describe and explain their own
professional learning (create their own living educational theories) I help to
establish validation groups in which groups of critical friends subject their
explanations to questions of the sort: Is the explanation comprehensible? Are
the assertions sufficiently supported by evidence? Are the values clarified
and justified in the course of their emergence in practice? Does the
explanation live in the sense of containing an evaluation of past practice and
an intention to create something better in the future?
I'm still holding to the idea of validity as crucially important to action
researchers who are seeking to make significant and original contributions to
knowledge.
Warm regards
Jack
Date: Wed, 10 Jun 1998
From: Bob Dick
<bdick@scu.edu.au>
Dear Judith
Before I get too far into the current discussion I think it might be useful
if I define my understanding of the terms. I have in mind "validity" and
"truth," both of which I find problematic.
By "validity" I mean something like "agreement with reality" or "more than
fiction or fantasy. "
By "truth" I mean something similar. In both instances I recognise that my
views of the world are constructions of mine. At the same time, I hope in
developing these constructions I am influenced by the data. I hope, further,
that these data (whatever form they take) are influenced by what happens in
the "real world. "
(Yes, I do assume there is a world out there, and that it is to some extent
knowable.)
When I do action research I have some interest in changing my behaviour or
in helping others decide if they wish to change their behaviour. To do this, I
think, requires that I or they develop models of the world that somehow match
that world. I think of this as "validity. "
Defined in this way, it is a concern of mine. From what you say, it may or
may not be a concern of yours.
In the interests of my own education, I invite you to say more about your
position.
Warm regards
Bob
Date: Wed, 10 Jun 1998
From: Judith Newman
<jnewman@hfx.andara.com>
I guess I hit some kind of nerve with my thoughts on "validity." This will
be a long reply because I'm going to respond to Bob, Jack and Brian by
dialoguing with their words.
Judith
========
Bob Dick wrote:
-
> Before I get too far into the current discussion I think it
might
> be useful if I define my understanding of the terms. I have
in
> mind "validity" and "truth," both of which I find
problematic.
>
> By "validity" I mean something like "agreement
with reality" or
> "more than fiction or fantasy. "
>
>
By "truth" I mean something similar. In both instances I
> recognise
that my views of the world are constructions of mine.
> At the same
time, I hope in developing these constructions I
> am influenced by
the data. I hope, further, that these data
> (whatever form they take)
are influenced by what happens in the
> "real world. "
>
> (Yes, I do assume there is a world out there, and that it is
to
> some extent knowable.)
I think I've become more of a deconstructionist than Bob is -- that is,
while I accept there is a "real" world "out there" I'm not convinced it's
knowable in any "objective" sense. I think we're always dealing as blind men
with the elephant -- and it's precisely our different views of "reality" that
are important to understand so that from time to time we can step out of our
own limited perception and entertain alternate interpretations of that
world.
-
> When I do action research I have some interest in changing
my
> behaviour or in helping others decide if they wish to change
their
> behaviour. To do this, I think, requires that I or they
develop
> models of the world that somehow match that world. I think
of this
> as "validity. "
I also have that same interest in changing my behaviour and in helping
others decide if they wish to change theirs. But I don't think it's possible
for anyone to develop models of the world that "somehow match that world"
because any "knowing" is necessarily "interpretation." So while I accept there
is some direct link between a "real" world out there and my sensory responses
-- my "knowing" what those responses mean is an interpretive act and it's at
that point that any direct connection to any "reality" is severed. I attempt
to "validate" my interpretations by touching base with other's interpretations
(it's like creating intersecting sets in a Venn diagram) -- and the extent to
which my interpretations have something in common with somebody else's
reflects a communal interpretation -- a "knowing" that is more than just my
own personal construction.
-
> Defined in this way, it is a concern of mine. From what you
say,
> it may or may not be a concern of yours.
>
> In
the interests of my own education, I invite you to say more
> about
your position.
I am concerned with tabulating my personal interpretations with those of a
wider interpretive community. I came to my current set of beliefs about
meaning and interpretation many years ago when I taught a course on "Reader
Response" -- one of the most interesting texts with which the teachers and I
grappled was Stanley Fish's "Is there a text in this class?" from his book of
the same name in which he contends:
a system of intelligibility cannot be reduced to a list of the things
that it renders intelligible. What Abrams and those who agree with him do
not realize is that communication occurs only "within' such a system (or
context, or situation, or interpretive community) and that the understanding
achieved by two or more persons is specific to that system and determinate
only within its confines. Nor do they realize that such an understanding is
enough and that the more perfect understanding they desire -- an
understanding that operates above or across situations -- would have no
place in the world even if it were available, because it is only in
situations -- with their interested specifications as to what counts as
fact, what it is possible to say, what will be heard as an argument -- that
one is called on to understand (p. 305).2
What I understand Fish to be arguing is that we make personal sense by
constructing meaning within an interpretive community and that the sense made
within any particular interpretive community must, in turn, be explored in
terms of other communities' interpretations but that there is no "match" with
an objective world other than a collective interpretation of that
world.
Jack Whitehead wrote:
-
> As a university researcher and teacher I have a commitment
to
> help my Ph.D. students to make original contributions to
educational
> knowledge. All my students are educational action
researchers who are
> asking questions of the kind: How do I improve
what I am doing? As they
> describe and explain their own professional
learning (create they own
> living educational theories) I help to
establish validation groups in
> which groups of critical friends
subject their explanations to questions
> of the sort: Is the
explanation comprehensible? Are the assertions
> sufficiently
supported by evidence? Are the values clarified and justified
> in the
course of their emergence in practice? Does the explanation live in
>
the sense of containing an evaluation of past practice and an intention
to
> create something better in the future?
I am asking the same questions, Jack -- How do I improve what I am doing?
What counts as evidence? How do my interpretations of a situation connect with
others' understanding? I'd contend that what's happening in the "validation
groups" is that personal understanding/interpretations are being tested within
an interpretive community, that is, the "data," the ideas are responded to so
that the "researcher" can see in what ways his/her perceptions are similar to
and different from those of others. It's important, however, to keep in mind
that the interpretive community creates its own sense of "reality" which is
still an interpretation, albeit a more widely shared one than one's personal
construction.
I do believe I need to situate my ideas/hypotheses within an interpretive
community but I'm hesitant to call that "validity" because of the baggage that
this word carries -- it implies some kind of more direction connection with
"reality" than I believe exists.
-
> I'm still holding to the idea of validity as crucially important to
action
> researchers who are seeking to make significant and original
contributions
> to knowledge.
I think I've abandoned a concern with "validity" and replaced it with a
need to find/create an interpretive community within which data, ideas,
arguments resonate. I am concerned about making "significant and original
contributions" not to knowledge but to the understanding of the interpretive
community.
Brian Murphy wrote:
-
> Judith,
> You wrote:
> the most any research account
can really do (no matter what the flavour) is to
> allow the reader to
take a fresh look at his or her own work.
>
> Do you mean by
"his or her own work" the work of the reader, as opposed to
> the work
of the individual reported in the research account?
When I read a research account, whether it's technical/rational (as Schon3 would call it)
or an account of reflection-in-action, I'm using it to test my ideas, beliefs,
experiences against those of the interpretive community. I don't read any
account as standing on its own, but rather as situated in a discourse
tradition, and offering interpretations of experience against which I think
about my own interpretations. As Margaret Meek 4 puts it: "the
text lets me read myself."
-
> If the former, then it seems to be suggesting that the only purpose
for
> reading research is to understand better what we are doing
ourselves -- we
> are not really interested in the "account in itself"
only in the leverage
> it gives us. This seems unduly self-centred.
It's not a matter of self-centred -- that is what text does, that's how it
operates -- the ink marks on the page (or coloured pixels on a screen) carry
no meaning in and of themselves -- what sense we make of the impression they
make on our retinas is the result of interpretive processes -- we make sense
based on what we bring to the text in terms of our personal experiences, our
beliefs, the collective meaning we carry around with us. I can "read" an
account from what Louise Rosenblatt5 calls an
"efferent" stance (what facts does this poem teach us?) but ultimately any
reading is done against the backdrop of "my theory of the world in my
head."
-
> Sure, when we read research accounts we try to relate them to our
own work
> and interests but we also sometimes try to "judge" them or
come to some
> opinion about them. In particular, where we have doubts
about the
> plausibility and credibility of the research we need to be
convinced by the
> nature of the evidence presented. (Though I accept
that some people will
> accept an account on the basis of a
researchers claims to "intensive,
> personal involvement'). So this to
me implies that validity is an important
> consideration and should be
taken seriously.
But I would argue that "judging" them or "coming to some opinion about
them" is part and parcel of relating them to our own work and interests --
since our work and interests are the backdrop against which we are reading.
It's not possible to come to a text as a clean slate -- we bring who we are to
each and every reading, and that's what makes reading so interesting because
who I am is changed by every reading I do, consequently rereading a text is a
new experience since it's a new "I" who is doing the reading.
Judith
Date: Thu, 11 Jun 1998
From: Brian Murphy
<bmurphy@iol.ie>
Hi Judith,
Thanks for the dialogue. After reading it a few times I couldn't but help
feel that I agreed with just about everything you say, and yet I still cannot
abandon "validity" as you do. The source of this agreement -- yet ultimate
disagreement -- puzzled me somewhat. Where in all these paragraphs did we
actually part company?
I eventually isolated two paragraphs:
-
> I do believe I need to situate my ideas/hypotheses within an
interpretive
> community but I'm hesitant to call that "validity"
because of the baggage
> that that word carries -- it implies some
kind of more direction connection
> with "reality" than I believe
exists.
>
> I think I've abandoned a concern with "validity"
and replaced it with a
> need to find/create an interpretive community
within which data, ideas,
> arguments resonate. I am concerned about
making "significant and original
> contributions" not to knowledge but
to the understanding of the interpretive
> community.
The first paragraph above suggests that a lot of our debate is due to the
"baggage" that the word "validity" carries, mostly coming from a positivist,
scientific community. I do not believe that the word needs to carry that
baggage and it need not imply direct connection with "reality." I view readers
of research accounts as being part of an interpretive community who apply
community interpretive rules or norms in response to the words on the page.
These norms differ greatly between communities but usually require (among
other things) a connection between the claims made in the account and the
evidence presented. It is this notion of validity that I would not like to
jettison, as otherwise it may be difficult to discriminate the bogus,
fictional, anecdotal or concocted.
Your second paragraph says you have replaced "validity" with the notion of
"resonance." Can you say a bit more about this? I do not understand it, so I
cannot say whether I personally could substitute it for validity.
So a lot of the debate could be due to terminology? Maybe definitions of
validity and resonance in qualitative research generally, and action research
in particular, would help. But I'm a novice here and so hesitate to put my toe
in the water.
Regards,
Brian
Date: Thu, 11 Jun 1998
From: Bob Dick
<bdick@scu.edu.au>
Hello all
I'm enjoying the discussion, and hope it will help me understand some
aspects of constructivism that I don't presently understand very well.
In taking the discussion further, I have no intention of trying to persuade
anyone to my point of view. I presume each of us have belief systems that work
for us. My interest is in increasing my understanding.
I'd like to follow Brian's post by trying to define what Judith and I agree
about. If we can establish common ground, the differences are more likely to
be educational, I think.
So, I think Judith and I are agreed ...
- that there is a real world
- that we can know it only indirectly
- that for this reason there are no objective grounds on which we can say
that this interpretation is certainly better than that interpretation
[I would add, except perhaps in the extremes]
and we may be agreed ...
- that we wish to impact in some way on that world
I also suspect the issues about "baggage" are about labels, not realities.
I don't expect them to be important.
"Validity" works well for me, for reasons I'll try to explain below. But
that doesn't give me any reason to expect others to find it useful.
I use a non-reductionist methodology within a faculty that is reductionist.
So I am often regarded as somewhat marginal. No doubt this has had some
influence on my language and practice.
Speaking personally, in such a situation I find it useful to be able to
justify my choice of methodology in terms that make sense to my colleagues.
Supervising theses in this situation, I find it helpful to be able to assist
thesis candidates to defend their thesis according to the rules of the
dominant paradigm.
For these purposes, "validity" is a useful term. My colleagues don't have
any reason to learn my dialect. It therefore makes sense to talk to them in
their own.
However, I don't have any ideological commitment to "validity" as a word.
"Resonance" will do fine if we can agree on what it means, and if it serves
the purposes I require it to serve.
There does seem to be a difference in the conclusions we draw about the
knowability of the world. This is what I'd like to explore.
Judith, you seem to be saying either that you can't know the world, or that
it doesn't matter. This puzzles me, not just here, but when I come across it
in some of the literature. What I don't understand is how I and others can act
effectively on the world if our perceptions of it bear no
correspondence to that reality.
Warm regards
Bob
Date: Thu, 11 Jun 1998
From: Jack Whitehead
<edsajw@bath.ac.uk>
Judith writes:
- > I think I've abandoned a concern with "validity" and replaced
> it with a need to find/create an interpretive community within
> which data, ideas, arguments resonate. I am concerned about
> making "significant and original contributions" not to knowledge
> but to the understanding of the interpretive community.
I agree with all the points Judith has made about interpretive communities.
I also agree with Bob's views on truth and validity. It may be that Judith can
abandon a concern with "validity" because she is focusing on "understanding"
rather than "knowledge." Like Bob I'm also interested in the meaning of
"resonate." One of the standards of judgement examiners of a Ph.D. Thesis are
required to use is that it should make an original contribution to
"knowledge." Because of this it still seems important to stress the importance
of "validity" in the sense of testing the validity of a claim to knowledge.
I'm happy to embrace both the concepts associated with hermeneutics and
epistemology in the sense of valuing "understanding, "ways of knowing" and
"knowledge." I don't yet see them as conflicting.
Warm regards
Jack
Date: Thu, 11 Jun 1998
From: Pam Swepson
<p.swepson@uq.net.au>
Dear Judith and Bob and all
I too am enjoying the discussion on validity and resonance.
I have read a few brief words about what Gareth Morgan has said about
resonance and I have some difficulty with the concept. Perhaps someone can
help me here.
My concern is that resonance sounds like either rationalisation or "group
think. " To explain further, if as some people think, human beings have a
tendency to form an opinion on very little data and then spend the rest of our
time finding data to justify it, either individually or as a group, is this
what resonance is? Feeling some justification for what we want to believe is
true? Where am I wrong here, please?
If I am right, then it seems to me that to check our own "validity" (if you
want to use that word) then you have to work very hard at finding very
different opinions to compare your thinking with and hanging in long enough to
make an honest comparison. Tough stuff, I think.
Any thoughts?
Pam
Date: Thu, 11 Jun 1998
From: Judith Newman
<jnewman@hfx.andara.com>
Brian and Bob, another longish musing.
What I find fascinating is how if we unpack our thoughts more fully we find
we have similar values -- that the language we are using serves as the
barrier. I guess I'm particularly sensitive to the language I choose to use.
Let me relate an incident which occurred twenty years ago which had a
significant impact on how I engage with the academic world.
I had recently finished my PhD. I was at a conference having just attended
a session given by my major mentor Frank Smith (a literacy researcher) and we
were talking about some reading/writing research I was then engaged in. I was
describing what was going on in the classrooms, the "reading" and "writing"
activities we were attempting with the kids when Smith turned to me and asked
"How can you distinguish them?" I paused for a moment and replied "I guess I
can't." And he responded "Well, why do you talk about it that way?"
It was the "Why do you talk about it that way?" that got me -- I couldn't
know if a child was really learning about reading from the "writing"
activities instead of the "reading" activities -- Smith's point, well taken,
is that there is no way of tracing exactly how a learner constructs his or her
understanding of how text operates. I found myself, after that, thinking a lot
about how I "talked" about ideas.
Now this notion of "validity" is one of those ideas that I've thought about
quite a bit -- why do I talk about it that way? I realized I couldn't. Like
you, Bob, I wasn't coming from the same place as folks who use the term -- it
comes from a technical/rationalist tradition. The idea was invoked to regulate
research activity and keep wide ranging inquiry in check. The problem is
nevertheless an interpretive one -- who gets to decide what is "valid" and
what isn't? It's still readers of a research account (whether that reader is a
thesis advisor, or journal reviewers, or colleagues in granting agencies) who
serve as the gate-keepers -- who make the interpretive judgement about whether
a piece of research seems plausible or not. For when it comes down to it a
judgement about "validity" is a judgement about "plausibility." Really, there
is no objective test of "validity" -- the technical/rational research
community is forced to judge "validity" in an interpretive way. And the myth
is that if there is "validity" and "reliability" then there will be
"replicability," right? But when you look at reflective practice6 (I'm using
Schon's dichotomy for convenience sake) it's clear that replicability isn't a
goal here. As Schon made abundantly clear in his work, the situations on which
we are reporting are unique from moment to moment and the characteristics of
the situation in which I work are likely to be quite different from those in
which you work and so the kind of test which makes some sort of sense isn't
this notion of "validity" but of something else -- I've opted for an openly
subjective idea like "resonance" -- my test when I read a piece of research
includes the following:
- does it offer enough thick description for me to "live" in the
situation, to see it in some depth (it's never possible to offer more than a
slice of an interpretation of what's going on, and what's written about is
selected from a huge store of evidence so that the writer can focus on what
he or she decides are critical ideas)
- does the account situate itself in the research conversation, does the
writer make clear the debates which have influenced his or her thinking?
- are the writer's assumptions made explicit and do they reflect on them?
- does the account offer some thoughts on "So What?" -- how has this work
affected the way in which the writer engages in his or her professional
work?
The stance I take as a reader is an aesthetic one as Louise Rosenblatt7 calls it -- I
read research accounts as I do novels -- I attempt to "live through" the
experience. As with a good novel, if the account lets me in, allows me to live
in that fictional world and I can see connections to my world, it's likely to
permit me to see my own work in new ways. I'm not looking for correspondence
-- I know my work situation is unique, as was the research situation being
described -- rather I'm looking for what I have come to call "resonance" --
does the account seem believable, does it help me think about the problematic
of my working situation, does it help me name or reframe the tensions in my
work so that I might do something about them?
Sondra Perl,8 a writing
researcher invokes the notion of "felt sense" in writing -- you can tell when
some writing is going well because it "feels" right -- she even goes so far as
to explore how the body identifies what this "feeling" is. I think it's the
same with research accounts -- particularly action research accounts which
have a strong narrative flavour -- they "feel" right. I go away from my
reading wondering in new ways.
Now, Bob, you're concerned about living with colleagues -- that's certainly
a concern. I guess I discovered I was what Doris Lessing9 referred to as a
"zone three person in a zone four world (The Marriages Between Zone Three,
Four, and Five) -- that novel helped me understand the dissonance I was
feeling living in an academic setting, it let me "read myself." It allowed me
to recognize that I was always going to be at odds with colleagues over
questions about teaching, about assessment, about learning above all. So I
made the decision that I could free myself from talking about things in the
way they did -- and having made that decision I've been able to get on with
the kind of inquiry I think is useful to me personally and which seems to
resonate for teachers -- they respond to what I write in a reflective way
which seems to impact on how they think about their own work. Since teachers
are my audience, not my academic colleagues, it's their judgement about the
utility of my research and writing for them that counts for me.
Judith
Date: Thu, 11 Jun 1998
From: Judith Newman
<jnewman@hfx.andara.com>
Bob Dick wrote:
- > Judith, you seem to be saying either that you can't know the world,
or
> that it doesn't matter. This puzzles me, not just here, but when I
> come across it in some of the literature. What I don't understand
is
> how I and others can act effectively on the world if our
perceptions
> of it bear no correspondence to that reality.
Bob, clearly there has got to be some correspondence between our
perceptions and reality "out there." I just don't think we can "know" it in
any objective way. My interpretation of what's happening in my middle/inner
ear as it's receiving speech depends on a whole host of judgements I'm making
about whose speaking, what they're likely to be speaking about, what they
know, what my expectations are, and on and on. There's a field of study --
Signal Detection Theory10 -- dealing
with radar operators -- the "reading" of a radar screen is very complex -- the
reader has to make a judgement about whether a blip represents an enemy
aircraft or not. They end up with "hits', "misses" and "false alarms" -- the
gist of the arguments is that it is not possible to make perfect
identifications because you can't completely eliminate the "noise" from the
system. Even reading a radar screen is an interpretive process!
Judith
Date: Sat, 13 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com
>
Jack, I don't have your message in front of me -- I seem to have lost it
from my hard drive. But what I remember of the gist of it is that you're
uncomfortable with my abandonment of the notion of validity, that you feel my
rejection of the notion somehow implies that you must rethink it, too.
I have entered this discussion, not to change anyone's mind about ideas
such as this, but to share my journey, to let you see the path I've taken and
why.
I simply stopped playing the academic game about 15 years ago. I decided
that the kind of work involving teachers that I felt I needed to do needed a
different character -- one that the academy didn't understand or support. I
remember a major funding agency rejecting my research proposal as a "promising
reject" because I couldn't lay out the inquiry in the expected fashion. I
decided, then, that rather than rewrite and resubmit the proposal, I would
conduct what research I wished to do unfunded. That has been a wise decision
for me. It has let me work with teachers, and the teachers with their
students, exploring questions of interest to us all. We have been successful
publishing some of this work in periodicals which do support teacher action
research, and as collected articles in books, now on the internet. We've been
able to get our ideas into the research conversation and to grow from other's
responses to what we've had to say. It's that public response that has been
important for shaping our thinking.
But as for attempting to change how you think about things -- that hasn't
been my intention. I've simply wanted to share how I've come to think about
things, and if those thoughts resonate for anyone, that's fine, but it's not
necessary that they do.
Judith
Date: Sat, 13 Jun 1998
From: John J. O'Brien
<jobrien@access.victoria.bc.ca>
Interesting thread. To add to the melange -- Pam queries whether resonance
is about "feeling some justification for what we wish to believe is true. " I
understand and concur with the danger inherent in seeking confirmation for
preconceptions. However, I do not think that resonance is about that. Things
resonant when we disagree with them, too. Perhaps resonance is a matter of
confirmation of a felt sense of truth in some regard -- but I don't think it
implies agreement. It is, perhaps, another facet for observation and
learning.
Thoughts?
John
Date: Sat, 13 Jun 1998
From: Judith Newman <
jnewman@hfx.andara.com>
John, you've put that nicely. I'd say that "resonance" is that sense that
this connects/ extends my sense of the world, this doesn't feel right -- the
sort of response that's happening with this very discussion -- for some, the
notion of "resonance" seems to fit and for others it's proving uncomfortable.
I'd say that we're testing the "validity" of the argument/data against what we
believe or have experienced about the world -- that's all any of us can do --
I think.
Judith
Date: Sat, 13 Jun 1998
From: Dina Boogaard
<DJBoogaard@aol.com>
Judith,
I just wanted to say that I deeply appreciate where you are coming from
regarding the concept of validity and the limiting aspects of the Academy. And
I think you have explained it eloquently and clearly. I also very much
appreciate the contributions of everyone inquiring into this arena. It is all
contributing to my own understanding.
I'm just going to add a few paragraphs from some thoughts about qualitative
research that I wrote down a few months ago, in my quest to better understand
what I'm dealing with as I plunge into my dissertation (this is an
excerpt):
Many scientists, particularly social scientists, have already come to
realize that a perfectly objective, measurable, predictable world is not
really an option. That means that all research is a mixture of subjective,
caring involvement with the people being studied and a honing of the
researcher's capacity to be appropriately detached, analytical, and
reflexive.
The feminist perspective, a form of critical theory, contributes a
particular and important voice to qualitative research. It emphasizes the
point that all research is subjective, that we socially construct our
worlds, and this does not indicate a lack of rigor or epistemological
validity to the work. What has come to the fore because of feminist inquiry
is that most knowledge has been generated and defined by white males and
that, although they espouse an important perspective it is not the only one,
and perhaps not always appropriate. "Feminists have challenged not only the
view of the way in which knowledge is produced but also whose view of the
world it represents." As Chisholm (1990)11 puts it,
"the collision between theory and praxis [abstracted reflection on practice]
is as emotionally significant as it is intellectually interesting" -- there
is a need to look coolly and passionately" (Robson, 1993, p. 65). 12
Reliability and Validity
Although qualitative research must contend with the issues of reliability
and validity, there are strong voices suggesting that these criteria are
neither directly applicable nor congruent to qualitative research, that they
are positivist notions that are not transferable to qualitative, postmodern
methodologies. As I mentioned earlier, Guba and Lincoln 13 have
suggested that alternate terms, "Terms such as credibility, transferability,
dependability and conformability replace the usual positivist criteria of
internal and external validity, reliability and objectivity." (p. 14)
It seems that in qualitative research we go to the participants
themselves and our own souls to verify, rather than validate, whether or not
we captured what was important and useful. It is a chorus of voices that
determine the standards and verification required for a study to be
authentic and done with the "right" intention and processes. Guba and
Lincoln 14 bring some
compelling thoughts to the discourse regarding quality issues based on their
long and extensive research into the area of emerging criteria in
qualitative research: "establishing the criteria of "fairness'(a balance of
stakeholder views), sharing knowledge, and fostering social action." The new
emerging approach to quality is based on three new commitments: to emergent
relations with respondents, to a set of stances, and to a vision of research
that enables and promotes justice. (Creswell, 1998) 15
Dina Boogaard
Date: Sun, 14 Jun 1998
From: Pam Swepson
<p.swepson@uq.net.au>
Dear Judith, Dina and John
I certainly have some sympathies for the idea of "resonance." So, for those
who have thought about it more than I, may I ask, how can you distinguish
between the ahha! of insight from pain old kidding ourselves to maintain old
prejudices?
Pam
Date: Sat, 13 Jun 1998
From: Judith Newman
<jnewman@hfx.andara.com>
Pam,
I guess what I attempt to do in my own work is focus on the problematic --
the stuff that pops up which I don't understand. And via the relating of what
I call "critical
incidents" I attempt to identify the gap between what I think I believe
and what my actions are likely conveying in these situations.
The process as it has evolved for me, and the teachers I have been working
with, is focused on exposing those "old prejudices" and trying to reframe the
problematic of our working lives in new ways. In other words, the process is
aimed specifically at naming prejudices and trying to gain a new perspective
on what might be going on.
For example, a year or so ago I was working with a group of teachers,
trying to help them deal with "change" initiatives they were facing. I have
long thought that if I could enable folks to talk about what was problematic
in their work they were more likely to find new ways of working. What I
discovered when I was reading their reflections was that it wasn't a lack of
knowledge about current educational issues that was holding folks back from
changing how they worked in their classrooms -- they named a whole host of
situational factors which create barriers of which I tacitly aware, but the
teachers' writing made me stand back and rethink how I was engaging with them
because I could no longer ignore the political realities of their lives and
working situations. [See: "We Can't Get There From
Here."]
So, for me, action research sorts of inquiry are most suited to uncovering
our prejudices and framing new ways of understanding.
Judith
Date: Sun, 14 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com
>
Dina, I like what you've written here. I think it's important to understand
that we don't have to play in a pre-defined court -- we can and must create
the game as we go along. I don't believe we will have much luck convincing
died-in-the-wool technical / rationalists anyway. Seems to me our efforts are
better used for building this alternative way of viewing the world.
This discussion we've been having reminds me a lot of James Burke's TV
series "The Day the Universe Changed" 16 -- in each
episode he explored an instance where our perception of the "universe"
changed. He traces the small shifts of thought which led to a radical
reformulation of how we (western/European thought) understood how the heavens
worked, how we reconceptualized medical practice, etc. His underlying message
is that while the physical reality remained unchanged, how we thought about it
was forever altered. I think we're embroiled in that kind of shift with regard
to research as an activity. Interesting times to be living in.
Judith
Date: Mon, 15 Jun 1998
From: Pam Swepson
<p.swepson@uq.net.au>
Dear Judith,
You wrote, in part:
- > So, for me, action research sorts of inquiry are most suited to
uncovering
> our prejudices and framing new ways of understanding.
Thanks for your story and explanation. Could you further explain, please,
how you think action research does help uncover prejudices. It would seem to
me that processes for involving divergent opinions (both of my own and of my
"group") is the critical factor here. It also seems to me that the literature
on AR is deficient in methodologies for this sort of critical
participation.
Look forward to your comments
Pam
Date: Mon, 15 Jun 1998
From: Jack Whitehead
<edsajw@bath.ac.uk>
Hi Judith -- I do like the idea that you are not trying to change anyone's
mind but sharing a journey you have taken. That's the spirit in which I
respond to your work.
You also said,
> I simply stopped playing the academic game about 15 years ago.
> I decided that the kind of work involving teachers that I felt I
> needed to do teachers needed a different character -- one that the
academy
> didn't understand or support.
My own journey has been focused on contributing to the transformation of
what counts as educational knowledge/theory in the academy so that teachers
could gain accreditation for their professional knowledge in a way which
doesn't distort their own educational theories.
One of the questions I have is that if action researchers abandon concepts
such as "validity" will this be contributing to the process of "balkanisation"
described by Donmoyer in Educational Researcher? I am wondering if you see
what you are doing in terms of "paradigms" or whether this concept isn't
significant in your enquiry.
Warm regards
Jack
Date: Mon, 15 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com
>
- Pam Swepson wrote:
- > Thanks for your story and explanation. Could you further explain,
> please, how you think action research does help uncover prejudices.
It
> would seem to me that processes for involving divergent opinions
(both
> of my own and of my -group-) is the critical factor here. It also
seems
> to me that the literature on AR is deficient in methodologies for
this
> sort of critical participation.
Pam, I think the way I've come to sort out this business of uncovering
prejudices (I talk about assumptions) seems to be via critical incidents which
serve as the evidence on which the research is based. In my experience, the
bulk of the incidents seem to focus on situations in which my assumptions are
brought into focus, are highlighted in some way or other.
A poem as example:
- Mirror 17
- Chaos confronted me
- as I entered the hall:
- kids running around
- pulling stuff from one another's lockers.
- I was standing there
- hands on hips
- when twelve year-old Sally
- sidled up
- leaned close
- and said,
- "Pretty rowdy, aren't they?"
- "Yes, I concurred, "like wild Indians."
- She leaned closer --
- "I'm part Indian, you know," she confided.
- A friendly overture
- reflecting an image
- I was shocked to see.
The moments I seem to capture all have this kind of flavour -- as a rule,
they make what's transparent visible in ways that allow me to clearly see
contradictions between belief and action, hidden assumptions, unexpected
prejudices.
Judith
Date: Tue, 16 Jun 1998
From: Bob Dick
<bdick@scu.edu.au>
Pam said, in part, about surfacing prejudices ...
-
> It also seems to me that the literature on AR is deficient in
-
> methodologies for this sort of critical participation.
For me, part of the appeal of Chris Argyris' 18 work is that
he provides models and processes for doing so.
Bob
Date: Tue, 16 Jun 1998
From: Pam Swepson
<p.swepson@uq.net.au>
Judith
Nice story, nice poem. I am still feeling unsatisfied but can't spell out
my concerns any further at the moment. I think it is something about "groups"
having the same experience that you have as an individual. I will try to keep
thinking to clarify my worry.
Pam
Date: Tue, 16 Jun 1998
From: Judith Newman <jnewman@hfx.andara.com
>
Pam, somehow I don't think "groups" have "experiences" -- individuals do,
and my hunch is that even when in a "group" the experience for each individual
is unique, dependent on his / her history, values, beliefs, etc. And I guess
what I look for from others is some element of common experience, common
values which allow me to sense community and at the same time offers me
difference in perspective which keeps me refining my interpretations of what's
going on. -- Like what's happening with this ACTlist conversation.
Judith
Date: Tue, 16 Jun 1998
From: Jack Whitehead
<edsajw@bath.ac.uk>
-
> Twenty years ago I abandoned positivist ways of viewing the
> world. Working with very young children making sense of language,
> oral and written, quickly moved me into a different theoretical
> place. I can't see going back.
Judith -- I can recall a similar experience in 1971 when I abandoned my
positivist approach to educational research because it did not permit me to
understand my educative influence with my pupils. When you say you moved to a
different theoretical place, I am trying to understand your view of
educational theory and educational theorising and to inquire if your
"theoretical place" can exist without the concepts of validity, knowledge and
explanation.
Warm regards
Jack
For some reason, I don't now recall, I didn't reply to Jack. The
conversation on ACTlist moved on and people began discussing other concerns.
But Jack's question affords me the opening to put this written discussion into
some kind of perspective for myself and for anyone else interested in such
arcane issues as "validity" in action research.
Jack asked me whether my "theoretical place" can exist without the concepts
of validity, knowledge and explanation. It's not that I'm not concerned with
what connections my perception of the world of schools and classrooms has with
some "real" world situations -- I try to offer as detailed descriptions of the
situations and the participants as I can so that readers may judge the
"reality" for themselves -- but I am also aware that whatever I write is an
interpretation, my interpretation, of what I've experienced or what the
others, whose stories I include in my writing, have experienced. Even if I
were to have used quantitative data, the written account would still my
interpretation of what those data might mean.
As I see it, there is simply no way to step outside of interpretation. So a
notion, like "validity" doesn't seem to me to contribute in any useful way to
my work as an action researcher. I am careful to situate events, moments,
critical incidents as fully as I am able but in the end I realize that, no
matter what I write, it is, in some sense, a fiction. Just as a more
traditional research account is a fiction -- there are no "average" students,
no "typical" teacher, no "representative" classroom; students, teachers and
classrooms are unique. The most any research account can afford is a partial
snapshot that may or may not offer insight about the daily concerns which
students and teachers face.
As for the construct "knowledge" and the role of "explanation" -- we come
back to a researchers' view of the world. As a "constructivist" I accept, as I
wrote Bob Dick, that there is a physical reality "out there" but my senses
filter all incoming data and interpret them based on my prior experience, on
my theories of the world that I've built up over a lifetime of living. How,
for example do I "see" 13 -- in the context 13all I
interpret letters, in 7134 I perceive numerals. There are marks on the
page, or lit pixels on a screen, but what sense I make of sensory data depends
on context and my interpretation of it.
I believe I have a responsibility to provide a well-documented account of
the particular experiences I choose to share. However, I am aware that my
"knowledge" is a personal construction, constantly being shaped by both
personal first-hand experience and by the conversations I have with others. I
am influenced by how others interpret the world and my understanding of
situations is certainly affected by accounts that I read and by any written or
face-to-face discussions in which I participate. As an action researcher I
struggle to maintain an "open" mind -- to allow myself to be surprised by
situations and to use that surprise to examine my unexamined assumptions. For
me, it's that continuously making visible of the taken-for-granted that
provides the rational for engaging in inquiry into my professional work.
Needless to say the conversation doesn't end here. The other day I received
email from Pat D'Arcy (Jack Whitehead had passed on to her an early version of
this written conversation I'd sent him) in which she added her voice.
Date: Mon, 26 Oct 1998
From: Pat DíArcy
<edppmda@bath.ac.uk>
Dear Judith,
I've been sitting here on a wild, wet and windy autumn afternoon reading
the batch of e-mails to you and from you that arrived in a package from Jack
[Whitehead] this morning and they have set so many thoughts racing through my
head that I just wanted to join in the conversation -- even if somewhat
belatedly!
So let me just plunge in! Validity... to me, if action research has any
validity then it is to show that what the researcher set out to change is
indeed changeable by providing demonstrable evidence that changes in actual
practice have indeed occurred. To take my research as the example (not
surprisingly) that comes most readily to mind. I set out to offer guidelines
for myself and for the teachers I was working with based on Rosenblatt's
aesthetically transactional approach 19 to reading
stories which would enable us to respond to pupils' stories in a way that was
both personally meaningful to ourselves and subsequently to the children who
had written the stories in the first place.
I was able to compare the responses that we made (in writing) to their
stories with the skill-based forms of response currently in use in our
National Tests to show how much those assessments were missing in terms of the
pupils' achievements through their highly creative constructs as
meaning-making individuals. To me, my research was valid because it provided
the evidence for these differences of what we 'saw' depended on how we looked,
which could then lead me to make a stronger case for more interpretive forms
of assessment which were both valid (the term that Louise Rosenblatt 20 also uses) and
warrantable (the term that Fran Claggett 21 uses in her
book "A Measure of Success")....
Pat
Date: Wed, 28 Oct 1998
From: Judith Newman
<jnewman@hfx.andara.com>
Pat,
Pat, thanks for jumping into this conversation. Your two cents at this
point are useful.
- > ... to me, if action research has any validity then it is to show
that what
- > the researcher set out to change is indeed changeable by providing
- > demonstrable evidence that changes in actual practice have indeed
- > occurred....
The sort of inquiry I find myself engaging in doesn't necessarily set out
with change goals worked out. I don't begin with anything specific that I want
to change -- as a teacher/teacher educator, I do have a broad intention of
helping teachers examine their assumptions and practices in order to
understand better the connection and the impact of their instructional
decisions on students. I do want them to examine prevailing paradigms in order
to understand the beliefs underlying them. But as for specific changes, I
can't say in advance what those might be. Nor can the teachers I work with.
For example, one teacher who is currently writing a Master of Education thesis
thought she was exploring how her grade one students learned to read, only to
discover that her piece really is about how her instructional decisions are
affected by outside pressures. She found that out from an analysis of her
daily writing about what was happening in her classroom. She didn't know to
what extent parents' beliefs, administrative directives, school district
policy, provincial government control affected her moment-to-moment classroom
decisions until she began the actual writing of the thesis. So the particular
criterion you set out for deciding whether the work has "validity" doesn't
apply in this situation. I'm not sure what sorts of criteria would, other than
the internal consistency and richness of the narratives, the thoroughness of
Claire's analysis, her connection with the research literature. Ultimately,
anyone reading her work will have to make some judgement about whether her
discussion raises any questions that are relevant or interesting for him or
her.
- > To me, my research was valid because it provided the evidence for
these
- > differences of what we 'saw' depending on how we looked...
I can see how, in a situation where you have a particular goal in mind as
you did, that demonstrating a difference between one way of engaging in
instruction or assessment and another (and examining the paradigmatic beliefs
which shape those different implementations) would lead to an argument in
support of one set of instructional or assessment practices rather than
another. However, I'm not sure why you need to invoke "validity" as a
construct to be able to say there's something interesting and perhaps useful
in that comparison.
My work seems to shape itself differently -- I find myself beginning in the
middle of whatever is going on and suddenly encountering something
problematic. I know before I begin a new course that I'll stumble across
something that I haven't thought about before but I have no idea before I
begin what that might be. So I find myself writing about issues of power and
control, or problems I face as a "change agent" in helping teachers actually
do something different in their classrooms, or I come face to face with the
pressures that are affecting them and limiting what they're willing to try.
But I only discover that when I stand back from the critical incidents I've
been collecting and look for issues which they allow me now to see.
It seems to me you have a responsibility as researcher / writer to provide
me, your reader, with as much evidence as you can (descriptive, narrative,
quantitative, whatever) so that I can understand your biases and then be able
to make a judgement about whether your arguments offer me useful things to
think about myself. I'm not interested in "what tips does this research offer
me" as many of the teachers I work with would be. I want your research account
to show me what impact your research had on you own theoretical development. I
want to know how the questions you were asking changed, how you now understand
better the political pressures which impact on your decision making, and so
on. The more clearly the "journey" you've taken is portrayed, the more useful
is your account for allowing me to think about my own situation and the
problematic I encounter.
Judith
Date: Thu, 29 Oct 1998
From: Jack Whitehead
<edsajw@bath.ac.uk>
I do feel that I'm coming to understand your views more fully. We do appear
to be understanding similar meanings using different words. I wonder if you
would expand a bit on the meanings you give to 'theory' and 'theoretical
development' when you say at the end of the paper:
-
> "I want your research account to show me what impact your research
> had on your own theoretical development".
What I'm really interested in is what, for you, characterises 'theory' or
'theoretical development' in the narratives/interpretations, which are
constructed by educational action researchers.
Warm regards
Jack.
Date: Tue, 03 Nov 1998
From: Judith Newman
<jnewman@hfx.andara.com>
Jack,
My first encounter with "ethnography" in education was through Marjorie
Siegel's PhD thesis. 22 Margie was
struggling with the situation in the classroom where she was
observing/participating and with her role in the research. One of her
advisors, Bill Corsaro, said to her " What you see as problems, I see as
data." Margie's thesis became an account of her changing assumptions as she
worked through problems.
Her experience shaped my subsequent work because I realized that my
responsibility as researcher was to make what was happening to me as a result
of my experience open for others to see. I stopped being concerned about
whether my experience could be generalized, (Schon 23 would argue it
was unique, anyway) and worked more at shaping the narrative and argument so
that others might use it to examine their own work more critically.
To respond to your question, then, Jack. The "theory" in my work can be
represented by the intersection set of "my experience", "what I've read that
allows me to take an interpretative stance on that experience" and "why I
think it's significant for my work; how I see my work differently as a result
of having explore some new contradictions." I've drawn a Venn diagram for you
to show how I see the interplay among these elements.
So, theory, for me is that constantly changing "space" where experience,
interpretation, and research discourse interact / intersect. "Theoretical
development" occurs during the construction of the narratives -- the struggle
to decide which critical incidents contribute to some bigger picture, how to
cast them, what issues do I now understand that I didn't before, my attempt to
relate how I see this inquiry affecting my work, etc. all are part of my
ongoing theoretical development.
Does any of this help?
Judith
And there, for now, I end my musings. At the end of this written
conversation I understand in a new way the relationship between my views about
the construction of knowledge and the research methodologies I (and the
teachers I've worked with) have been evolving over the past fifteen years.
During that time we've learned how to record our experiences and the insights
they promote, to concatenate events so that a story emerges and the issues
which surface are discussed explicitly. Our objective is to write in such a
way that others may use our journeys to help them "read themselves."
NOTE :
Email addresses may not be accurate; several have
changed. I've left them in the excerpts to reflect the international character
of the conversation. Return
1. The conference
description and the papers which served as the basis for the month-long
on-line discussion can be found at http://users.andara.com/~jnewman/Papers.html
Return
1
2. Fish, Stanldy 1980 "Is
there a text in this class?" In: Is There a Text in this Class?
Cambridge MA: Harvard University Press: 303-321. Return
2
3. Schön, Donald 1983
The Reflective Practitioner: How Professionals Think in Action. New
York: Basic Books, Inc., Publishers. Return
3
4. Spencer Margaret
(Meek), 1987 Text in Hand: Explorations in the Networking of Literacy and
Literature or New Literacies, New Texts, Old Teachers. Paper presented at the
5th Invitation Riverina Literacy Centre Conference, Wagga Wagga, NSW 20-22
August, 1987. Return
4
5. Rosenblatt, Louise 1978
The Reader, the Text, the Poem: The Transactional Theory of the Literary
Work. Carbondale, IL: Southern Illinous University Press. Return
5
6. Schön, Donald 1983
op cit. Return
6
7. Rosenblatt, Louise 1978
op cit. Return
7
8. Perl, Sondra 1983
Understanding Composing. In: J.N. Hayes et al (Eds) The Writer's Mind:
Writing as Mode of Thinking. Ubrana, IL: NCTE: 43-51. Return
8
9. Lessing, Doris 1980
The Marriages Between Zones Three, Four, and Five (as narrated by the
Chroniclers of Zone Three). London: Granada. Return
9
10 . Swets, J.A., W. P.
Tanner, jr., & T. G. Birdsall 1961 Decision processes in perception.
Psychological Review, 68: 301-320. Smith, Frank 1971
Understanding Reading: A Psycholinguistic Analysis of Reading and
Learning to Read. New York: Holt, Rinehard & Winston: 23-26. Return
10
11. Cited in Robson,
Colin 1993 below. Return
11
12. Robson, Colin 1993
Real World Research: A Resource for Social Scientists and
Practitioner-Researchers. United Kingdon: Blackwell Publishers, Ltd. Return
12
13. Guba, Egon G. &
Yvonna S. Lincoln 1994 Competing Paradigms in Qualitative Research. In:
Denzin, Norman & Yvonna Lincoln (Eds) Handbook of Qualitative
Research. Thousand Oaks, CA: Sage Publications, Inc. Return
13
14. Guba, Egon G. &
Yvonna Lincoln 1994 op cit. Return
14
15. Creswell, John W.
1998 Qualitative Inquiry and Research Design: Choosing Among Five
Traditions. Thousand Oaks, CA: Sage Publications, Inc. Return
15
16. Burke, James 1985
The Day the Universe Changed. London: British Broadcasting Corporation.
Return
16
17. Newman, Judith M.
1998 Sabbatical In: Tensions of Teaching. New York: Teachers' College
Press, pp. 176-177. Return
17
18. Argyris, Chris 1976
Increasing Leadership Effectiveness. New York: Wiley & Sons. Return
18
19. Rosenblatt, Louise
1985 The Transactional Theory of the Literary Work. In: Charles Cooper (Ed)
Researching Response to Literature and the Teaching of English. Norwood
NJ: Ablex Pub. Corp. Return
19
20. Rosenblatt, Louise
1985 op cit. Return
20
21. Claggett, Fran 1995
A Measure of Success. Portsmouth, NH: Boynton/Cook Publishers. Return
21
22. Siegel, Marjorie 1982
Reading as Signification. Bloomington Indiana: University of Indiana.
Unpublished Doctoral Thesis. Return
22
23. Schön, Donald op
cit. Return
23
Judith M. Newman is a writer and education
consultant. Visit her teacher and action research web site Educating
as Inquiry or contact by e-mail
| 
