Home ›
After our fourth session
Posted 4 years ago, 10 comments
We're continuing our discussion about ethnography on Thursday - digging into the issues of validity and reliability.
Please post your questions/comments/reactions to the readings by 3pm on Thursday.
The readings are available on the Schedule page.
- Login to post comments
I found the description of methodologies very useful, and in general each method adds a type of systematic approach that itself establishes the credibility. One thing that struck me with the Creswell paper is how it is basically mentions that the presented frameworks "helps researchers select procedures based on who assesses the credibility" of the research. I know that this thinking exist in other quantitative, but I don't think it is made so explicit.
I also noticed that with the Member Checking method and the Collaboration methods differ greatly with typical quantitative methods that would not include the participants in the analysis of information.
One question is, does adding more methods make the paper more credible.
LECOMPTE
this is such a good summary of many aspects of ethnography that we've previously discussed in class:
"Ethnographic research differs from positivistic research, and its contributions to scientific progress lie in such
differences. These may involve the data gathering that necessarily precedes hypothesis formulation and revision or may focus on descriptive investigation and analysis. By admitting into the research frame the subjective experiences of both
participants and investigator, ethnography may provide a depth of understanding lacking in other approaches to
investigation." (page 32)
can you ever have external reliability? i think you can have it on some big more obvious issues some of the time, but less
often on smaller details, and still it depends on the ethnographer.
this paper seems to want to try to validate ethnography by comparing it to positivistic methods. Is that necessary?
verify vs discover?
"In a sense, experimental researchers hope to find data to match a theory; ethnographers hope to find a theory that explains their data." (page 34)
really? what sense of "systematically"?
"Estimating the internal validity of a relationship is a deductive process in which the investigator has to systematically think through how each [factor] ...may have influenced the data .... In all of this process, the researcher has to be his or her own best critic, trenchantly examining all of the threats he or she can imagine" (page 50)
CRESWELL
a little dry
I like thick descriptions as a way of developing credibility. I think there's a lot in trusting the author that's not being addressed. I judge credibility by knowing people's history. If I don't know an author, I judge the credibility of what they write based on whether some of what they say enlightens me and other of what they say resonates with what I know to be true. That is my test.
I was struck by the note in the lecompte about the relationship btw method and theory:
"In a sense, experimental researchers hope to find data to match a theory; ethnographers hope to find a theory that explains their data." A footnote backs off from this generalization and talks about it as more continuous. I wonder why this more complex view was left for a footnote, and the oversimplified generalization left in the main body of the paper.
I think at least part of the problem is differing definitions of "theory" between methods, but exactly how do they differ?
This struck me too. Of course both researchers do both, but there is kind of a main purpose: to verify vs to discover, each of which has a truckload of implications. But I do wonder about it in the same way you do.
LeCompte and Goetz
I expected the positivists to endorse camera and video use but on pg. 43 they find"they are an abstraction and yet they may preserve too much data."
The suggestion that ethnographers temporarily withdraw from the field in order to defamiliarize themselves from the social scene dovetailed nicely with last week's Bell article. But I wonder if this is a practical suggestion? Perhaps researchers sent to far away places might ill afford (economically) to keep going away and returning to their field site?
Creswell and Miller -
"The purposes of a _______ is that it creates versimilitude, statements that produce for the readers the feeling that they have experienced, or could experience, the events being described...." p. 128-129.
I think the word 'novel' could easily fit in the blank, but the original contents are 'thick description.' I can see Stephanie's point that this technique could easily be misused, just as quantitative data can be misused. But I still very much like this concept. I believe it has value as a means for encoding and validating subjective experience, which of course can never be transmitted perfectly. But I think in most cases 'good enough' transmission will do. At the risk of sounding even squishier, I'll try to give a brief example.
I once had a client who reported that he'd been raised in a motorcycle gang and abused in all sorts of ways, trained as a Navy Seal and ordered to build explosives from scratch and plant them on oil tankers, and hinted that he'd killed some guys to avenge the murder of a relative (and never been apprehended or prosecuted). There were many contradictions as you might imagine, and based on these and some intuition it was clear to me that his stories were never completely or objectively true.
I was tempted to disregard a lot of what he said. But in sticking with him over the course of treatment I began to recognize that the facts he reported were sometimes better understood as communications about his subjective experience. From being in the room and monitoring my own subjective responses to him, I began to recognize that he brought up scary or violent parts of his story during times when he felt threatened. This often occured after he'd shared something genuine or emotional with me and, like many adult survivors of abuse, was afraid that he had let himself be too vulnerable.
This is a subtle kind of communication that took me many sessions to recognize. And while his statemeents were not all factual, the subtext behind them that indicated his experience in the moment generally was. As we worked together he became somewhat better at identifying and describing this experience more directly, as in "I'm feeling kind of pissed off and worried I'm gonna get burnt by you," instead of "Hey let me tell you about the time that I got shot in a gunfight (so you know I'm tough and not someone to be messed with.)" So in short - even if all the details aren't "true", there's generally always something in the subjective communication that they are pointing to. The process of uncovering this requires a researcher (or counselor) who can be aware of and describe their own experience and its relationship to the stimulus coming from the subject.
So I think techniques like the ones described in Creswell and Miller, especially thick description, are necessary and useful when transmitting information about subjective experience. Thick description could never be enough to establish validity alone - but when mixed with many other kinds of validation, as the authors prescribe, it can contribute to it.
Creswell:
Interesting that they consider prolonged exposure in the field a plus for validity. I understand that the longer a researcher studies and is exposed to a group, the more perspectives she'll be exposed to and the more rich her understanding of participants, but it seems that there might be concerns that prolonged exposure might lead to indoctrination.
Thick, rich description: I can't help thinking this can be misused like data. Stress what you want readers to pay attention to so they'll overlook the sparser accounting of other, weaker items in your study. I'm not sure researchers set out to do this. It just seems it could so easily be abused that it's interesting it's presented as a credibility test.
Creswell et al:
Lecomte et al:
Questions from group discussing Bell et al are below. Group members: Selene, Stephanie, Yannick, Sajid.
Disclaimer: these are my representations of the issues discussed.
general discussion (how/why)
how can defamiliarization be used to aid design?
how do you reflect on the overly familiar?
how do you reflect about the home, when you cannot easily elicit reflection?
how do you challenge yourself about the status quo?
what's the goal? what should be the goal (for the designer)?
ethnography as a discovery tool (rather than a find an answer)
discover what's needed rather than what the results of the intervention is - "uncover the unexpected"
discover an issue that wasn't an issue for the researcher
design opportunity vs. design guideline (and unexpected design opportunities)
what is the difference between opportunity and guideline?
when can we feel comfortable in being prescriptive?
why is ethnography being used?
ethnography as a technique for describing across cultures
exploit contrast?
how do you prototype/stereotype?
surveys -- wouldn't have worked
highlight particularities
provide contrast for interesting facets
how was it used?
this wasn't about generalizations
how can/should ethnographic writing be used to promote/invoke critique?
side discussion: when can you make conclusions?
is familiarization the self check tool?
what do you do with the data? how do you make conclusions
when is an example sufficiently generalizable?
side discussion: defamiliarization and scientificness
defamiliarization as a scientific method: why is it not scientific?
isn't defamiliarization a tool for scientifically pulling apart things so that you can see variables?
how well can you isolate defamiliarization
techniques used:
observations
interviews
this paper is not too clear on the specific techniques, but rather focuses on the exemplars from those
some of it can be inferred...