Qualitative Research for Public Health and Clinical Investigation

[MUSIC PLAYING] Hello, everyone, and
thank you for inviting me. So in looking at the
schedule and in listening to this last
presentation, I hope everybody’s ready
for a change of pace, because we won’t be
talking about any numbers, but we will be
talking about ways to do a different kind
of rigorous research. And I’m happy to take some
questions during this, but I have to say probably
not a lot of questions– only because I was
asked to do kind of a soup-to-nuts-presentation
in an hour. And so I’ll try to move through. And you’ll notice that I’m not
able to give a lot of detail on any particular
type of method, but I hope that what
I can do is give you enough detail and enough
of an awareness raising of what kind of
qualitative methods are commonly used in clinical
research and public health research, and how you might
use them in your research. So my objectives are
basically to give you that kind of overview. And I’ll also be giving small
examples as we go along, and finishing up at the end
with a few examples of studies that I’ve done that show
you how I’ve integrated the different kinds of methods. So the first thing
to really think about is why I conduct
qualitative research. Years ago, when I
first started working with physicians and with
public health professionals, qualitative research was
very, very strange to people. It wasn’t used. The idea was to maybe
throw in one focus group– and I’ll talk a little bit
later about focus groups– but it was very unknown. And now, you’ve probably
noticed that even if it’s not your primary type of a research
method that you want to use, or that the researchers you
are involved with want to use, you’ll notice that people
are more and more actually involving qualitative
research at different stages. It really can be a
terrific addition because what you get
from qualitative research is meaning and context about the
topic that you’re looking at, in a way that’s very,
very different thing you can get from other methods. And that’s really what I
want to talk about today. I’m not ever going to say
that qualitative research is better than quantitative
or vice versa. It’s different ways of knowing. You get different kinds of data
and when you put them together, you have a very,
very strong project. So one important qualitative
anthropologist, James Bradley, said a long time ago, “I
want to understand the world from your point of view. I want to know what you know
in the way that you know it. I want to understand the
meaning of your experience– to walk in your
shoes, to feel things as you feel them, to explain
things as you explain them. Will you become my teacher
and help me understand?” And basically that’s what
qualitative researchers are saying to the participants
in their studies– that we want to know
what you are thinking and you are doing
from your perspective. So the nature of
qualitative research is very different from
quantitative research. And all through the years that
I’ve been collaborating people, a nasty rumor keeps popping up,
and that is that qualitative and quantitative methods are
so antithetical to each other, that they cannot
be used together, or they cannot be used in
the same research design. And I hope that by the
end of today’s talk, you might realize,
well maybe that was just a rumor after all. And done well, it can
be very, very helpful to put the things together. What you do is– in
qualitative research– it attempts to make sense
of the social world, in terms of the meanings
people bring to it. So when I say social world, it
can be as broad or as narrow as makes sense for your setting. So it could be what is going
on in this particular ICU. What’s going on in this
broader immigrant community? What’s happening in all
of Boston in relation to a particular topic? Whatever it is that your
research study is about, that’s what your
social world is, and the interactions of people
that are going on within there is what you’re looking at
with qualitative research. So you want it to uncover ideas
inside, and ways of thinking about phenomena of
which little is known. So before you do any kind
of quantitative research, you might want to do
some qualitative in order to understand what
the concepts are from the perspective
of the people who will be in your study. Also you can get a new
way of looking and getting a different perspective
on something that quite a bit
is known, but maybe not from every perspective
that’s out there. And again, you can contextualize
your understanding of an issue. And that word– I’m going
to keep coming back to it– because that’s what I think
qualitative research is so good at, is giving
you the social context of what you’re looking at. So Howard Stein, who is
a medical anthropologist in a family medicine
department in Oklahoma said, “We are all natives–
insiders to some groups and outsiders to others, even as
we aspire also to be scientists and transcend the tribal.” And what he’s saying, is
actually very, very important to probably most people in this
room, because historically, when anthropologists did
qualitative research we went far away. We went to places we
weren’t familiar with, and we brought our own
ways of thinking to places that were very strange to us. But many people who are
doing health research, might be studying phenomenon
in the same hospital in which they’re employed. So in some ways, you’re
wearing different hats and you bring your own thinking. You bring your own
tribal way of thinking, but you have to
transcend that in order to also look at what’s
going on with people who think differently about
the same environment that I’m in every day. And if you don’t do
that, then you’re not really doing the
research because you’re just thinking your own thoughts. So wearing those double hats
is important to think about, who are you within
the whole thing? So there’s different
ways of looking at it. In anthropology, we
have this concept– this dichotomous concept
of emic and etic. It comes from linguistics,
but it’s very, very useful in thinking about
whose perspective are you collecting the data from. So emic is really considering
the insider’s perspective. So if you interview a
person about something and she tells you what her
thoughts are about that, that’s her emic perspective. If you watch or doing
something but nobody is explaining it
to you and you’re using your own scientific
concepts that come from theory or what you know from the
literature– or any other areas that you might
have collected ways of thinking about
something– that’s the etic. So let me give you an example. So you’re interviewing your
patients about what they think about taking an
anticoagulant, for instance– including such topics as reasons
they think they’re prescribed the medication, how they
think the medication works, whether they think
they should take it, what they think would happen if
they don’t take it or if they take it differently than
you’ve prescribed it, what labels they use to refer to
the medication and what it does to them, how much
they should take, how they know how
much they should take, when they should take it, what
dietary considerations they think they might need to
make while they’re taking it, and– not the same thing– what dietary habits they
say that they are actually practicing, why
they think they’re told to have INR testing,
what the INR test really mean to them, and
you can see I could go on and on and on
and on in this topic. But all of these
questions get at what that patient is thinking. So sometimes it’s
considered subjective because it’s not an
outside observer, it’s subjective thoughts
of those patients. So in contrast, you could
look at the etic outsider’s perspective. And in this case,
on the same topic you would ask questions–
you wouldn’t ask questions, you would be observing
the patient’s behavior. If let’s say somehow
you could put yourself in a position to observe the
patients’ medicine taking behaviors, and
you would document what they do in terms of how
much they take, when they take it, how they in reality manage
their diet, which could be very, very different from what
they told you they’re eating– how adherent they are
to the INR testing, what the results of the tests are. All of these might seem to
be more objective results. And so then what you
do is you interpret what’s going on with the
patient from what you think you know about what you see. So the way I’m wording
this, you might be able to see
that there could be some problems inherent
in both, like how do you know which is reality? How do you know which is true? Your qualitative data is only as
good, in some ways, as the data that people give you. But on the other
hand, what you look at is only as real as
how you interpret what you see people are doing. So often when you
watch people do things, you might want to know
why are they doing that? How often do they do that? What was she thinking when
she decided this was the way to take the medicine? You can’t tell that
from observing, but if you only asked
her, you don’t really know what she’s doing. So that’s why triangulation
of research methods is so important. And when you’ve heard the
concept of mixed method studies, actually
qualitative research has historically
been mixed method because one method alone
only gets you so far. You need different ways of
knowing and putting them together. So these days, people put
together qualitative methods and then they also put
together qualitative methods with quantitative methods. And that’s where the power
is– putting all those methods together. So I won’t go very
much into this because your next
lecture will be about community-based
participatory research. But one of the things that’s
so important for qualitative research is that if you’re
doing population-based research or community-based research,
that you are involved in the community
and the community is involved in
what you’re doing, which also was something that
was very, very unusual when I first came into public health
research in the early ’90s, and now is pretty common. But it’s easy to forget
a lot of the steps. So I won’t go into
the details, but these are sort of the overall
guidelines that I like people to think about. And when I say get
involved in the community– all aspects of
project management should be out in the
community for people to see who’s involved. And involve people early on. I will say I remember very
early in the ’90s going to a regional conference about
health disparities, in fact, in Boston. And somebody stood up from one
of the Latino health institutes and said, the thing that’s
really upsetting to me is that the Latino community
is brought into the research only at the stage where
recruitment is necessary. Can you get me enough people
for five focus groups? And it was insulting. It was insulting and it also
was detrimental to the research. Now people have gone a
lot further than that, and people are
much more cognizant of the need to involve
community at early aspects. But also, it’s not easy to do. You need to plan for it,
and have time for it, and spend time in the
community, and do it well. So if you do though,
the study will be more relevant to community
needs, more feasible due to community
buy-in and community acceptance of what
you’re doing, and also the fact that community
will have had input so it makes more sense. –and still more likely
to achieve the desired sample because
you’ll probably get a lot of help with
recruitment, and more ethical, because the questions, and the
priorities, and the recruitment methods, and the
data collection, and all the way through to the
analysis and the presentation will be much more respectful. So I needed to put that plug
in, even though I’m not giving an entire talk on that topic. So qualitative research
approach– Sometimes what people try to do is
use qualitative methods but retain a quantitative
approach to their thinking about research and their
design of research, and their design of samples. So you don’t want to do that. You want to think about what
is different about qualitative research, and what is
necessary to do really good, rigorous
qualitative research. And so what you can find
out in qualitative research is the range of phenomena
in the sample study. You’re not going to
get representation. You’re not going
to get p-values. You’re not going to
use random samples. You are going to do what makes
sense for qualitative research. And s,o often– I can’t
tell you how often– the first review of fully
qualitative papers that I submit, come back saying,
I bet you have numbers. I know you have numbers. I can sense you have numbers. Why didn’t you
report the numbers? And I actually have developed
a pretty successful paragraph that I send back to
the reviewers saying that I respectfully
actually refuse to provide you with numbers,
because it doesn’t make any sense to provide
numbers, because it’s not representative. I learned early on
in my collaborations with epidemiologists that
numbers on a small sample are not valid. It doesn’t make sense. So don’t do it. So don’t let anybody talk
you into providing numbers for your small-sample
qualitative research. What you can tell people
is a really rich story about the meaning,
and the context, and the why’s and
how’s of what’s going on regarding the
phenomenon under study. And so that’s what
you want to do. You don’t want to
apologize for not being able to do what
quantitative research can do. You want to showcase
what you have done with qualitative research,
which gives another perspective on the phenomenon,
that maybe you’ve also researched from a
quantitative perspective. You could put them together
in a mixed-methods study. So what you use is what we call
purposive, stratified samples, because you want your samples
to be really, really good. A random sample, I also
learned from my epidemiologist colleagues, that is
small, is not necessarily going to be represented. So what’s the point of doing it? But what you want,
is you want people who will talk to you about
the issue under study and who demonstrate the
diversity of the population that you’re looking at. So I did a study of
Dominicans and Puerto Ricans in Providence around
cancer prevention attitudes and beliefs. And it was 146 two-hour
home-based qualitative interviews. So instead of doing– that
was a pretty large sample and maybe, I don’t know,
I could have gotten away with a random sample of that. But what I wanted was, I wanted
people who had health insurance and people who
didn’t, people who were younger people who were
older people, who still live with children and people
who didn’t, people who are new immigrants and
people who weren’t, and on and on and on. So I had a stratified sample. And I had a big chart that
we were constantly filling in as we did the
recruitment, to make sure that I was hearing
from people who had the characteristics
that illustrated all the different
kinds of situations that Latinos or Dominicans and
Puerto Ricans in Providence had. And so you can see that a random
sample wouldn’t necessarily have gotten me that. But this was a very,
very strong, very, very clearly sought that
sample, but it wasn’t random. So you describe
it as what it is, as opposed to apologizing that
it’s not random because you didn’t want a random sample. And then as I said, it’s
generally multi-method. And the important thing
is that people often say to me, what are the
best methods to use? And then my first
question is, well, what’s your research question? Because it’s just like
with any kind of study you’re doing– the research
question determines your choice of methods. And it always does. Yeah? So quick question– whenever
you have not random, to me it sounds like possible bias. How do you get around that? Well, bias is a big issue. I have to put to
you the potential of bias occurring in
quantitative research as well. So I think the idea of
discussing bias only in qualitative research is an
erroneous approach in research. So if you’ll be willing
to go forth from here and consider a bias in
quantitative research for the rest of
your career also, then I think that we
can talk about bias in qualitative research. What you do is– and
I think it’s something that I can’t go into today in
detail, because we could have lectures and lectures on the
issue of looking for validity, and rigor, and credibility,
and all the kinds of quality indicators that you’re
looking for in research. They’re not identical. Some will say that
they are identical as quantitative research, and
some will say they’re not. And I think I gave
you two articles to read that cover two
different approaches to looking at how you get at quality. But in this case,
bias can happen when your sample is
very small and you don’t know how you are recruiting. So for instance, I did
a study of overweight in black women in Boston
in the early ’90s. And we did a lot
of focus groups. We did 30 focus groups and we
used community recruiters– people in the community
to help us recruit. So what we ended up with was
bias in every single focus group. But we did a lot
of focus groups. So when we had somebody who
belonged to a Pentecostal church do the recruiting,
the focus group ended up in that
Pentecostal church with members of that
Pentecostal church. So that’s an extraordinarily
biased focus group. But had we not had a
recruiter from that community, we probably wouldn’t
have had any participants from that community. So the fact that we
had that focus group, if that have been our only focus
group, or one of two or three focus groups, it would’ve been
a horribly biased sample– but that was one of 32. Then we had a focus group
that was recruited by somebody who had started a walking
group– who was overweight and started walking group. So most of the women in that
focus group were– everyone in the focus groups
had to be overweight– but most of the women that
focus group were really motivated to lose some weight. Really biased sample, but
that was the second one of 32. Then we had a focus
group where it turned out that women were telling
me that they had gained a lot of weight, and
about six months ago, six or eight months
ago, they decided it was time to
start losing weight. And after a few minutes
I finally asked, why? What happened six
or eight months ago? What happened six
or eight months ago is most of those women
had gotten out of jail. So the recruiter
had been in jail and she met most of these
women within that community. We never would have had
anybody in our focus groups, had I done the recruiting. And if we had only had that
focus group, imagine the bias and stereotypes. So what you do is– we knew that
we wanted to interview people from all walks of life, all
educational backgrounds, all economic backgrounds,
and we went to places and repeatedly interviewed
and interviewed interviewed. And what you do
is, is you do this until you get what’s
called data saturation. And when you start hearing
the same thing over and over and over again,
then you realize, seems like we’ve pretty much
cover the range of possibility here. So there is bias. But you have to
acknowledge the bias. You have to be extraordinarily
aware of A, your own biases. And there are some journals
now, like the Annals of Family Medicine,
where you must have a paragraph in the
beginning of your methods saying, who are you? So when I published
a cholesterol study, we had one doctor who
was American and Israeli and didn’t really believe in
cholesterol testing that much. And we had another
doctor who was gung-ho on cholesterol testing. And we had me who was
the anthropologist, who didn’t have an opinion
on cholesterol testing. And you have to do that. So I would say that it’s
there, but it’s everywhere. The difference is,
we’re called to task on a continuous basis
in qualitative research to make it transparent. And I think it’s a good thing. So I’m glad you
asked that question. When you get a lot of
different bias groups, you essentially are
randomizing in totality. In the end, when you look at the
whole spectrum of your results. it sounds like you approached
randomization in a way. In a way. I wouldn’t call it
randomization but– I was just going to
simply supplement what you just said about
qualitative research on the other end, quantitative
research [? and the gap ?] that quantitative
research– the investigator, just like in qualitative
research, defines the question. And who you are and how
you define a question can be totally bias. And in internal reports of
medical journals for example, many individuals will be
set [? in a basic ?] science or clinical randomized–
clinical trial perspective. So that’s the perspective. So the things that are
going to get published in certain journals are
going to be simply looking at those methodology. That’s bias, although
it’s quantitative. I’d like to make a brief comment
from a statistical perspective on this issue of bias– that
actually the way you’re using bias is a quantitative concept. You’re thinking that there’s
a population characteristic and you’re trying
to estimate it. And there’s a numerical bias. So I would describe
the distinction here is that– Dr. Goldman is
talking about characterizing the population in
a holistic way. She wants to understand
the range of views, the different perspectives. And it’s not so much, what
is the average perspective, or what’s the
typical perspective, as really characterizing
the community holistically. So of course, bias
has other meanings. But I think when we’re
using the term right here, we’re talking about
a systematic error in estimating some
attribute of the population. And I would submit
that actually, as you said at the
beginning, that’s not really relevant
to what you’re trying to accomplish
in this work– the quantitative
meaning of bias. What’s your reaction? Yeah. I totally agree. And you can see, going
back to my earlier example, that if we just tried to purely
randomly select for these focus groups, I probably wouldn’t
have gotten any of the people that I mentioned who were
in these focus groups that I described to you, because
we wouldn’t have accessed them. There’s just no way we probably
would have run into them. So that’s a great way to–
glad we have that on tape. It’s a good way
to describe that. So OK. What I’d like to do
now is start going through some of the
really typical methods in qualitative
research that I think could be very, very
useful in conjunction with quantitative methods,
or maybe on their own as a formative study. So these are the ones
that I’ll go through. These various methods– and
briefly, but the main thing that I also want to say is
that all of these methods, no matter what
method you use, just like in any kind
of research, you will want to use these
methods in conjunction with– preferably following–
a broad literature review. And the reason I put that in
there is because another rumor I’m constantly hearing is
that qualitative research must be done with as
blank a slate as possible. We can’t know very
much about what we’re doing before we start
the qualitative research or we will– and then there’s
the word again– bias it. And I don’t come from
that perspective. So I think where this came
from, is several decades ago there was one type– one kind
of newly formatted type– of qualitative research called
grounded theory, that you might have heard of. And the people who
are grounded theorists had this idea that they’re
coming from the ground up, and that they are going to
look at the data for what the data is and they won’t use
any kind of overarching theory. And they won’t try to put
any overlaying framework on what they’re doing. I don’t know whether
it’s entirely possible for human
beings to do that, because we all do have
our own ways of looking at things based on everything
that we know of today. But what I would say is,
for qualitative research you want to know what’s
been done already. You want to know what
people have already found in the communities
and with the populations that you’re studying, on the
topics that you’re studying. And you want to
take it from there. You don’t want to always
reinvent the wheel. So yes, please do do
literature research before you do your
qualitative work, and don’t feel bad about it. Can you use PubMed, or
is PsychINFO better, or are there– Well it depends on what
your research topic is. So interestingly, all the
papers that I’ve written are indexed in PubMed. The research could go
in an oncology journal or it could go in patient
education and counseling. It all depends on what it is. But if you, for instance,
are new to an area and let’s say you want to
look at child– substance use– A study that
I actually did– a community-based
participatory research study that the Latino
community in Providence asked me to do with them was
about substance use and risk for substance use in the
Latino youth community. So let’s say you were
going to do that, and you had never studied–
you don’t know what’s going on in Providence– and
you had never studied Latinos before. And you don’t really know
much about youth behaviors. So what you might want to do
is use PubMed, use PsychINFO, and then go into
sociological abstracts and find some
ethnographic studies. And I when I say broad,
I really mean broad. I think it’s a great idea,
especially for people who are going to be
studying populations that they are unfamiliar with. Look at the
ethnographic literature. Find out what people already
know about a particular community or a particular
ethnic community, even if they’re not living
exactly where you’re– just read really really
broadly to find out, because I do believe in this
notion of different ways of knowing. And you will get very
interesting articles in PubMed, but chances are
the slant– as you said the slant– is going
to be on the medicine part and not so much on
the people part. And it could be
very, very helpful for you to go into a
community knowing something about immigration patterns,
about cultural beliefs, or– it could go anywhere
depending on your topic. So again, just like
with anything else, it’s your research
question that’s going to drive what you
do and how you do it. So starting with participant
observation– what is it? It is observation. It’s observation but it’s
not just hanging around. Sometimes it looks like
just hanging around. One of the first
research I did was in a highland village
and Peru, and it really looked like I was
just hanging around. And the population thought I
was the laziest, richest person they had ever met
because how is it that I just have the leisure to
sit around in the town plaza? I was actually looking
at reconstruction after an earthquake. So it was relevant, but
people couldn’t understand. And they knew that I
was a student but it just– So even though it looked
like I was just hanging around, if somebody had done participant
observation, to watch what I was doing, they would have
seen that I was in that plaza morning, afternoon,
and evening– different days of the
week, on the weekend, sometimes with other
people, sometimes alone– always different but
always structured. I knew what I was doing. I knew that I needed to
view what I was looking at, which was basically– at
that particular moment– how people were
using the new plaza. But I needed to look at it at
all different types of day. So if you are interested in
how patients and their family interact at an
inpatient hospice unit, you don’t want to just go at
10:00 on a Tuesday morning, because it’s going to be very,
very different interactions on a Tuesday morning than
on a Tuesday afternoon, than in the evening,
than on a Saturday, than on a Sunday morning. So you need to go at
all different times. And you need to map
that out for yourself. So it’s structured
and it’s rigorous, even if it looks
pretty darn informal. The other thing that happens
during participant observation is you do a lot of
reflection on what you’re seeing–
note-writing afterwards, and you make questions for
yourself– and then you go back and look again. It can range from high to
no participation at all. So when I did do participation
on an inpatient hospice unit, clearly it was the low end. I had absolutely
no participation. I was just hanging around
and informally talking to the family members
in the common rooms. But sometimes you
might find there are people who are interested
in what is the toxin exposure or the carpal
tunnel risk of working in a particular kind of factory. And I’ve known people
who have actually gotten jobs in these
factories and work there. So that’s high
participation, obviously. And then there’s
everything in between. So to study what’s going
on in a unit– well let’s say I was a nurse in
that unit, and I was also getting a Master’s
in public health, I might be doing my
Master’s thesis on what’s going on in that unit. So it’s very high participation,
very, very, very heavy risk for bias because I have
my own way of thinking about what’s going on in there. And I teach a course and
in qualitative methods here at the School
of Public Health. And often my students
are physicians who are doing studies
in their own locations. And so we talk a lot about
that– about recognizing bias. So you need to figure out
what’s going on in that range. And actually, if you are
suddenly becoming a researcher, in an environment where
you’ve been a clinician, you need to let people know that
you’re wearing a different hat. They shouldn’t be interacting
with you as a clinician if you are a researcher
and they don’t know that. So that’s a whole other thing. But the idea is that you
are watching what’s going on and you are really,
really, really, really seeing what’s going on. Because– how many people
have gone to a place and spent some time
there, and then somebody that you’re
with has said, wow, did you see that really
interesting chair that was in the corner? People seem to be
afraid to sit on it. And you might say, oh no,
I didn’t notice a chair. Or you might have
noticed the chair, but it didn’t occur to
you one way or another that people seem to be sitting
on it or not sitting on it, or perching on it or whatever. And you might be thinking
right now, who cares? But let’s say– transfer that
into some kind of health topic that you’re thinking of. What I’m pointing out is that
we go through life all the time, and there are dozens and dozens
of things that we don’t see unless we put our mind to it. So what you want to do is
really put your mind to it. You want to decide what are
you going to be looking at. And really look at it and also
be open to seeing new things, because you don’t want
to get so boxed in. Some people like to just
go in with a notepad, or just remember things
and write it later, but other people like
to in with a template. And especially if you
have a project where you have participant observers
going all over the place, you might want them all to
be looking at similar things. So you might create an
observation template for them, but you also want to remind
yourself and your observers, what else is going on? You want to see things
you didn’t expect, otherwise it might not even
be worth doing the observation because you might just be
documenting what you already know what’s going on. So the idea is that you can
use participant observation at different times, because
you do some interviews. People tell you things,
and they tell you, oh yeah, when I’m doing this particular
task on the construction site, it’s too uncomfortable to
wear goggles so I don’t. And you don’t really know that
much about construction sites, but you’re doing a
study of safety risks on construction sites. And so you’re listening
to this person. You don’t even really
know how to probe. You don’t know how to really
ask the questions because you’re not really sure what
he’s talking about. You do some participant
observation. You see it in action. You are much better equipped
to ask your questions. Or– and or– you’re on
the construction site– and this was another
study that we did here at the School of Public
Health, this exact study– and you’re on the
construction site and you see people doing things. And you see sometimes
they’re wearing goggles and sometimes
they’re not wearing goggles for the same activity. Or there’s different
activities which look equally as dangerous to you. And the sparks are flying
and you can’t figure out why they are wearing
goggles sometimes and not wearing goggles other times. So then you might do some
informal interviewing during that participant
observation, but you might also work
those questions in. You’d be able to get
more credibility with your interviewees when you say,
the last time I was on a site I noticed that when they were
doing this they were wearing goggles, and when they were
doing this they weren’t. What do you think about that? What was going on? And then they can
explain that to you and you can find out
what their behavior is. So often qualitative
research is iterative, where you do one thing
then you do another thing, then you go back again, then
you move a little bit forwards. And maybe go sideways and then
you move a little bit forwards. You’re moving towards the same
goal of finding something out that you put forth
in your proposal. But your route to get there
might be a little bit variable. So I’m worried that that
clock is not working. It’s a half hour late
[INAUDIBLE] red clock [INTERPOSING VOICES] OK yes. Good. So we went through this
in terms of really, really seeing what you kind
of expect to see and what you don’t. And interviews–
there are basically four different
structures of interviews. And the one in red,
semi-structured interviewing, is the one that we use most
often in health research. I don’t really recommend
using informal interviews except during participant
observation, when it’s unplanned and you
don’t really have a script, and you just ask
something spontaneously. It works very well there. Unstructured planned
interviews I think are kind of silly because if
you’re planning an interview, plan what you want to know. Especially if you have more than
one person interviewing then you want to have a
core set of questions that everybody asks everybody. And you use spontaneous
probes to clarify what the person’s
saying, and also to follow new avenues of inquiry
that the person brings up. So that’s what we call
semi-structured interviewing, and that’s the one that you
use more often than not. And structured
interviewing is where the question list is fixed. It may be open-ended
questions, but you always ask them in the same order
and you always only ask those. And I don’t know why you’d
want to limit yourself. So usually you don’t
use that either. It’s the semi-structured
that really gets you the rich
information– that allows you to realize
that you’re getting to saturation, because if you
don’t allow people to bring in new ideas, then
you don’t really know whether there are
things that you’re missing. So you want it
structured in a way that you’re flexible enough
to follow those avenues. And it’s OK to reorganize. And it’s OK also to revise. You do a few interviews
and some questions aren’t working
out– change them. You can do that. You just modify them
to get the information, to get people talking on
the topics that you need. Also often after the
first few interviews, I don’t even use the script. I know the interview
script so well that I let the person
tell me the story. And then I probe
to get them to fill in the pieces that are missing. So it doesn’t matter if it’s
not done in the same order. And you want to do it in a
way that really, really gets them talking and comfortable. So I used to think that
qualitative interviewing can only be done in person. And actually I’ve done
quite a number of studies in the last number
of years where, for one reason or another,
it had to be by telephone. For instance, I
was doing a study with people who had
hematologic cancers. And they came to
Dana Farber and then to other tertiary care hospitals
from all over the country. So it was impossible to do
the interviews in person because we were
doing them two weeks after a second opinion
consult. And it worked out really, really well. And if you get a good pace
going and get very good at doing these
qualitative interviews, sometimes it’s good
to do it on the phone, especially with people
who are very sick. I could do it on their own time. They were in their own house. They could take a break and
tell me to call them back in five minutes. It worked very, very well. And then even I did
shorter interviews with more local
people, but it was more convenient for the
people to do it that way. So you could do it that way. Now– I’ve never tried
it– but with video, Skype, and all the new modalities
that we have out there, the opportunities
are just growing. Written qualitative
interviews I don’t think are all that useful because
people don’t generally like to be asked to write a lot. And it would only work
for very literate people, very motivated people. But whatever the
modality, they’ll be open-ended questions. So open-ended questions. They’re hard to write
and people often start with
closed-ended questions. And so what I tell my
students and the residence that I work with
in family medicine, is just blitz out your questions
and then look at them again. So at least you know what
you want to find out about. And then you tinker
with them, and you make them more open-ended. Yeah? Just a quick question– are
these interviews and questions, do they have to be
approved by an IRB? In which case, can you
change them as you go along? Well it’s up to your IRB. Some years the IRBs
that I’ve worked with have wanted to see exactly
how the question was worded. And some years they just
wanted bullet points or topics. So it all depends on
your institution’s IRB. So you need to find out
what it is that they expect. It’s sticky with
qualitative research since– to do
rigorous research, you don’t need to stick with
a poorly-worded question. It’d be a shame to do it. So talk to your IRB
and say, I’m not going to change the
nature of the question, I’m just going to
change the wording so it doesn’t sound so stilted. It’s kind of a touchy
issue, yeah, IRBs. So my advice would be to get a
good relationship with the IRB and work with them
to figure out what’s the best way to get
the research done within the requirements
of the IRB. Yeah. So what’s an
open-ended question? Have you ever tried to gauge
how much happy memories are involved in the recall
of these things? If you ask people what they
typically eat for dinner, they might start
recalling that, OK well, I eat vegetables and maybe
some potatoes and meat, and forgot to mention that
they have a can of Coke with every meal. If that’s what you want to
know, that’s what you do. Do you try to make it OK? To say, I really enjoy a
can of Coke with dinner, do you as well? Well, you try not
to lead the person. But usually what I do
is– I did actually a study of sugary
beverages, in fact– and first you start off saying,
what do you like to drink? And then they
mention tea, water. And then you say,
well what else? What about on a
hot day, you know you just finished playing
basketball or something? And so I don’t think I’d
ever go as far to say, I really enjoy a can of Coke,
because it’s normalizing in kind of a false way. But what I like to
do is– you start really open-ended to see what
is generated from the person. And then if they’re not
even touching the topic that you really want
them to comment on, then you ask about it, after. You know like when
I did the study of the different
screenings, and I would say, what tests do you use
to look for– have you ever had a test for
cervical cancer? And then to a person,
almost every woman said no, which I
didn’t really believe. But then I was
thinking, I don’t think that they’re denying
that they’ve had a pap, I think they don’t realize it. So then after we go
through all of that, then I say, so do you
ever have a Pap smear? Oh yeah, sure. I get one every year. So, I didn’t say it
in the beginning. So what I learned from that
is that they are getting paps, and they’re not really
sure what it’s for. So you learn a lot that way. But if I had just started
with the pap, or said, well I get paps. Do you get paps? I don’t know. It’s a little– I
wouldn’t go there. I probably wouldn’t
do that. [LAUGHING] But the idea is to
be really open-ended. So instead of saying
do you eat vegetables as part of your dinner? You would say, what do you
typically eat for dinner? And then come back in and
say, so what about vegetables? What about drinks? What do you– that
kind of thing. Put it together. Or how would you
rate your experience at your doctor’s office today? And the person might
say, well it was OK. Or, it was very good– which is
fine if you’re doing a survey. But don’t you want to know more? Don’t you want to know
what was good about it, and why was it good, and is
it always goes like this? Or is it different
than it was last time? Do you expect it to be
this good next time? Or was there a particularly
nice medical student there today who made you feel better? What was it? So then you would
say, so tell me about how your doctor’s
visit went today. And you would start
getting all of that. And then if she’s
not bringing up the particular thing
you want to know, like what is the
impact of having a medical student in the room? Then you ask about
that kind of thing. I can see I’m not
going to really be able to get through too much. But there’s four kinds
of individual interviews that we find to be very,
very useful in clinical and public health research. And I will go
through each of them very briefly because I
think you would use all of them, one way or another. In-depth interviews
are just what they are. They can be on any topic,
whatever it is you’re studying. But you want to explore
the topic in detail with the follow-ups
and the probes that I’ve been alluding to. And you want to know what’s
going on in people’s own words, with the structure and
the flexibility that I’ve described. And so that’s what you do. There isn’t any
particular other way. Some people call them
depth interviews. Some people call them
in-depth interviews. Some people erroneously call
them cognitive interviews. Don’t do that because
cognitive interviews are something else, which I’ll
tell you about in a minute. But any kind of just basic
open-ended Interview, you can call it an
in-depth interview. Another kind, that I have
used a lot in my studies here with people in the
School of Public Health, is a specific kind
of in-depth interview that you look backwards and
you look forward in time. And being diachronic can
be extremely helpful, because understanding what
somebody is doing or thinking now, may not be as
informative to you as understanding how she’s
come to make that change and whether she’s planning on
keeping it in that direction. So you look backwards
and you look forward. And you still use the
spontaneous probes and all the regular
techniques, but you go backwards and forwards. So as an example, if you’re
interested in parents’ attitudes regarding their
obese child’s weight. So instead of harping
on them right now, so what do you feed this child? And what is this child drinking? And does this child– what kind
of exercise does this child get? Instead of just focusing on that
where you don’t really know– did they just move to a
new neighborhood so now they’re going to the park? But actually until three
months ago, they never did. Or grandma moved in and
since grandma’s cooking now the food is different
than it was before. These are the kind of things
that you could imagine would be very, very helpful. So you want to know, I think
you usually want to know, how are things different
now than they were maybe in the immediate
past and a little bit further and a little
bit further– whatever makes sense for your study. So you want to know
maybe also what they were thinking and
doing in previous years, what they thought and did
regarding their other children. What were the
attitudes and behaviors regarding children’s food
and activity when they were growing up themselves? And so on. All of this can broaden your
contextual understanding of what’s going on. So that’s what’s nice about
life history interviews. Key informant interviews
are basically– they’re kind of different
because your interview might be a little bit different
with each person. Like let’s say you’ve got 10
or 12 key informant interviews before you start doing
the rest of your study. Because these are people
who know something about your topic in a
different way than you do. So some patients could be
key informant interviews, but it could be–
let’s say you’re looking at domestic violence
and you’re a physician. So you know everything
that you know from a physician’s perspective. So maybe you also want to
talk to community advocates against domestic violence,
community-based counselors, and social workers, and
mental health professionals, shelter staff, police,
lawyers, academics who have studied this
from the victim’s perspectives, patients,
other [? ways. ?] So these are people
who can broaden your understanding of a topic. And what they might
help you do is actually change your research
question, because you might be thinking
of it this way, and you talk to people who know
more about, say, the victim’s experience. And suddenly you
realize, hold on. This is narrow. And this has been
done before anyway. I want to actually
look at it this way. And so that’s what’s nice
about key informants. So we call them
insiders-outsiders, because they’re inside–
they have an insider knowledge of one way
or another, but they’re outside enough that they
can explain to you what’s going on in the
context that make sense to you from a
scientific perspective. So they’re usually a
little bit different than the people
who are going to be the participants in your study. They might be community people,
they might be professionals, but they can communicate across
that boundary of research. So it’s very helpful. I think you’re one slide behind. Oh am I? Sorry. OK. So cognitive interviewing–
cognitive interviewing actually comes out of police work for
getting people to remember. It’s sort of like what
we were talking about, to get people to really
remember what they were doing or what they saw in
some kind of crime. But it’s used in
public health, and I suppose for clinical
research too, to help people understand how
their survey questions are being received by
the participants. So you do it to test
your survey questions. And I can’t go into
all the details of how. But basically, you
have people, either in a group or individually,
who will tell you what they’re thinking as
they look at the questions. And that is an eye-opener,
let me tell you. Because you think your
questions are crystal clear, but you know what
your study’s about. You know what your question
is supposed to mean. So you find out from people who
are just like your participants to be, what people are thinking. So they read the question,
they answer the question, or you read it to them, in
whatever modality you’re going to be administering
this survey. And then you ask them, why
did you choose that answer? What were you thinking about
when you chose that answer? Or you hesitated– was that
because you were confused, or is there something that
you didn’t understand? And you do this– it’s very
tedious– but you do this with every single question. And I’ve been through this where
we sent a cadre of research assistants into a community
center, and people file in and we just keep
doing the survey over and over and over
with all of these probes, and asking them. And then we find out
which questions really are kind of problematic,
because people think they mean something else. And there was an anthropologist
that once said, you only know the meaning of the
answer to your question until you know what
question was being answered. And in surveys,
sometimes you really can’t be sure that you know what
questions people our answering. So it’s worthwhile doing
this kind of technique and it’s used very, very
widely in health and medicine. So focus groups, very quickly. There are great primers
out there on focus groups. You can Google it. They’re easy to understand
and kind of difficult to do. So group facilitation–
and you would want to use focus groups only
and if what you’re looking for is to elicit the dynamic
interaction among participants. If you want in-depth,
uninterrupted stories of people’s experiences on the
topic that you’re studying, you want to do
individual interviews, because in focus groups
you just get snippets. It’s very superficial. But what you do get
is that interaction. You can also get bias, because
somebody says something, somebody else decides
they want to agree. But with the skill
of facilitation, you can keep cutting
through that. You can keep
undercutting when you see that happening,
and asking people different questions
in different ways, and moving them through the
topic and circling back. It takes a lot of
facilitation to do it, especially if you have noisy
group or a very talkative group, but mostly
because you want to get at those
issues of– let me try to get at as honest
opinions as possible. Usually 8 to 10 people–
I’ve had focus groups with more, I’ve had
focus groups with less, but that’s considered the ideal. You want to select the
participants carefully because you don’t
want people to be upset by other people in
the group necessarily. So you wouldn’t necessarily
have physicians and CNAs in a practice in the
same focus group, because they may not
be as open as they might if they weren’t together. You of course audio record it,
usually have it transcribed. You definitely,
definitely want a script, but you want to know that
thing backwards and forwards. You want to have practiced
that in the mirror and with your
friends ad nauseam, because people will be
continuously bringing up question 13 when you’re
only up to question three. And if the flow is good,
you want to go with it. So you move your way
through everybody’s story. And in the end,
usually what you do is an analysis of
the group as a whole, not so much of the
individuals, because you’re not really sure who said what. It can be very, very, very,
very good data if done well. And focus groups are
usually only as good as your recruitment techniques. So that’s the short
version of focus groups. Other methods– now these
days with the internet, there’s all kinds of
things that people do. Obviously people have
done analyses of TV ads for junk food and the impact
on kids’ eating behaviors. And so there’s all kinds
of things that have nothing to do with live
human beings that can have qualitative content
analysis applied to them. So advertisements–
somebody I work with did a study of tobacco
company documents, analysis of comparative state-level taxes
of health-related legislation. You can look at
web page content. You could look at discussion
forum content, blog content, you name it. And the other that is
used– actually there’s a term now in the literature. You can look in PubMed under
something called photovoice. And something that I did,
actually back in Peru all those years ago, was
give cameras to people and have them
document their lives. And people have
done that recently, for people to document, for
instance, their life in pain and see what they
take photographs of. And then you print
those out and then you have them explain the
photographs or take videos. There’s all kinds of very, very,
very creative and interesting ways to get at, again,
other ways of knowing, and for people to bring the
social context of their health problem, whatever it
is you’re studying, into more vivid detail for you. And there’s great
articles on these things and actually, if anybody
wants to get in touch with me, email me or something. I’d be happy to talk to you
more about any of these methods. So in summary, the uses
of qualitative research are formative research in the
very beginning of your study, to really find out what’s going
on in a community or in an area where you don’t know very
much about the topic. Process-tracking, to
try to figure out– so I’m doing an intervention. How are people experiencing
this intervention? I’m noticing a lot of
people aren’t showing up on certain days. What’s going on? So you might want to do
some process-tracking to see what’s going on. Or outcome evaluation– I
did a study here with people in the School of Public
Health of fruit and vegetable and physical activity
behaviors in small businesses. And so at the end of this
very long intervention, and many small businesses,
we did qualitative interviews with people who participated
a lot in the study, and people who
hardly participated at all, to find out what
was different about them. Not that we could
statistically look at that, but to try to get some
trends and understand. And also to find out why,
why they didn’t participate very much, or why they did
decide to participate a lot. So it’s useful at all
different levels of the study. You can obtain data
that’s useful on your own. Sometimes the
qualitative research that we do in the
beginning of the study generates the first papers
of a very, very large study, because the data does
stand on its own. And it was also used to
help design the surveys, to help design
the interventions, and the intervention material. So it’s very, very multi-use
once you get that data. Also you might want to
use it as building blocks, as I just mentioned, to help
you figure out what are the best responses to put in my
survey, because I don’t know what people are drinking here. So if I get them
to talk about it, then I’ll have a better idea of
what the closed-ended responses should be on the survey. Or cognitive interviewing
to test the surveys, or to complement or maybe
even explain– sometimes we do qualitative
interviews at the end– to help explain the statistical
results that we got. Like to try to get a sense of
maybe why a certain result was [INAUDIBLE] So let me super quickly
give you three examples. And you can mostly see what the
different methods were there on the slide. So the health promotion among
unionized construction laborers that I’ve talked a little
bit about already– the formative research
wasn’t all qualitative. Sometimes it’s both
qualitative and quantitative. So we did a cross-sectional
telephone survey with 1,100 people
around the country. We did focus groups in
four regions of the country for 16 focus groups. And then we did
participant observation on the construction sites. So that was a really,
really nice way to get a very good sense
of what was going on before we designed the
intervention and the materials. And just to give you
an example of how did we use this information– So
for the intervention materials, we figured out from all
of this formative research how we would engage the laborers
in this union in the study, how we would get them to
open the invitation letter and to read it,
because there were so many warnings that
people wouldn’t open the letters coming from us. So should it come from us? Should it come from the union? Should it have the
union logo on it? Should it have the president
of the national union’s picture on it? Or should have all the
local union presidents around the country? Actually the answer
was the latter. And what are the symbols
that people in this union resonate with? What are symbols that we should
make sure to stay away from? What topics would
catch their interest that we could use as a hook? And on and on and on
and on– all of this came from the
formative research, because a similar study
hadn’t been done before. And we really didn’t have
the answers to figure out where to start designing from. So a study that I
did in Providence, called Habelmos de Ti:
Let’s Talk about You, was about the menopausal
transition among Latina women– mixed Latina women
in Providence. And I did a series
of focus groups with women in different
parts of Providence and from different countries. And it was looking
at their perspectives on the social, cultural,
and the physical elements of the menopausal transition. And then the women wanted to
keep getting together so much, and they kept
showing up on nights that I wasn’t at the community
center to hold focus groups. So I coined a new term,
reunion focus groups. And I had a whole other
set of focus groups are open to anybody in any focus
group who had ever come before. And so then we had
continued discussions. And then we also did
digital interviews with a different set of women,
to try to get more information in-depth. And sometimes what we do, is we
pick people out of focus groups in order to interview
them in-depth, or we can interview
other people, because you want to get both. The dynamic was great
but we were missing some of the richness of the story. And then I also designed
an interactive internet intervention, which
women participated in. And further information and
data for qualitative analysis was the content of the postings. So that was multimethod
qualitative. And then I won’t go much into
it, but you can see in a study that we did here of cancer
risk and low-income housing, we used all of these different
methods for the study. And it’s not so much that
we analyze them together, but we analyze them in
respect to each other to try to understand the problem
from all different sides. So that is the end
of my talk and I don’t know if there’s
time for any questions. Sure. Are there any
questions that anybody has on any of the
topics I brought up? [INAUDIBLE] Well let me say that, I think
that my perspective has been this entire topic that so we–
the School of Public Health we’re so oriented toward
quantitative thinking, that one can have the
illusion that research is quantitative research. But then as I’ve begun
to appreciate more the significance
qualitative research, I’ve realized that one can’t do
quantitative research until you have well-formulated questions. And those
well-formulated questions really fall in the domain
of qualitative research. So every quantitative
investigation that we do has been preceded by
open-ended analytical work– whether it’s in the laboratory,
whether it’s in the community– to form an understanding
of the questions sufficient to construct the
quantitative investigation that we’re doing. So in a very fundamental
way, qualitative research is really the
foundation of science. So that would be my
comment on this topic. Thank you. Robert, thank you. I’d like to get a free
consultation from you, and that is– NIH
spends a lot of money to fund the clinical
translational science centers. And as part of the requirement
when you submit a grant, which Harvard did– it has the
[? catalyst ?] grant of course, there’s some form of
evaluation as to whether or not what you’re doing is working. So from our perspective in
post-graduate education, we are very interested
in trying to understand whether the kinds of educational
experiences that we offer, or if the range of
courses that we have and whatever other creative
things we try and do, are actually helping. So have you had any experience
in trying to assess something like that, and advise
that you might have for how we would make a judgment
about whether we’re heading in the right direction
with the kinds of courses we put together. Yeah. Well it’s actually interesting,
because being– with my Brown appointment– being in
the school of medicine and being in the department
of family medicine, we do a lot of
educational research. So that’s what we’re
doing is evaluating educational innovations
on a continuous basis. And it helps. It does. What we like to do is
usually multi-method. We might do some surveys and
things, but what I like to do is interview people
from all spectrums. So we’re doing some
innovations in trying to get family medicine residents
to do more scholarly work. And so we’ve implemented
those things, but we’re interviewing the
faculty who are teaching, the faculty who serve as mentors
for the scholarly projects, the faculty who
precept in the clinic to see how they feel about
mentoring and fostering research questions. And we’re interviewing
all of the learners who are going through at
different stages of learning to understand. And we’ve compared
them to learners who were third-year residents
before we start the program. So that’s my advice really,
is– get people from all angles, because I want to
know, does the faculty feel prepared to
mentor the projects? We’ve just said, every
project needs a mentor. Great, but how do the
faculty feel about that? And how to do the residents
feel about asking? So I can certainly go more
in-depth at another time, but I think really covering
the range of participants is important. Yeah? So a lot of the lecture
focused on adults. I’m wondering if you have any
insight into doing research on children and qualitative
research on children, and what the challenges might
be and your recommendation would be– Yeah. Well I guess a lot
of what you might run into you would be
ethical and IRB issues. So it really depends on what
your topic is– whether it’s appropriate to be talking
open-endedly with children about a certain topic. One consideration,
which would be the same with any
kind of research, is you need a different
kind of consent. So when I did the study of
inner city dwelling youth about risk for
substance use, I needed to get a consent from
their parents and an assent from them. And you do get into
the immaturity factor of talking to kids, especially
if you’re doing focus groups. You probably have
a better chance of getting a good story
individually with kids. But you might have
a research question that would benefit from
the dynamic interaction. So do both in that sense. But kids’ attention spans, and
some people– that whole issue of out of the mouths
of babes– maybe you get a more truthful
story from kids about what they’re
really eating at home, or what’s really going on, than
you might from their parents. But you don’t know how prepped
they were by the parents before. So I think that there are dozens
and dozens of considerations in doing qualitative
research with kids. It really depends
on your project in terms of– But people do. People do. –the experience of
cancer treatment– there’s tons of studies
that are out there with pediatric
populations, but you just want to make sure you’re
doing it in the way that makes good sense. And maybe also–
this is a funny thing about qualitative
research– maybe also if you’re the
right person to do it, not just with kids
but all around. Do you have the patience to
let someone tell you the story without interrupting a lot? Do you have personality
to put people at ease? Some people frankly
probably aren’t cut out for doing their own interviews
or their own focus groups. And so it doesn’t mean you
can’t use that, but collaborate with someone who might do it. I can see a lot of issues with
pediatrics, which probably aren’t all that different
than with adult populations, but they manifest themselves
differently overall. Unless– I don’t know if there
was something specific you were thinking of that– I think we’re out of time. Yeah. –chat with you offline. OK. Is there one more question?


  1. This is very useful. THANK YOU SO MUCH!!

  2. I watched this video beginning to an end it was very useful. I'm taking stats and it was a fascinating study of designs!

  3. HI, I was wondering if the slides are accessible anywhere. In addition, do you have the suggested reading that accompanies this lecture. This would be extremely helpful!

  4. I am beginning my thesis and I chose qualitative methods. Your explainations are clear and on point. You have a friend accross town at the PVAMC library. Ray

  5. I watched thi video until finished, very veru helpful thank so much

  6. what is investigative methodology ?

  7. Very helpful. Thank you

Leave a Reply

Your email address will not be published. Required fields are marked *