White Paper
White Paper
A Comparative Review of Online Qualitative Methods
February, 2010
By Anni Macht Gibson, Senior Research Executive
Sigma RMG
Online
is very much the wave of the future for qualitative research. In fact,
for some Qualitative Research Consultants, it has come to represent 75%
or even more of their businesses. It is both cost efficient and
effective and more and more clients are finding that online tools meet
their needs as well as or better than traditional face to face methods.
We are not suggesting that online should be a replacement for all
traditional face to face qualitative techniques such as focus groups.
There are somewhat different dynamics between the two platforms and the
methodology to be employed should be driven by the specific research
objectives. However, there are several reasons for the growth of online
methodologies:
1) online qualitative tools are generally cost efficient,
2) the techniques have the potential to be as effective as face to face sessions (often
more effective in “sensitive” categories where anonymity is a real plus), and
3) consumers’ access to computers/internet and skill levels are at an all time high.
We hear over and over
from participants how positive their online research experience has
been. Clients appreciate the somewhat shorter time from project
inception to report (including the immediacy of transcripts) and the
convenience and cost benefits of eliminating travel altogether. In
addition, online methods offer geographical diversity (including rural
respondents) that cannot be duplicated in a face to face setting. In
fact, for all intents and purposes, the online techniques can be used
globally. Online also offers anonymity (we often assign pseudonyms to
the participants) and it eliminates the possibility of appearance bias
and reticence or embarrassment about discussing personal subjects. Using
an online platform generally “levels the playing field” for
participants, usually negating the need to for the moderator to spend
time balancing the group
“wall flower” and the “dominator.”
Online techniques also
offer access to the consumer in the home in a way that a traditional
central-location focus group does not. Of course, one can always do
in-home and/or one-on-one interviews, but they are far more costly and
time-consuming than online sessions. With the online tool, the client
can look at the in-home environment while still benefitting from the
efficiencies of a group study.
The primary objection
from people who have NOT used an online technique is “But what about
body language? We can’t SEE them.” Some tools we discuss below offer the
opportunity to view respondents online; however, if the non-camera
online technique is being used for the correct research reasons, we have
not found the lack of visual contact with the respondent to be a
drawback. People are direct and honest online -- often more honest than
they are in-person as anonymity breeds forthrightness. Importantly, both
the quality of the information and depth of the responses are
excellent.
Let us now take a look at the various tools available for use in online qualitative projects:
Real Time Online Focus Groups (Text Based) –
In this case, all the
participants log into a chat room at a specific time. They can be from
diverse parts of the country (or the planet). The moderator types in the
questions (actually, usually we pre-load a discussion guide) and waits
for all the participants to respond, probing as needed, then moves on to
the next question. The advantages are ease of participation (no
schlepping to a focus group facility), geographical diversity (you
obviously can get people from various locales…not just 3 cities and
rural consumers) and elimination of travel costs. A white-board and
linking allows us to show graphics,
audio and video. There is
also a virtual “back room” that allows for clients to observe the
sessions, and to interact with each other and the moderator regardless
of their geographic location.
Online Real Time Focus Groups incorporate the advantages inherent in online research:
•Geographical diversity
•Reduction of the dominant respondent factor
•Timeliness of data (instant transcript)
•Elimination of travel costs
•Capability to present stimuli via graphics or video
•Virtual back room
The real-time focus
group, however, does require respondents to be online at a specific time
and for a specific period of time (generally 1 1/2 - 2 hours).
Real-Time Focus Groups
with Web Cams (Text and Video) – Generally respondents are sent a web
cam to ensure everyone has an up to date, functioning model with the
proper software. In this format, one can see and hear the respondents.
The Web Cam group offers the advantages of the Online Real-Time (Chat) Focus Group:
•Geographical diversity
•Reduction of the dominant respondent factor
•Timeliness of data (instant transcript)
•Elimination of travel costs
•Capability to present of stimuli via graphics or video
•Virtual back room
As with other real-time
formats, this technique requires respondents to be online at and for a
specific time (generally 1 1/2 - 2 hours).
Asynchronous Bulletin
Boards (Text Based) - The flexibility of Asynchronous Bulletin Board
Sessions (BBS) and depth of information that can be obtained have
allowed this technique to become the current favorite among Qualitative
Research Consultants.
(Qualitative Research Online, Jeff Walkowski, Thomas W. Miller, Research Publications, LLC 2004)
This tool allows
respondents to answer questions at a time when it is convenient for them
to do so. Typically, we ask that participants log on to the BBS twice a
day to address the topics in a series of questions from an approved
discussion guide (though in reality they can answer all of the questions
at once at the end of the day, or catch up for a missed day if they
need to do so). The BBS allows respondents to participate at their
convenience and without the pressure of time or video. This results in
generally well thought out and depth responses. The sessions are usually
conducted over a three or four day period. While the participants
respond individually, once they have posted a reply to the moderator’s
question, they can see others’ responses and interact with them, often
resulting in a fruitful discussion thread.
While most boards that we
moderate take place over three or four days, the technique can be used
for diaries and journals (such as those that might be used for new owner
feedback or longitudinal experiential studies) lasting weeks or months
depending on the research objectives. Online bulletin boards are also
well-suited multiple-phase projects. For example, if the client wants to
first get input on a category or concept, then send the participants a
new product formulation to try and have them come back to answer more
questions -- the process could take up to 3 weeks and the board would
remain open the entire time.
Because bulletin boards
occur over time, we are able to assign respondent tasks. For example, we
can have them use a specified mix to bake a cake and decorate it, then
take a picture of it and upload the photo. We can have participants each
make a collage representing their experiences with, for example,
grocery shopping, or using tax preparation software, visiting the
doctor’s office -- any topic our client wants to learn about. We are
able to ask about what products are on hand in the pantry and have
participants upload a photo of their storage areas. Because participants
answer at their leisure, we find that the “transcript weight” of a
single online board with 12-15 participants is equal to 2-3 face to face
focus groups. When appropriate, the “masking” feature of our bulletin
board platform allows us to prevent the participants from seeing each
other allowing us to conduct virtual one-on-one interviews – across the
globe if necessary without the hassle, cost and unpredictability of
travel.
Hybrid Real-Time Groups
(Text/Voice/Video) – This is essentially an online meeting facility type
of platform, including conference call for audio discussion and with
participants being able to interact and respond to presentations and
other visuals. There is also the capability to have participants type
the answers into a “chat pod” on their computers when it is deemed
advantageous – for example, when we want an individual to answer without
hearing other respondents’ ideas first. The group is moderated and the
moderator can be seen via webcam with the option of having the
participants also on camera. Advantages of this technique are
geographical diversity, elimination of travel and two methods of
response -- audio and via “chat pods” on the computer. However, like the
other real-time techniques all participants, clients and the moderator
must be present at a specific time and for a specified length of time.
SMS/Hand-Held Sessions
SMS (Short Messaging
Service), commonly known as texting, is available to more than more than
240 million subscribers in the United States alone. Further, in the age
groups between 13 and 44, the mobile device is used more frequently for
texting than phoning; and its expansion among all age groups is
exponential.
The hand-held device
offers marketing researchers the opportunity for ultimate portability in
obtaining participant input. We can interact with the participant
(notice, we did not say respondent) at, or at least closer to what for
many marketers is “the moment of truth” – in the decision-making
process.
In general, the hand-held sessions are considered as one-on-one research; however, several
research platforms allow us to gather and aggregate the participant input.
How It Works Plus a Few More Benefits
In essence, virtually
everything that we can do in face to face sessions we can do online
including showing concepts/visuals/video, assigning home-work projects
and viewing them and using projective techniques. Again, however, we
reiterate that we are not suggesting that online can, or should, replace
all face to face qualitative research. The methodology must be driven
by the research objectives.
We continue to expand the use of projective techniques such as Personality Image Profile,
Metaphorical exercises, collage, Archetype identification, etc., in the online setting. These
projective techniques allow us to dig deep as well as to “cut to the chase” in terms of the
consumer experience.
Importantly, with all of these online techniques, the client can “be
there” in the virtual backroom to experience the group. And we even send
you M&M’s so that you’ll have the Official Nourishment of
qualitative research!
As mentioned in the
individual descriptions, transcripts of online sessions are available as
soon as the session ends. Audio transcripts are physically transcribed
and take slightly longer. Recruiting methods are consistent with other
marketing research activities. Online panels can be employed for some
projects, but generally phone recruiting is used where practical as our
experience has shown a higher “show-rate” than online recruiting because
the potential participant has made a verbal commitment to a “real human
being” on the phone. We have put together on-going consumer panels for
some clients. In these cases, we may work with the firm’s Marketing
Research or Insights Department – and our contact coordinates setting up
projects with his or her internal clients. We recruit a group of
respondents to be “on call” for a period of 2-3 months. Each panelist
may participate in 2 or even 3 studies during his/her tenure.
Sigma Research has been a
full-service Marketing Research partner to our clients for nearly 30
years. Our offerings include traditional face to face groups, one-one
one interviews, in-homes, shop-alongs and other in-situ research, as
well as all the online techniques outlined in this document and
quantitative surveys (also mostly online these days).
We are always available to answer questions and are happy to provide a proposal for any
projects that are being contemplated.
AMG
Anni Macht Gibson | Senior Research Executive
Sigma Research Management Group
4404 Brazee Street
Cincinnati, OH 45209
513-979-2163 | 513-979-2166 (f)
© Sigma Research Management Group, Ltd. 2010
We listen between the lines