White Paper

 

A Comparative Review of Online Qualitative Methods

February, 2010

By Anni Macht Gibson, Senior Research Executive

Sigma RMG

Online is very much the wave of the future for qualitative research. In fact, for some Qualitative Research Consultants, it has come to represent 75% or even more of their businesses. It is both cost efficient and effective and more and more clients are finding that online tools meet their needs as well as or better than traditional face to face methods. We are not suggesting that online should be a replacement for all traditional face to face qualitative techniques such as focus groups. There are somewhat different dynamics between the two platforms and the methodology to be employed should be driven by the specific research objectives. However, there are several reasons for the growth of online methodologies:


1) online qualitative tools are generally cost efficient,

2) the techniques have the potential to be as effective as face to face sessions (often

more effective in “sensitive” categories where anonymity is a real plus), and

3) consumers’ access to computers/internet and skill levels are at an all time high.


We hear over and over from participants how positive their online research experience has been. Clients appreciate the somewhat shorter time from project inception to report (including the immediacy of transcripts) and the convenience and cost benefits of eliminating travel altogether. In addition, online methods offer geographical diversity (including rural respondents) that cannot be duplicated in a face to face setting. In fact, for all intents and purposes, the online techniques can be used globally. Online also offers anonymity (we often assign pseudonyms to the participants) and it eliminates the possibility of appearance bias and reticence or embarrassment about discussing personal subjects. Using an online platform generally “levels the playing field” for participants, usually negating the need to for the moderator to spend time balancing the group

“wall flower” and the “dominator.”


Online techniques also offer access to the consumer in the home in a way that a traditional central-location focus group does not. Of course, one can always do in-home and/or one-on-one interviews, but they are far more costly and time-consuming than online sessions. With the online tool, the client can look at the in-home environment while still benefitting from the efficiencies of a group study.


The primary objection from people who have NOT used an online technique is “But what about body language? We can’t SEE them.” Some tools we discuss below offer the opportunity to view respondents online; however, if the non-camera online technique is being used for the correct research reasons, we have not found the lack of visual contact with the respondent to be a drawback. People are direct and honest online -- often more honest than they are in-person as anonymity breeds forthrightness. Importantly, both the quality of the information and depth of the responses are excellent.


Let us now take a look at the various tools available for use in online qualitative projects:


Real Time Online Focus Groups (Text Based) –

In this case, all the participants log into a chat room at a specific time. They can be from diverse parts of the country (or the planet). The moderator types in the questions (actually, usually we pre-load a discussion guide) and waits for all the participants to respond, probing as needed, then moves on to the next question. The advantages are ease of participation (no schlepping to a focus group facility), geographical diversity (you obviously can get people from various locales…not just 3 cities and rural consumers) and elimination of travel costs. A white-board and linking allows us to show graphics,

audio and video. There is also a virtual “back room” that allows for clients to observe the sessions, and to interact with each other and the moderator regardless of their geographic location.


Online Real Time Focus Groups incorporate the advantages inherent in online research:

  1. Geographical diversity

  2. Reduction of the dominant respondent factor

  3. Timeliness of data (instant transcript)

  4. Elimination of travel costs

  5. Capability to present stimuli via graphics or video

  6. Virtual back room


The real-time focus group, however, does require respondents to be online at a specific time and for a specific period of time (generally 1 1/2 - 2 hours).

Real-Time Focus Groups with Web Cams (Text and Video) – Generally respondents are sent a web cam to ensure everyone has an up to date, functioning model with the proper software. In this format, one can see and hear the respondents.


The Web Cam group offers the advantages of the Online Real-Time (Chat) Focus Group:

  1. Geographical diversity

  2. Reduction of the dominant respondent factor

  3. Timeliness of data (instant transcript)

  4. Elimination of travel costs

  5. Capability to present of stimuli via graphics or video

  6. Virtual back room


As with other real-time formats, this technique requires respondents to be online at and for a specific time (generally 1 1/2 - 2 hours).


Asynchronous Bulletin Boards (Text Based) - The flexibility of Asynchronous Bulletin Board Sessions (BBS) and depth of information that can be obtained have allowed this technique to become the current favorite among Qualitative Research Consultants.

(Qualitative Research Online, Jeff Walkowski, Thomas W. Miller, Research Publications, LLC 2004)


This tool allows respondents to answer questions at a time when it is convenient for them to do so. Typically, we ask that participants log on to the BBS twice a day to address the topics in a series of questions from an approved discussion guide (though in reality they can answer all of the questions at once at the end of the day, or catch up for a missed day if they need to do so). The BBS allows respondents to participate at their convenience and without the pressure of time or video. This results in generally well thought out and depth responses. The sessions are usually conducted over a three or four day period. While the participants respond individually, once they have posted a reply to the moderator’s question, they can see others’ responses and interact with them, often resulting in a fruitful discussion thread.


While most boards that we moderate take place over three or four days, the technique can be used for diaries and journals (such as those that might be used for new owner feedback or longitudinal experiential studies) lasting weeks or months depending on the research objectives. Online bulletin boards are also well-suited multiple-phase projects. For example, if the client wants to first get input on a category or concept, then send the participants a new product formulation to try and have them come back to answer more questions -- the process could take up to 3 weeks and the board would remain open the entire time.


Because bulletin boards occur over time, we are able to assign respondent tasks. For example, we can have them use a specified mix to bake a cake and decorate it, then take a picture of it and upload the photo. We can have participants each make a collage representing their experiences with, for example, grocery shopping, or using tax preparation software, visiting the doctor’s office -- any topic our client wants to learn about. We are able to ask about what products are on hand in the pantry and have participants upload a photo of their storage areas. Because participants answer at their leisure, we find that the “transcript weight” of a single online board with 12-15 participants is equal to 2-3 face to face focus groups. When appropriate, the “masking” feature of our bulletin board platform allows us to prevent the participants from seeing each other allowing us to conduct virtual one-on-one interviews – across the globe if necessary without the hassle, cost and unpredictability of travel.


Hybrid Real-Time Groups (Text/Voice/Video) – This is essentially an online meeting facility type of platform, including conference call for audio discussion and with participants being able to interact and respond to presentations and other visuals. There is also the capability to have participants type the answers into a “chat pod” on their computers when it is deemed advantageous – for example, when we want an individual to answer without hearing other respondents’ ideas first. The group is moderated and the moderator can be seen via webcam with the option of having the participants also on camera. Advantages of this technique are geographical diversity, elimination of travel and two methods of response -- audio and via “chat pods” on the computer. However, like the other real-time techniques all participants, clients and the moderator must be present at a specific time and for a specified length of time.


SMS/Hand-Held Sessions

SMS (Short Messaging Service), commonly known as texting, is available to more than more than 240 million subscribers in the United States alone. Further, in the age groups between 13 and 44, the mobile device is used more frequently for texting than phoning; and its expansion among all age groups is exponential.

The hand-held device offers marketing researchers the opportunity for ultimate portability in obtaining participant input. We can interact with the participant (notice, we did not say respondent) at, or at least closer to what for many marketers is “the moment of truth” – in the decision-making process.


In general, the hand-held sessions are considered as one-on-one research; however, several

research platforms allow us to gather and aggregate the participant input.


How It Works Plus a Few More Benefits

In essence, virtually everything that we can do in face to face sessions we can do online including showing concepts/visuals/video, assigning home-work projects and viewing them and using projective techniques. Again, however, we reiterate that we are not suggesting that online can, or should, replace all face to face qualitative research. The methodology must be driven by the research objectives.


We continue to expand the use of projective techniques such as Personality Image Profile,

Metaphorical exercises, collage, Archetype identification, etc., in the online setting. These

projective techniques allow us to dig deep as well as to “cut to the chase” in terms of the

consumer experience. Importantly, with all of these online techniques, the client can “be there” in the virtual backroom to experience the group. And we even send you M&M’s so that you’ll have the Official Nourishment of qualitative research!


As mentioned in the individual descriptions, transcripts of online sessions are available as soon as the session ends. Audio transcripts are physically transcribed and take slightly longer. Recruiting methods are consistent with other marketing research activities. Online panels can be employed for some projects, but generally phone recruiting is used where practical as our experience has shown a higher “show-rate” than online recruiting because the potential participant has made a verbal commitment to a “real human being” on the phone. We have put together on-going consumer panels for some clients. In these cases, we may work with the firm’s Marketing Research or Insights Department – and our contact coordinates setting up projects with his or her internal clients. We recruit a group of respondents to be “on call” for a period of 2-3 months. Each panelist may participate in 2 or even 3 studies during his/her tenure.


Sigma Research has been a full-service Marketing Research partner to our clients for nearly 30 years. Our offerings include traditional face to face groups, one-one one interviews, in-homes, shop-alongs and other in-situ research, as well as all the online techniques outlined in this document and quantitative surveys (also mostly online these days).


We are always available to answer questions and are happy to provide a proposal for any

projects that are being contemplated.


AMG

Anni Macht Gibson | Senior Research Executive


Sigma Research Management Group

4404 Brazee Street
Cincinnati, OH 45209

513-979-2163 | 513-979-2166 (f)

www.sigmaresearch.com

© Sigma Research Management Group, Ltd. 2010

We listen between the lines