As online survey software becomes more sophisticated, organizations are increasingly turning to web-based research as a viable business intelligence tool. That can be a wise decision, as under the right circumstances it can deliver timely and cost-effective results. However, carefully consider all the factors before deciding if an online methodology will effectively meet your research objectives.

Can we access the audience for an online quantitative study?
In many industries, it can be very difficult to rent reliable email lists as compared to phone lists. In addition, the available lists may not be representative of the entire population you need to survey. Furthermore, given that many people won’t respond to the survey request, the number of available email addresses may not be large enough to deliver a statistically valid sample. However, if you’re conducting a customer or employee survey and have an in-house database with email addresses, an online survey may indeed be the best approach.
�
Is a blind survey the best approach?
For many types of research, such as brand awareness and competitive analysis studies, disclosing the research sponsor upfront can negatively impact people’s willingness to participate or, equally detrimental, bias their responses to questions. At the same time, response rates to unsolicited blind email surveys can be extremely low, as the email must first make it past spam filters and then compete with many other emails for the recipients’ attention. As a result, online surveys are more successful when the sponsor’s name can be revealed and when they’re targeted at customers, employees, or other groups that are already familiar with that company.
�
What’s our best option for obtaining qualitative data?
If your research objectives call for qualitative exploration and conducting in-person focus groups is a viable option, by all means do. Important insights are often gleaned from the immediate reactions and interactions, eye contact, facial expressions, tone of voice and other cues that only a live focus group can deliver. However, in some cases it’s not possible to recruit for and conduct focus groups with people spread across the country. Or, for example, if the audience is made up of high-level executives, they may be unwilling to travel to a focus group facility at a specific time. In these cases, the next best choice is in-depth interviews (IDIs) conducted either in person or via telephone. Though IDIs don’t offer the interactivity of a focus group, they still provide rich qualitative data.

Certainly, the technology exists to conduct online focus groups, which typically take place over several days in a dedicated chat room with a professional moderator. However, virtual focus groups aren’t optimal in business-to-government and business-to-business markets, as most professionals are unwilling to log in multiple times over numerous days to participate in a discussion.

Is it an either/or decision between phone and online?
No, sometimes a hybrid approach is optimal. For example, phone is often mandated by the fact that the study requires sample and quota control to meet the objectives (i.e. perhaps the marketer needs to talk to people within a certain spending range). In these instances, researchers can give people who decline to participate on the phone the option of receiving the survey via email to complete at their convenience. This combined approach often lifts response rates. Or, by directing willing participants to a website while on the phone with them, researchers can measure effectiveness of advertising and direct marketing creative treatments with a statistically valid sample. Finally, because some people tend to not be thorough when completing an online survey, researchers can improve the data quality by calling those individuals for clarifications or further input.
Keep in mind that “online” doesn’t mean “do it yourself.” Making important decisions based on the findings of poorly designed and/or executed research - regardless of the methodology - can have costly consequences. So, even if you’ve bought that online survey software that sounds so easy to use, always consult an experienced research professional for guidance before conducting your study in-house.

When preparing to conduct a research study, your research provider may use some terminology with which you’re unfamiliar. Here are explanations of some of the most commonly used terms and concepts in the research business.

Quantitative Research
Quantitative studies collect statistically valid data from large respondent samples. Usually conducted via telephone, mail, or email with a carefully crafted survey instrument, their primary function is to confirm or disprove preliminary assumptions, insights, or ideas to drive informed decision-making and appropriate courses of action. Unlike other forms of research, the numerically calculated results of a quantitative study with a large enough number of respondents can be projected onto the market as a whole with an acceptable level of confidence.

Qualitative Research
On the other hand, qualitative studies, typically conducted via focus groups or in-depth interviews, explore participants’ emotional and rational reactions to, perceptions of, and attitudes toward a concept, product, or service. They can also reveal in-depth information on the product/service features most likely to drive increased usage. The exploratory nature of qualitative studies makes them valuable for developing initial insights and providing direction for further research. Due to the large amounts of open-ended data collected from smaller subsets of people, qualitative research requires a subjective interpretation of the data, which cannot be meaningfully quantified.

Population
The word “population” refers to the entire universe of people targeted for the research. In short, it’s your target audience for the study. It’s often necessary to collect different information from different segments of the population, such as varying titles, decision-makers, influencers, and such.

Sampling Frame
A sampling frame is a group that is representative of the population to be surveyed. For example, let’s say you want the opinions of federal IT decision-makers. A possible list source would be a federal government publication subscriber list, such as FCW. The subscriber list would be your sampling frame that you use to make inferences about the federal decision-maker population.

Sample Size
Sample size refers to the number of respondents in the sample who actually participate in a survey. This number is very important in determining the confidence interval in quantitative research studies.

Confidence Interval
You may recall hearing the plus-or-minus margin of error figure when opinion poll results are reported in the news. Derived from a formula based on the size of the total population as compared to the size of the sample surveyed, it defines how accurately the research results can be applied to the study’s population as a whole. For example, let’s say out of a total population of 25,000 federal program managers, you survey a small subset of 350, giving you a margin of error of +/- 5%. If 70% of the 350 people interviewed say they prefer X provider of networking services, given the margin of error of +/- 5%, the confidence interval tells us we can be reasonably confident that the preference of the entire population will fall between five percentage points above or below the preference rate of the sample. Therefore, the confidence interval indicates you can be reasonably confident that anywhere between 65% and 75% of the entire population would report the same preference of networking services if surveyed.

Screener/Filter Questions
Screeners or filters are questions that qualify respondents for subsequent questions or ensure that the survey is within their realm of experience. For example, if you are attempting to reach federal decision-makers, you should have a screener question that ensures you’re speaking to a federal government employee and that the person has at least some level of decision-making responsibility relevant to your offering.

Unfortunately, the federal government has yet to embrace teleworking programs. In its April 30 issue, FCW features a commentary on the fact that teleworking has not taken hold to the level many anticipated three to five years ago. In her article, The Telework Dance, Judy Welles cites the results from two different studies that point to a primary reason – lack of support from management. Our 2006 Telework Study, released just about a year ago, revealed similar feelings among federal workers. This resistance still exists despite legislative support for telework presented in a new bill sponsored by Sens. Stevens (R-AK) and Landrieu (D-LA).

Clearly, the potential big-picture benefits of teleworking are significant: improved workforce flexibility and morale, a cleaner environment, and reduced traffic and gridlock. However, these effects don’t readily make a strong enough business case for teleworking. Loss of management control, trust, and office politics all play a role in management’s uneasiness with the telework concept.

I always come back to this question: what can the contractor community do to help ease the skepticism of government managers and promote adoption of teleworking? I believe that contractors can and should help their government management counterparts understand the available technical tools that enable effective monitoring and management of remote workers as well as those that enable dispersed teams to effectively collaborate and interact. Contractors can also share teleworking success stories from government agencies and the private sector. Understandably, most decision makers appreciate hearing about the experiences and lessons learned of others that have adopted a new idea, technology, or program.