MWI realizes that tabulation results are not necessarily all-telling. For example, a mean response score of 3.00 on a question may reflect a high tally of “3” responses, or contrarily, a great number of “2” and “4” responses that average 3.00. As a supportive service, our analytic staff can look at the distribution of responses for all questions, and for those questions with a wider range of responses, MWI can conduct a Respondent Profile Analysis, citing the differences between high and low response groups, across the various consumer/business dimensions. For consumer respondents, these dimensions could be demographics, and may also include customer based groups (customer vs. non-customer, customer-segment, etc.). For business respondents, more relevant dimensions include number of employees, net assets or standard-industry classification codes (SIC codes), and may also again include customer based groups.
MWI can also analyze the pattern of all question responses, and measure to what degree responses run in parallel with other question responses. Statistically, we term question pairs experiencing this phenomena “highly collinear”. We feel our clients should be concerned when responses to question pairs are “highly collinear” – this is an indication that there may not be any incremental insight by having both questions in the survey, as opposed to just one. In response to this MWI can provide clients a Question Collinearity Analysis, rank ordering the question pairs most “highly collinear” in the survey. Clients typically review the top ranked question pairs more closely, and look to modify one or both questions, or eliminate one of the questions entirely, when preparing next year’s survey.
Quite often we are asked to determine the appropriate sample size based on study parameters. Our experience in conducting Sample Size Determination exercises is extensive. Different statistical techniques and research designs will require the employment of an appropriate sample size formula, but all sample size computations encompass these four basic components:
- The sample size
- The degree of confidence, or significance level
- The anticipated effect size
- The population variance
When three of these components are known, the fourth can be computed, and this aspect can be leveraged to provide actionable information to our clients. For example, the study’s budget may limit the maximum final sample to 500 respondents – the client would then be interested in what the minimum detectable effect size would be for this sample.
Unlike other Market Research Vendors, we consider the important differences between initial & final sample sizes, and can provide estimates for each, based on study incidence and refusal rates. And of course, MWI realizes the world of Market Research is not a perfect world, with data for partial-fill response records handled accordingly through response imputation if required, and sampling bias dealt with via sample weighting techniques.
As previously mentioned, MWI stays on top of the latest and most innovative approaches to conducting research and analyzing research data. We work in conjunction with our account teams to identify potential areas of improvement on the front-end of the study, during research design formulation and questionnaire development. One of the innovations the analytic group is currently promoting is the design of survey questions based on Choice Task Methodology. Traditional attribute response questions are framed using classic Likert scales, depicting degrees of relative importance or agreement on a 1-5 scale. Choice Task Methodology uses a different approach – respondents directly compare a set of attributes, and denote the single most important and least important attribute within the set. An example appears below:
Analytically, we prefer the use of Choice Task Methodology, because forcing respondents to make choices and tradeoffs amongst a set of attributes:
- Provides better discrimination ability (when computing derived importance maps, discriminant analyses, etc.)
- Provides relative proportions on how much more important one attribute is over another (i.e., “taste of chicken” is twice as important as “convenience of location”).
- Supplemented by demographic profiling, provides improved identification of market segments that have differing attitudes/perceptions on an attribute.
Use of Choice Task Ratings is not advocated for all client studies. It is not recommended for clients with long-standing surveys, for which year over year comparisons or trend analyses are conducted. It does make sense when attributes on the survey can be described by brief, concise labels, and when the maximum number of attributes rated would be 12-16.
Click here to open a full description all the most common StatWork™ offerings. (.pdf file)
Contact us today at 770-449-6767 to find out how StatWork™ can work for you.
The Marketing Workshop, Inc…
…Research that Works |
|
