“Many a small thing has been made large by the right kind of advertising.”
Mark Twain
… Well then, check out some of the many articles floating around (here and here , for example) that depict this “fact.” Apparently, a group called the Innovation Center for US Dairy conducted a survey of 1,000 “representative” adults in the U.S. and this was one of the results. The same study also “found” that 48% of U.S. adults do not know how chocolate milk is actually made, (I’ll leave that little factoid for someone else to expound upon.)
Now, you may be asking yourself why I keep putting words in “quotes” when introducing this topic. One reason I’m doing so is that I’ve yet to come up with a better way for inserting “air quotes” into the written word. OK, I’ll stop with that now (it’s even starting to bother me!) The other reason is that I really don’t know what to make of this study. Try as I might, I have not been able to find the verbatim question that was asked which resulted in this 7% result. Was it open-ended? Was it multiple choice? What were the options to choose from? Consider the following possible ways this question could have been presented to the audience:
“Please tell me, in your own words, where does chocolate milk come from?”
“Which of the following responses best depicts where chocolate milk comes from?” Answers could have included any combination of the following:
– Mars
– Brown Cows
– Chocolate syrup mixed with white milk
– Mom
– Other
You get the idea. As time has passed from this study being released, we’re starting to learn that the study was meant to be more lighthearted in nature than scientific . And more for PR motivations than anything else. Well, that’s great, but that certainly isn’t how it was reported on from the outset.
So, where do I start with the implications and lessons for those of us in the marketing research industry? I’ll touch on one or two…
First of all, how you ask a question in a survey is huge. Yes, we all know this, but I don’t think we typically scrutinize this aspect of study design as much as we should. One of my favorite questions to ask is, “What would have to be true for this to be the result?” This is a good place to start when crafting a questionnaire. Think ahead about how the results/responses to the question will be interpreted after the fact. Will the question be taken seriously? Have we assumed all the possible response options (if closed-ended)? Have the preceding questions created an unintentional bias in the answers? Lots to think about. And I think the vast majority of the time we don’t think about it and just plow forward.
How many times have you looked at your own research – or someone else’s – and criticized it after the fact for the way a question was asked, or not asked. It happens. A lot. Mostly because we want to get the study in the field quickly and we don’t take the time to “critique-proof” the research. There is much more that could be said about the wording of questions here but I think you probably understand my point, so I’ll leave it at that.
The other lesson I’ll mention briefly involves the motivation for the research. Let’s not try to be cute with our results. When reporting on results, be clear from the outset concerning the underlying reason for the study. Now, that doesn’t mean that you disclose your or your client’s strategies or tactics, but if the intent is to be “lighthearted”, let that be known. You may not be able to completely control the message as it goes out, but the onus is ultimately on the entity commissioning the research in the first place to do just that. I think an organization that plays these games with research (like allowing public misperceptions to persist or not releasing the actual questions asked) mostly hurts their reputation, only. However, I feel it also harms the reputation of the marketing research industry, in general, and for that I do take issue.
To be clear, I am not at all opposed to having a little fun with things like this study. What I don’t like is playful advertising with a distinct agenda being presented as “real” research (sorry, there I go again.) In the current environment of “fake news” and “statistics can be made to say anything”, we as marketing researchers need to steward our reputation as best we can. I don’t think being fast and loose with “research” results is entirely helpful to our cause.
Now, the question I really have is… Where can I find those green cows that produce mint chocolate chip ice cream? Ireland, maybe? Anyone?
~ Marketing Workshop