Dr. Tim Says....

Lying With Statistics: Kellogg's Does It So Well

I'm a fan of breakfast. Those of you who follow Dr. Gourmet know how important breakfast is. In fact, I think that breakfast is so important that I made it the first chapter of my new book, Just Tell Me What To Eat.

I am a fan of having cereal for breakfast and this may come from my parents. They were eating healthy cereals before it was really fashionable. There were always things like Shredded Wheat, Grape Nuts and 100% Bran Flakes around the house. I loved the large shredded wheat biscuits and when I was a teenager thought that they were quite cool to eat in that quirky way of "I'm eating large wheat biscuits so I must be a hipster." They really are, however, quite tasty and when the bite sized version came along I would have a bowl of them as a snack, thinking that was pretty hip too.

Those were the smaller Nabisco version and it wasn't until later that Kellogg's started making Frosted Mini-Wheats. Since then they've made quite an industry of the larger than bite sized and smaller than big biscuit version that they coated with sugar and began marketing to moms as a "healthy" choice for their kids. Certainly, shredded wheat can be a great choice for you and your kids, with around 6 grams of fiber in a serving, but the pre-sugared ones include about 4 teaspoons of sugar. While I do put sugar on my shredded wheat, I only add about a teaspoon.

And that's where the problem is: a little sugar in your cereal is OK - but not 4 teaspoons.

Now, Kellogg's makes a fair number of cereals that contain a lot of sugar and they work hard to market them as healthy. With their Frosted Mini-Wheats they've gone over the top, however. They have been making claims in their advertising (directed at moms) that a breakfast of Mini-Wheats helps keep kids focused in school. They cite "research" that proves this, but I dug deeper and found their "study" design:

How were the kids tested?

The study was conducted by an independent research group and then reviewed by third-party cognition experts. A series of cognitive tests were conducted in 8 - 12 yr old children from various backgrounds. Children were tested prior to eating breakfast to get a base measurement. Then, children were either provided a breakfast of Kellogg's® Frosted Mini-Wheats® cereal or water. Next, the children were given a series of tests (the same tests and measurements as prior to the breakfast) each hour for three consecutive hours. The results were taken for three hours after breakfast since this is most likely when children may start to feel hungry, which may lead to distraction and decreased ability to retain and recall what they are being taught.

I really love the audacity of this. "We tested giving kids a cereal with 4 teaspoons of sugar against NOTHING." I laughed until I almost cried. Using the same study design we might be able to test other "healthy" foods like an Egg McMuffin, a Coca Cola, Oreo Cookies or cotton candy and get the same results!

I can see the ads now: "Our study shows that your child will be more attentive in school if you feed him a Milky Way candy bar for breakfast."

This is what is known as lying with statistics.

It is possible to design almost any study to get the outcome that you want, and this "research" that Kellogg's carried out is a great example. For instance, I think we could prove that people like being hit in the head with a ball peen hammer (if we compared it to being stabbed with kitchen knife), but that doesn't mean that people LIKE being hit in the head with a hammer. This would be a great study for the Stanley Tool Company, but not one that the Chicago Cutlery company would be quite so happy about.

Me? I'm not happy about any of them. Good research is designed with reasonable controls, subjected to scrutiny prior to collecting data and then peer reviewed. In the world that I work in, higher education, universities have a panel known as the Institutional Review Board (IRB). It is the IRB's responsibility to ensure that research is well designed, not harmful and ethical.

The report by Kellogg's is NOT a clinical study and the fact that they don't name the "independent research group" is telling. Likewise, there's no indication that this was submitted to any oversight. I don't believe that any institution would approve this "study" on children because we already know from (good, well-designed) research that kids do better when eating something for breakfast rather than just having water. Likewise, this was not submitted to an unbiased source for peer review.

A poorly designed study that is unethical and not subject to oversight is not research but a perfect example of how to lie with statistics.

I actually like Kellogg's and many of their cereals. I think that they have some good products, but it's a shame that they feel that they have to lie to their customers to sell their cereals.

I do believe the people who designed this "study" and carried it out must be members of their marketing and advertising department. I wonder how well they sleep at night. They are smart enough to know what they are doing, but they do it anyway.

Must be a tough way to make a living, being such a scoundrel.