Why Posting Calorie Counts Will Fail, Part I: The Number Posted is Often Wrong

Jul 14 2010

Introduction to this series here.

image stolen from some article about the new policy that I lost track of because I had 70 tabs open  When you see 450 posted, that might really mean 530. Or more.

Publishing caloric values right on the menu seems straightforward and transparent. The numbers offer what appears to be a simple way to compare items no matter how different they are based on what many people believe is, as Margo Wootan said, the “most critical piece of nutrition information.”  But even setting aside for a moment the issue of whether the number of calories should be the most important factor governing food choices or all calories are equal, there are problems with the numbers themselves.

Give or take 20%…but almost always give

According to a recent study at Tufts where a team of nutrition scientists led by Susan Roberts used a calorimeter to measure the actual caloric value of 39 prepared meals purchased at supermarkets and restaurant chains:

Measured energy values of 29 quick-serve and sit-down restaurant foods averaged 18% more than stated values, and measured energy values of 10 frozen meals purchased from supermarkets averaged 8% more than originally stated. Some individual restaurant items contained up to 200% of stated values and, in addition, free side dishes increased provided energy to an average of 245% of stated values for the entrees they accompanied. (Journal of the American Dietetic Association; full-text is subscription only—here if you have UM library permission)

As Roberts told Time, she decided to do the study because when she was trying to follow the diet advice in her own book, substituting prepared or restaurant meals, “the pounds stopped dropping off. Just as suspiciously, she always felt full” (more on the idea the fullness means a diet must be failing when I get to the issue of why calorie-restriction doesn’t work for long-term weight loss).

It’s worth noting that the results of the study didn’t reach statistical significance “due to considerable variability in the degree of underreporting.” However, they “substantially exceeded laboratory measurement error” and—as noted above—the average discrepancy was 8% or 18% higher, it didn’t even out. However, the average is actually within the Federal regulations—from the same Time article:

Federal regulations are strict about the accuracy of the net weight of a package of prepared food, which must be at least 99% of the advertised weight. When it comes to calories, the count can be a far bigger 20% off. The Federal Government plays no role in checking the calorie claims in restaurants, which means it's up to the states to handle the job — with the predictable patchwork results.

What Roberts’ research suggests is that calorie counts aren’t just wrong, they’re wrong in one direction. As anyone who’s ever tried to count calories knows, a difference of +18% could be devastating to a diet. Say, for example, you think you burn 2000 calories/day, like the supposed average American adult, and you’re trying to generate a ~250 calorie/day deficit through your diet. Assuming you continue to burn 2000 calories/day, that diet should make you lose about 1/2 lb per week or 26 lbs in a year. However, if you were actually eating 18% more calories than the 1750 you’ve budgeted, or 2065 calories/day, and the caloric algebra worked perfectly, you’d gain 6.8 lbs in a year instead.

Even if you’re being reductive, food is more than the sum of its parts

One factor that may work in the opposite direction: the method used to determine the caloric  content of food may systematically overestimate how much energy most people get from some foods. A quick primer on the calorie (most people who are reading this probably already know this, but since lots of people don’t): a nutritional calorie is a measure of the energy contained in food. The base unit, a gram calorie, is the amount of energy required to heat 1 gram of water 1 degree Celsius. A nutritional calorie is a kilocalorie (kcal) or “large calorie” (C), the amount of energy required to heat a 1 kg water 1 degree.

William Olin Atwater c. 1900 from the USDA via the Wikimedia CommonsHere’s the part a lot of people don’t know: the caloric value on labels is calculated according to the “Atwater system” named after the USDA chemist William Atwater, who spent his career burning food and excrement (cue Bevis & Buthead laughter). Based on the formula Metabolizable Energy = Gross Energy in Food – Energy Lost in Secretions, Atwater came up with average energy values for each macronutrient: 9 Kcal/g for fat, 4 Kcal/g for protein, 4 Kcal/g for carbohydrates, and 7 Kcal/g for alcohol. For the purposes of nutrition labeling, even though fiber is technically a carbohydrate, it’s subtracted from the total carb weight before the calories are calculated since it’s not digested.

However, there appears to be considerable variation within macronutrients. Sucrose burns at a lower temperature than starch and isolated amino acids vary in their heat of combustion. Additionally, the Atwater system doesn’t account for differences in how macronutrients behave in when combined—for example, fiber seems to change the amount of fat and nitrogen that turn up in feces, which suggests that its effect on caloric value might not be entirely accounted for by simply subtracting fiber grams from the total carbohydrates. And, as you might expect, “variations in individuals are seen in all human studies” (Wikipedia).

The differences between estimated calories and the actual caloric value (as measured by a bomb calorimeter like the one Roberts’ team used in their study, which still might not correspond exactly to how food is turned to energy in the human digestive tract--I’m not entirely sure how calorimeters account for fiber given that fiber is combustible even though it isn’t digestible) might not be very large—but perhaps more importantly, the discrepancies probably aren’t consistent. The Atwater system is probably more accurate for some foods than others, and seems especially likely to overestimate the energy value of high-fiber foods and distort the differences between starchy and sugary foods.

That might help to explain the discrepancy seen in studies on nuts: in controlled nut-feeding trials, people eating more calories in the form of nuts don’t gain the weight that they should based on their greater energy intake. Additionally, they excrete more fat in their feces (Sabate 2003, American Journal of Clinical Nutrition). This is similar to another issue I mentioned in the introduction—not all calories are the same—but it’s not actually the same problem. Non-random variance in the reliability of caloric estimation means that even if all calories were the same, the numbers on the menus might not be accurate, i.e. the way we estimate calories might not correspond reliably to the amount of energy people actually derive from the food they eat.

So what?

Well, this means that there are (at least) two possible ways that providing consumers with “more information” in the form of calorie counts might actually lead to worse decision-making:

1) Even if people do base their decisions about what to order on the posted calorie counts, they might end up getting many more calories than they want and eating more than think they are.

2) Certain kinds of foods—including high-fiber foods and nuts, which might be “healthier” than items with lower posted calorie counts according to more holistic metrics—might have misleadingly high calorie counts based on the Atwater system. That could dissuade customers from ordering them or restaurants from offering them in favor of less “healthy” foods that may  have lower counts based on the Atwater system but actually provide more energy.

NOTE: Comments disabled due to spam. Feel free to e-mail me with responses.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

When the next diet craze happens

"people eating more calories in the form of nuts don’t gain the weight that they should based on their greater energy intake. Additionally, they excrete more fat in their feces"

"Stuff yourself with nuts and leave the fat behind! Literally!"

we'll know who started it...

On item 2, you yourself said

On item 2, you yourself said that the fiber was removed before the calories were calculated. And according to the Wikipedia article on food energy, calorie counts on food labels are not determined by bomb calorimeter. Therefore, I discount the argument of high fiber foods being disadvantaged. Note that I'm not saying that all calories are equal - far from it - but I just don't think this is a reason for significant error as you suggest.

As for your point 1, I'm not sure this matters. It's one of the best kept diet secrets that there are mean shifts all over the place here. Metabolisms are individual (not as much as oft cited, I think, depending on one's definition of "metabolism"), digestion is individual. So when you start to control intake, you just have to realize that and realize the absolute numbers don't really matter. All you need to do is correlate rate of change of weight to caloric intake (preferably kept consistent, but weighted moving averaging is also useful), then work to understand the relationship. As long as you're not all hung up about it, the mean shift isn't really any sort of problem. Same as with my bathroom scale or handheld BIA device - I know they both have mean shift, and it doesn't matter as long as they tell me in which direction I am moving. Of course, most of your sites and information sources act as if there is no mean shift, and that one should be able to head to mypyramid.gov, get calorie numbers, run from packages or published numbers, and be good. True? Perhaps most of the time, but not all the time.

Of course I also believe it's best to minimize amount of calories eaten at restaurants and amount eaten out of packages. Plus using mass measures rather than volume measures (for most ingredients, ideally).

One of my favorite principles, "You are an experiment of one." I discount anyone in health/dieting/fitness who does not acknowledge a role for metabolic individuality - as well as lifestyle (whatever that means) individuality.

subtracting fiber is not sufficient

"according to the Wikipedia article on food energy, calorie counts on food labels are not determined by bomb calorimeter"

Yes, I know. As I explained, calorie counts on food labels are determined using Atwater estimates. What I'm trying to explain in this entry is how the Atwater system is flawed--which is not at all a revolutionary argument. Most people who've looked at it agree that it's inadequate, but coming up with a more accurate system would be very difficult. The problem is not that Atwater estimates don't account for the fact that fiber is indigestible, it's that they don't account for how the presence of indigestible fiber in the digestive process might affect how much energy the body gets from other molecules--like fat. Thus the discrepancy in nut studies.

Bomb calorimeters are actually considered far more accurate at measuring food energy than the Atwater system. I only said I *don't know* how they account for fiber, not that they don't--they must, somehow. But even calorimeters can't take into account the ways that macronutrients interact in the human digestive system.

Most of the discrepancy in the Tufts study wasn't caused by things like that, but by actual portion sizes differing from the labels--often significantly and usually in one direction (up). That's why the licensed nutritionist attempting to follow her own diet plan using the calorie information provided by restaurants and food labels stopped losing weight (according to her). I'm simply suggesting that I don't expect untrained consumers to do better with the imperfect estimates being printed on menus than a trained nutritionist.

Fair enough

All points taken. I don't understand it either. But being an engineer, I'm all for models, and the simpler the better. I don't really think the Atwater system is flawed so much as that its application is flawed. I understand the logic behind the commerce laws (consumers must receive at least as much as promised), and I also understand the engineering problem that there is variation in every process and item. Given that, as you said, it would be difficult and complex to create another model, I think we should run with what we have, not discount it altogether. I don't really have a great answer in combining the two ideas above - other than the bomb answer that we already know - don't eat out or from packages very much.

I can tell you that I'm taking part in a large, well-known site which encourages exercise and calorie counting. By tracking my intake and adjusting, I and many others I know (mostly virtually) are able to control our weight. I know of at least one woman and one man who have deprecated calorie counting and gone back to some level of intuitive eating - but still monitor their weight and (I'm sure) will track if necessary. I think this is a well-worn path, one that is probably better described by Skwigg ( http://www.skwigg.com/ ) than I ever could. The guy has even become a foodie - this is the guy who runs Second Helping.