Yesterday’s headlines read “Stanford Scientists Cast Doubt on Advantages of Organic Meat and Produce” (NYT), “Why Organic Food May Not Be Healthier for You” (NPR), “Organic Food No more Nutritious than Non-Organic,” (MSNBC) and “Little evidence of health benefits from organic foods, Stanford study finds” (Stanford School of Medicine News). In other words, if you’re looking for nutrition benefits from organic food, don’t bother.
Being inquisitive by nature, I read beyond the headlines and found some gems, including the following, which I took straight from Stanford School of Medicine’s web site, with no filter through news sources that may put their own spin on it:
They did not find strong evidence that organic foods are more nutritious or carry fewer health risks than conventional alternatives, though consumption of organic foods can reduce the risk of pesticide exposure. [Oh. Well. Is that all?]
Although there is a common perception — perhaps based on price alone — that organic foods are better for you than non-organic ones, it remains an open question as to the health benefits.
Wait! I thought their study found that there was “little evidence” of health benefits (according to Stanford’s headline). Apparently, it’s an “open question.” Hmmmm.
And, most tellingly, this:
There were no long-term studies of health outcomes of people consuming organic versus conventionally produced food; the duration of the studies involving human subjects ranged from two days to two years.
In other words, the meta-analysis tells us absolutely NOTHING about the long-term effects of consuming conventional food vs. organic food. The LONGEST studies analyzed were merely two years long. Most were much shorter – down to two days long. What can we really tell about the health benefits of food in two days? Even two years is very short in terms of human lifespans.
Look, I’m not a doctor, scientist, or nutritionist. I don’t even play one on the Interwebs. I am an attorney. As such, I am trained to read things like statutes, contracts, and news stories very carefully. So, my purpose here is not so much to challenge the outcome of this metastudy or the methods used in the studies underlying it (although I’m skeptical of metastudies as a matter of principal). Rather, my purpose is simply to read the story, as reported by Stanford Medical School and others, with an eye toward understanding what they are really saying, determining whether it has any meaning, and deciding whether the real news has any relationship to the headlines.
After reading a bunch of takes on this study, I’ve come to the conclusion that it is absolutely meaningless. Here’s the best they could do:
After analyzing the data, the researchers found little significant difference in health benefits between organic and conventional foods. No consistent differences were seen in the vitamin content of organic products, and only one nutrient — phosphorus — was significantly higher in organic versus conventionally grown produce (and the researchers note that because few people have phosphorous deficiency, this has little clinical significance). There was also no difference in protein or fat content between organic and conventional milk, though evidence from a limited number of studies suggested that organic milk may contain significantly higher levels of omega-3 fatty acids.
Stanford’s story goes on to admit (albeit with dismissive and minimizing language) that there were differences found even in the short term (remember, no study surveyed in their metastudy was more than two years):
Two studies of children consuming organic and conventional diets did find lower levels of pesticide residues in the urine of children on organic diets, though the significance of these findings on child health is unclear. Additionally, organic chicken and pork appeared to reduce exposure to antibiotic-resistant bacteria, but the clinical significance of this is also unclear.
I wonder what differences we’d see over 10, 20, or 30 years of consistent exposure to chemicals in conventional foods.
From The New York Times:
The organic produce also contained more compounds known as phenols, believed to help prevent cancer, than conventional produce. While the difference was statistically significant, the size of the difference varied widely from study to study, and the data was based on the testing of small numbers of samples. “I interpret that result with caution,” Dr. Bravata said.
So, the doctor who conducted the metastudy interprets “with caution” the statistically-significant result that organic food contains more of the good stuff, yet exercises no caution in reporting that there is “little evidence” of any health benefits to consuming organic food? I definitely support exercising caution in reporting results, but can’t figure out why that same caution was not exercised in crafting the headline that reported that there is “little evidence of health benefits” to consuming organic food. And, no, saying there is “little evidence” of something, then stating only one side (health benefits) is not the same as reporting no findings from the study. If that had been their point, they could as easily have said there is “little evidence” that organic food is not more healthful than conventional food.
Buried way beneath the headlines – only two paragraphs from the end of Stanford’s story, is a discussion of the limitations of the study:
In discussing limitations of their work, the researchers noted the heterogeneity of the studies they reviewed due to differences in testing methods; physical factors affecting the food, such as weather and soil type; and great variation among organic farming methods. With regard to the latter, there may be specific organic practices (for example, the way that manure fertilizer, a risk for bacterial contamination, is used and handled) that could yield a safer product of higher nutritional quality.
Any organic farmer could have told them that. But a metastudy of nutritional benefits of organic foods (aggregated) vs. conventional foods (aggregated) tells us nothing about the differences between the health benefits of food grown in various types of soil.
The New York Times also buried its reporting of the benefits of organic food deep in its news story:
Over all, the Stanford researchers concluded that 38 percent of conventional produce tested in the studies contained detectable residues, compared with 7 percent for the organic produce. (Even produce grown organically can be tainted by pesticides wafting over from a neighboring field or during processing and transport.) They also noted a couple of studies that showed that children who ate organic produce had fewer pesticide traces in their urine.
. . .
Similarly, organic meat contained considerably lower levels of antibiotic-resistant bacteria than conventionally raised animals did, but bacteria, antibiotic-resistant or otherwise, would be killed during cooking.
This tells us two things: 1) conventional farming methods can taint organic farms so that no farm is completely free of pesticides, despite best efforts; but 2) those best efforts pay off nonetheless. In fact, if this were reported on in the same manner such things are usually reported when the author wishes to prove a point, the headline would have read, “There are 450% more chemical residues in conventional produce than organic.” Because that is actually what the study shows.
Not only did this study determine absolutely NOTHING about the comparative health benefits of organic vs. conventional food, but the researchers’ own headline reporting on their conclusions was misleading, at best: “Little evidence of health benefits from organic foods, Stanford study finds” (Stanford School of Medicine News). It’s almost as though they felt compelled to report something in order to justify their expenditure of time and resources.
Further, despite use of “health” in the headline, the report dealt only with the “nutritional” differences between organically-grown and conventionally-grown food – and THAT, only on the very short term. It did not consider the other advantages to organic food, as acknowledged by its author:
Dr. Bravata agreed that people bought organic food for a variety of reasons — concerns about the effects of pesticides on young children, the environmental impact of large-scale conventional farming and the potential public health threat if antibiotic-resistant bacterial genes jumped to human pathogens. “Those are perfectly valid,” she said.
The analysis also did not take factors like taste into account.
But if the choice were based mainly on the hope that organic foods would provide more nutrients, “I would say there is not robust evidence to choose one or the other,” Dr. Bravata said.
In the final analysis, after we get WAY past the headlines, this whole thing is really much ado about absolutely nothing. By admission of the study’s own authors, nothing was found.
I’m sure there are sound science-based responses to these headlines, too, and I’d love to hear about them in the comments below. My role was simply to look at what was actually reported vs. what the headlines said and pull the buried lead up to the light of day.
- Amy Salberg, The Real Food Lawyer