The scientific method is by definition based on facts. Observations are
made as objectively and precisely as possible. The methods are supposed to
be described in as detailed a way as is needed for another person skilled
in the area to be able to repeat the same study. Naturally, where human
greed and pride are involved, games are sometimes played but when
it is important, the person reporting the results has to allow competitors
into his or her laboratory to see exactly how things were done. Otherwise,
that person loses standing in the scientific community. And, if the same results cannot be
obtained by other people, the results are not accepted and the person loses standing.
(See the history of cold fusion to see how this works.)
Shouting and insults as methods of persuasion have always been allowed but
they provide only color and interest for observers.
Albert Einstein, for example, whose theories of special and general
relativity have proved correct (so far) despite almost a century of
that continues to this day, is said to have remarked
that he did not believe that "God plays dice with the universe," meaning
that he did not accept the probabilistic nature of quantum mechanics. (He
apparently believed that, if the underlying physics were understood better,
what appeared probabilistic would turn out to be deterministic, as
Dr. Stephen Hawking.) However, quantum mechanics, as bizarre as it
seems to be, does in fact seem to explain correctly how, for example,
modern electronic devices work. No one has found an underlying
deterministic mechanism to account for it, and no one would reasonably
assert that quantum mechanics could not be correct because a noted
authority (Albert Einstein) did not accept it. The same reasoning applies
to all aspects of scientific discourse.
Cholesterol. A natural sterol present in all animal tissues. Free cholesterol is a component of cell membranes and serves as a precursor for steroid hormones (estrogen, testosterone, aldosterone), and for bile acids. Humans are able to synthesize sufficient cholesterol to meet biologic requirements, and there is no evidence for a dietary requirement for cholesterol.
Blood Cholesterol. Cholesterol that travels in the serum of the blood as distinct particles containing both lipids and proteins (lipoproteins). Also referred to as serum cholesterol. Two kinds of lipoproteins are:
High-Density Lipoprotein (HDL-cholesterol). Blood cholesterol often called "good" cholesterol; carries cholesterol from tissues to the liver, which removes it from the body.
Low-Density Lipoprotein (LDL Cholesterol). Blood cholesterol often called "bad" cholesterol; carries cholesterol to arteries and tissues. A high LDL-cholesterol level in the blood leads to a buildup of cholesterol in arteries.
Dietary Cholesterol. Cholesterol found in foods of animal origin, including meat, seafood, poultry, eggs, and dairy products. Plant foods, such as grains, vegetables, fruits, and oils do not contain dietary cholesterol.
Poultry. All forms of chicken, turkey, duck, geese, guineas, and game birds (e.g., quail, pheasant).
Lean Meat & Lean Poultry. Any meat or poultry that contains less than 10 g of fat, 4.5 g or less of saturated fats, and less than 95 mg of cholesterol per 100 g and per labeled serving size, based on USDA definitions for food label use. Examples include 95% lean cooked ground beef, beef top round steak or roast, beef tenderloin, pork top loin chop or roast, pork tenderloin, ham or turkey deli slices, skinless chicken breast, and skinless turkey breast.
Processed Meat & Processed Poultry. All meat or poultry products preserved by smoking, curing, salting, and/or the addition of chemical preservatives. Processed meats and poultry include all types of meat or poultry sausages (bologna, frankfurters, luncheon meats and loaves, sandwich spreads, viennas, chorizos, kielbasa, pepperoni, salami, and summer sausages), bacon, smoked or cured ham or pork shoulder, corned beef, pastrami, pig’s feet, beef jerky, marinated chicken breasts, and smoked turkey products.
Cross-over study. A study in which a group of participants is divided in two or more subgroups that are asked to take different diets or treatments. After a period of time, they are asked to resume their normal diets or to stop the treatments. Then, after a further period of time, each subgroup is given the diet or treatment that had been assigned to one of the other subgroups. In this way, each participant acts as his or her own "control" subject because each person winds up being tested with each of the diets or treatments in the study. In this way, differences in response due to individual variations cancel out to the extent possible.
Lipoproteins. Combinations of a lipid like cholesterol with a carrier protein like LDL ("low density lipoprotein") or HDL ("high density lipoprotein"). The terms "high" and "low" density refer to what is seen when blood is spun in a centrifuge at high speed. The high density particles wind up toward the bottom of the tube and the low density particles wind up toward the top of the tube. Please see the wikipedia articles for more information.
Surrogate endpoint. A variable (such as a type of cholesterol or thickness of the inner lining of the carotid artery) that is known to vary proportionately with the risk of a certain outcome (such as cardiovascular disease).
Ways to get confused by scientific studies
A major source of inaccuracy is simple bias. No matter how well intentioned, someone who has vested interest (such as making money) from a given viewpoint, is likely to find a way to promote that viewpoint. In this article, we have a bias that unprocessed grass-fed beef is healthy. We are trying to be as objective as possible, but it is certainly possible that we have found a way to present a case in favor of this conclusion that is actually not justified by the data. If you think so, do not hesitate to let us know (especially if you can justify your thoughts based on objective data!))
Association is not causation. The fact, for example, that people in countries where animal protein is commonly eaten also have a high rate of heart attacks does not mean that the animal protein is responsible for the rate of heart attacks. The fact is merely an association. More data are needed to show that the relationship is due to causation.
In public health, Bradford Hill's criteria are one of several methods that have been proposed to try to show causation in a given phenomenon. In microbiology, Koch's postulates are more powerful and can be implemented because the proposed causative agents can be identified and isolated. The relationship of diet and cardiac risk factors is intermediate because the proposed causative agents can be identified but assessing their effect on the cardiac risk factors is more ambiguous than is the case with infectious agents.
Lumping multiple groups together does not permit inferences about the groups that were lumped together (fallacy of division). For example, studies cannot be used to learn the effects of eating meat if they do not distinguish between (that is, they lump together) foods that include processed meat with foods that include only unprocessed meat.
Unaccounted confounding factors. This is an important concept because it is much less expensive to do a simple study that does not randomize all important variables, and then "mathematically adjust" the data to correct for the known variables that were not randomized, than to do a fully randomized clinical trial. The problem with the less expensive approach is that not all important variables may be known, and the selected mathematical adjustment may not be the correct one for the given data.
This problem is magnified by "meta-analysis" or "metanalysis", a technique in which many different, unrelated studies that study the same question are lumped together in the hope that their deficiencies will average out and their strengths will emerge triumphant. This technique is much less expensive than doing a fully randomized clinical trial, and in fact is widely used when doing such a trial is impossible to do for various reasons. However, a seminal study published in the prestigious New England Journal of Medicine in 1997 reported that meta-analyses of 12 important clinical questions were wrong about 1/3 of the time as assessed by subsequent large-scale randomized clinical trials of the same questions. Being correct only 2/3 of the time is actually pretty good for a technique in the biomedical sciences, but is not nearly good enough to for the technique to be believed blindly in all cases.
Reproducibility of experiments in the biomedical sciences is (almost) never 100%. Be aware: randomized controlled trials are enormously expensive and time-consuming but are the only known reliable ways to know the truth. Knowlege is achieved in baby steps. For big issues, many identical or very similar randomized controlled trials are needed to make sure we actually know (as best as possible) the truth.
For example, treatment of hypertension reduces the frequency of adverse cardiovascular outcomes in almost all, but not all studies.
This can be seen, for example, in the excellent review by Liu et al. In Figure 4 of that paper, the effects of blood pressure lowering on fatal and non-fatal recurrent stroke are shown by individual study ("trial"), by the type of antihypertensive medication used (diuretic, renin-angiotension system (RAS) inhibitor, and felodipine (a calcium blocker) in the FEVER trial), and all trials combined. You can interpret the results by first locating the vertical line over the "1.0" in the Odds Ratio section of the Figure. This is the line showing the result that shows no benefit with either treatment (drug or placebo). Next, for each trial or aggregate result, locate the horizontal line with a square, circle, or diamond in it that is at the same level. That line represents the 95% confidence interval. By statistical convention, if a result occurs with a probability of 5% or less, it is considered to be significant and not the result of random variation in whatever is being measured. Therefore, by the same convention, any result that lies within the middle 95% is considered due to random variation and is not considered significantly different from the average result. So, if that line crosses the vertical line, that trial or result showed no significant difference from "no benefit either way".
As you look at this Figure, you will see that several trials showed no significant benefit: one of the four diuretic trials, 5 of 6 RAS inhibitor trials, and the one felodipine trial. The aggregate result was not significantly different from "no benefit either way" for the RAS inhibitors and felodipine. However, the aggregate result for all the medications tried was highly signficant and showed, at worst, a 10% benefit with drug treatment. This pattern is not unusual even with large clinical trials, and is the reason that even randomized clinical trials have to be repeated several times to be certain that whatever results are obtained are most likely real and not the result of random, irrelevant, and not reproducible factors.
Amin T and JG Mercer. "Hunger and Satiety Mechanisms and Their Potential Exploitation in the Regulation of Food Intake." Curr Obes Rep. 2016; 5: 106–112.
doi: 10.1007/s13679-015-0184-5. PMCID: PMC4796328. PMID: 26762623.
Appel LJ, Sacks FM, VJ Carey, et al. "Effects of Protein, Monounsaturated Fat,
and Carbohydrate Intake on Blood Pressure and Serum Lipids. Results of the OmniHeart Randomized Trial"
JAMA, November 16, 2005—Vol 294, No. 19 2455-2464.
William E. Boden, M.D.; et al. (April 2007). "Optimal Medical Therapy
with or without PCI for Stable Coronary Disease." The New England Journal
of Medicine. 356 (15): 1503–1516. doi:10.1056/NEJMoa070829. PMID 17387127.
Chiu Y-H, Williams PL, Gillman MW et al. for the EARTH Study Team. "Association Between Pesticide Residue Intake From
Consumption of Fruits and Vegetables and Pregnancy
Outcomes Among Women Undergoing Infertility Treatment
With Assisted Reproductive Technology."
JAMA Intern Med. 2018;178(1):17-26. doi:10.1001/jamainternmed.2017.5038
de Souza RJ, Mente A, Maroleanu A, et al. "Intake of saturated and trans unsaturated fatty acids and risk of all cause mortality, cardiovascular disease, and type 2 diabetes: systematic review and meta-analysis of observational studies." BMJ (Clinical Research Ed.). 351 (11 August 2015): h3978. doi:10.1136/bmj.h3978. PMC 4532752. PMID 26268692. https://www.bmj.com/content/351/bmj.h3978
Grundy SM, Stone NJ, Bailey AL, et al. "2018
AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA guideline on the
management of blood cholesterol: a report of the American College of
Cardiology/American Heart Association Task Force on Clinical Practice
Guidelines." Circulation. 2018;. DOI: 10.1161/CIR.0000000000000625. https://www.ahajournals.org/doi/10.1161/CIR.0000000000000625
Hawking, S. http://www.hawking.org.uk/does-god-play-dice.html
Jönsson T, Granfeldt Y, Ahrén B, et al. "Beneficial effects of a
Paleolithic diet on cardiovascular risk factors in type 2 diabetes: a
randomized cross-over pilot study." Cardiovasc Diabetol. 2009; 8: 35. doi:
10.1186/1475-2840-8-35 PMCID: PMC2724493 PMID: 19604407 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2724493/
Klonoff J. "The Beneficial Effects of a Paleolithic Diet on Type 2 Diabetes and Other Risk Factors for Cardiovascular Disease." Diabetes Sci Technol. 2009 Nov; 3(6): 1229–1232.
Lauren E O’Connor, Douglas Paddon-Jones, Amy J Wright, and Wayne W
Campbell. "A Mediterranean-style eating pattern with lean, unprocessed red
meat has cardiometabolic benefits for adults who are overweight or obese in
a randomized, crossover, controlled feeding trial." Am J Clin Nutr
2018;108:33–40. doi: https://doi.org/10.1093/ajcn/nqy075
Liu L, Wang Z, L Gong, et al. "Blood pressure reduction for the
secondary prevention of stroke: a Chinese trial and a systematic review of
the literature." Hypertension Research (2009) 32, 1032–1040. https://www.nature.com/articles/hr2009139
Micha R, Wallace SK, and D Mozaffarian. "Red and Processed Meat Consumption and Risk of Incident Coronary Heart
Disease, Stroke, and Diabetes Mellitus A Systematic Review and Meta-Analysis." Circulation. 2010;121:2271-2283.
Neil A, Cooper J, Betteridge J, et al. "Reductions in all-cause, cancer,
and coronary mortality in statin-treated patients with heterozygous
familial hypercholesterolaemia: a prospective registry study." Eur Heart
J. 2008;2(21):2625-2633. doi:10.1093/eurheartj/ehn422. https://academic.oup.com/eurheartj/article/29/21/2625/530400
O'Connor LE, Kim JE, and WW Campbell. "Total red meat intake of ≥0.5
servings/d does not negatively influence cardiovascular disease risk
factors: a systemically searched meta-analysis of randomized controlled
trials." Am J Clin Nutr 2017;105:57–69. doi.org/10.3945/ajcn.116.142521
Shepherd J, Cobbe SM, Ford I, et al. for the West of Scotland Coronary Prevention Study Group.
"Coronary heart disease with pravastatin in men with
hypercholesterolemia. West of Scotland Coronary Prevention Study Group."
The New England Journal of Medicine. 333 (20):1301–7. doi:10.1056/NEJM199511163332001. PMID 7566020.
Stevenson JL, Paton CM, and JA Cooper. "Hunger and satiety responses to
high-fat meals after a high-polyunsaturated fat diet: A randomized trial."
Nutrition 2017; 41: 14-23. https://doi.org/10.1016/j.nut.2017.03.008
Wang A, Richhariya A, Gandra SR, et al. "Systematic Review of
Low‐Density Lipoprotein Cholesterol Apheresis for the Treatment of Familial
Hypercholesterolemia" Journal of the American Heart Association 2016; 5(7).
Whelton PK, Carey RM, Aronow WS, et al. "2017
ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA Guideline for the
Prevention, Detection, Evaluation, and Management of High Blood Pressure in
Adults A Report of the American College of Cardiology/American Heart
Association Task Force on Clinical Practice Guidelines." Journal of the
American College of Cardiology 2018: 71 (19, May 2018). DOI: