I was recently a guest on my friend Tim Ferriss’s podcast discussing one of my favorite topics, which is how to read and interpret studies. As part of that discussion I referenced our five-part “Studying Studies” series on the topic as a good starting point to understand the limitations of scientific work. Much of the work we focus on in the Studying Studies series is aimed at exposing some of the pitfalls of epidemiology. This might leave you coming away thinking that experiments — that is, controlled experiments with randomization — as opposed to observational studies, are foolproof. But this is not the case. Experiments have a number of hurdles they must jump over in order to establish reliable knowledge:
- Poor design
- Poor execution
- Poor analysis
- Poor interpretation
- And if that is not enough, even if 1-4 are done correctly, a typically scientifically-illiterate media will often layer on their version of some bastardized interpretation in an effort to make the study more appealing to the general population.
Some of these issues were recently put on display when I came across a study titled “Rapamycin impairs bone accrual in young adult mice independent of Nrf2.” This group of investigators tested the impact of rapamycin on the skeletal structure of young mice and found that the mice receiving rapamycin had lower bone density and bone volume compared with control mice. Should this raise concern for those who believe rapamycin is potentially the most geroprotective molecule out there? At first blush, absolutely. Bone health is crucial for healthy aging. Given rapamycin’s remarkable potential to benefit human health, it’s important to understand its potential risks as well.
There are risks with any drug, and rapamycin is no exception. Prior to 2009, rapamycin was primarily used as an immune suppressor for people receiving organ transplants to prevent them from rejecting the foreign organ. But the context — and the dosing — matters. Immune suppression occurs under daily dosing of rapamycin, but under different dosing and scheduling, rapamycin (and its analogs) can actually enhance immune function. I spoke with Lloyd Klickstein and Joan Mannick on the podcast about their work showing mTOR inhibition improves immune function in the elderly.
A closer examination of the recent mouse paper demonstrated a few important details that should weigh heavily on any interpretation, beginning with the age of the mice when treated. To my delight, my friend Matt Kaeberlein (@mkaeberlein on Twitter) had already dug into the study himself and published a detailed and nuanced response on Twitter in the form of a 14-tweet thread, which is reproduced verbatim below (between the section breaks).
I think Matt’s thread is worth sharing for two reasons.
- Understanding the risks of rapamycin is important given that it seems to be the most geroprotective molecule we know about, and
- It’s a great example of a study being poorly interpreted.
§
§
Matt’s thread highlights a very important rule when studying studies. Be skeptical of the evidence and the interpretation in experimental research.
