Rigor Mortis

This book is a real black pill (a grim truth) for those who are optimistic about medical science delivering cures to disease.

Too often we imagine that scientists and the institutions of medical science are like the benevolent clergy of the church and idealized organized religion - that they are on an uncorrupted, principled mission with the purest of intentions to uplift humanity, empowering us all with the truth we need to live better. Unfortunately, this is pretty far from reality, often scientists and especially the institutions of science, are more like politicians and the institutions of politics. They are motivated a lot more by selfish personal enrichment and self-aggrandizement than an unflappable devotion to the truth. When they learn that they are wrong, they double down on their assumptions, retreat into dogmatic positions, and staunchly ignore evidence of their errors instead of correcting the record - this lack of humility has an astronomical civilizational cost to all of us.

But don't despair! In this book review, I share some guidelines for biohacking in this era of bad science.

Rigor Mortis

The book's title is a clever double entendre, the death of the rigor in science causes actual death.

It also turns out that scientists have been taking shortcuts around the methods they are supposed to use to avoid fooling themselves. The consequences are now haunting biomedical research. Simply too much of what’s published is wrong. (p. 1)

Scientists often face a stark choice: they can do what’s best for medical advancement by adhering to the rigorous standards of science, or they can do what they perceive is necessary to maintain a career in the hypercompetitive environment of academic research. It’s a choice nobody should have to make. (p. 3)

The much harder challenge is changing the culture and the structure of biomedicine so that scientists don’t have to choose between doing it right and keeping their labs and careers afloat. (p. 4)

Consider the story of Charles Darwin, as he developed his theory of evolution through natural selection. That discovery became the organizing principle of biology. And the story of how it arose bears almost no resemblance to the way biology and medicine advance today. Darwin spent decades gathering observations and gathering his thoughts. He studied odd little finches in the Galapagos Islands. He pored over collections of insects. Barnacles held his interest for nine years. He spent decades breeding pigeons and soaking seeds in saltwater to see if they could survive long ocean voyages and take root across the sea. He didn’t start out with a coherent hypothesis; he was simply driven by curiosity. In fact, today’s science institutions would reject his approach... (p. 169)

 The Funding Fiasco

Unsurprisingly, the big problem with bad science is the money.

The ecosystem in which academic scientists work has created conditions that actually set them up for failure. There’s a constant scramble for research dollars. Promotions and tenure depend on their making splashy discoveries. There are big rewards for being first, even if the work ultimately fails the test of time. And there are few penalties for getting it wrong. (p. 12)

As of January 2016, researchers must take some basic steps to avoid the most obvious pitfalls. When applying for a grant, they need a plan to show that the cells they are using are actually what they think they are (this is not a trivial issue, as we shall see). They need to show they’ve considered the sex of the animals they will use in their studies. They need to show that they’ve taken the time to find out whether the underlying science looks solid. And scientists must show in their applications that they will use “rigorous experimental design.” Researchers are supposed to be held accountable for all this during the annual reviews of their grants. (p. 60)

Many academic scientists already spend more than half their time writing grant proposals, and because money is so tight, most of those don’t get funded. (p. 61)

Labor economist Paula Stephan at Georgia State University likens it to a shopping mall: The university owns the building and charges rent; the scientists have become the tenants, spending their grant money on rent as well as research assistants and materials. If they can’t keep bringing in the money, tough. They’re out of business. (p. 190)

This is the classic skin-in-the-game problem that seems to be at the cause of so many of our civilizational problems. In science, almost nobody is spending their own money. $30 - $40 billion annually in funding comes from the taxpayer via the National Institute of Health, which is then dispersed to universities and scientists via grants. Floors and floors of apathetic government employees in the bureaucratic maze of organized irresponsibility stand between the scientists' accountability for results and the taxpayer rendering unto Ceasar.

Congress inadvertently made the problem worse by showering the NIH with additional funding. The agency’s budget doubled between 1998 and 2003, sparking a gold rush mentality. The amount of lab space for biomedical research increased by 50 percent, and universities created a flood of new jobs. But in 2003 the NIH budget flattened out. Spending power actually fell by more than 20 percent in the following decade, leaving empty labs and increasingly brutal competition for the shrinking pool of grant funding. The system remains far out of balance. (p. 189)

This deluge of federal funding creates a serious noise-vs-signal problem in science. Imagine if there were $30 billion in federal funding available to sponsor the work of travel bloggers...

Currently, travel bloggers mostly spend their own money and resources to travel and create content for their audiences and they are very passionate about travel so they create great articles, books, and YouTube vlogs that capture different destinations. But imagine if there were billions of dollars up for grabs for those travel bloggers who could write hyperbolic grant proposals and play the political influence game with the bureaucrats who controlled the purse strings, you would get millions more travel bloggers than the market actually demanded. You would get a bunch of grifters, who didn't care about travel, who were just in it for the money. You would also get a tremendous proliferation of disinformation; do Argentinians prefer to salsa dance or do the tango in Buenos Aires? You couldn't figure it out because there would be so many poorly researched articles on the topic, instead of just a few articles written by those who actually went there and hit the dance floors.

Big pharma funds a lot of science, but they, of course, do so in a very self-interested way to get their drugs to market.

some drug companies had the habit of simply not publishing studies if the results were not favorable to the drug they were investigating. This is known as the file drawer effect because studies end up getting filed away rather than appearing in the scientific literature. (pp. 147-148)

As of 2017, Pubmed requires authors to declare their potential conflicts of interest and it displays them publicly below the study abstract, while this does nothing to inform of unflattering studies that are hidden from the public, it's a good reason to view studies published prior to 2017 with increased scrutiny.

The Replication Crisis

The book describes in depth the reproducibility crisis (as it's alternatively called) in which many published studies' experiment results could not be reproduced by similar labs and scientists duplicating the experiments.

Extrapolating results from the few small studies that have attempted to quantify it, they estimated that 20 percent of studies have untrustworthy designs; about 25 percent use dubious ingredients, such as contaminated cells or antibodies that aren’t nearly as selective and accurate as scientists assume them to be; 8 percent involve poor lab technique; and 18 percent of the time, scientists mishandle their data analysis. In sum, Freedman figured that about half of all preclinical research isn’t trustworthy. He went on to calculate that untrustworthy papers are produced at the cost of $28 billion a year. (p. 14)

In grade school science we learned that reproducibility is what makes science legitimate. The red pill is that there's just not that much reproduction of studies going on. Let's say that you were an author who was passionate about writing and you wanted to get famous by writing a great book. Would you spend a lot of time grammar-checking other authors' books to make sure that their books were perfect? Not unless they paid you handsomely to do so, right? This is at the core of why reproducibility just isn't that great of a priority.

I would propose a little better alignment of the ego incentives. Scientists who labor to reproduce other scientists' studies should get author credits there at the top of the published study - which is sometimes the case, this is why I hold multi-center, multi-university studies in higher regard.

Bias Begets Bad Science

Part of the everyday challenge of research is trying to avoid fooling oneself through bias. Inevitably it creeps into even the best scientific efforts. Bias is often impossible to avoid because it frequently involves pitfalls that scientists simply can’t foresee. So it too is part of the fabric of scientific research. (pp. 40-41)

Reporting bias

Reporting bias is another common problem in biomedicine. Scientists are much more likely to report the results of an experiment that “worked” than one that failed, even though discovering the lack of an effect can be just as important as a positive finding. (p. 41)

Observer bias

Observer bias is another big problem. Scientists pursuing an exciting idea are more likely to see what they’re looking for in their data, and that alone can completely skew the results. (p. 41).

Batch Effect

The samples from the women with ovarian cancer had been run on one day and the samples from the comparison group on another. Apparently there was some subtle difference in how the mass spectrometer operated from one day to the next. The “ovarian cancer” test was really measuring nothing more than spurious signals from the machine. (p. 125)

The batch effect is a stark reminder that, as biomedicine becomes more heavily reliant on massive data analysis, there are ever more ways to go astray. (p. 126)

To quote Richard Henderson of the Medical Research Council Laboratory of Molecular Biology in Cambridge, United Kingdom

“One must not underestimate the ingenuity of humans to invent new ways to deceive themselves,” (p. 192)

HARKing

HARKing stands for “hypothesizing after the results are known.”  (p. 140)

It often starts out in all innocence, when scientists confuse exploratory research with confirmatory research. This may seem like a subtle point, but it’s not. Statistical tests that scientists use to differentiate true effects from random noise rest on an assumption that the scientist started with a hypothesis, designed an experiment to test that hypothesis, and is now measuring the results of that test. P-values and other statistical tools are set up explicitly for that kind of confirmatory test. But if a scientist fishes around and finds something provocative and unexpected in his or her data, the experiment silently and subtly undergoes a complete change of character. (pp. 140-141)

This is why nowadays many trials are pre-registered, this makes studies more credible as the scientists go into the study with a clear hypothesis that the data must prove or disprove. They can't change their hypothesis halfway through the study when some provocative pattern emerges from the data - I'm imagining that this is where a lot of those "New study shows..." mainstream media headlines come from...

The Furry Problem - Animal Studies

Have you ever wondered why so much science is done with animals? Why do so many scientists do studies with rats or mice? According to the author, it's simply because everyone else does studies with rats and mice. 

Scientists most often start a research project by building on what their mentors or peers have done before. An entire field may rely on a particular animal model, even though scientists often have no idea whether it’s a valid surrogate for human disease. It’s quite common to cure a disease in a mouse model, only to discover that it’s irrelevant for treating human disease. And that, for scientists trying to conduct rigorous scientific research, makes mice a big, hairy problem. (pp. 68-69)

Silberberg realized that nobody applied that same degree of care when it came to evaluating the animal studies upon which the human experiments were based. (p. 58). 

“Nobody knows how well a mouse predicts a human,” said Thomas Hartung at Johns Hopkins University. (p. 72)

Animal studies often mislead scientists to waste millions of dollars and years of work because...

At root, the problem is that lab animals aren’t just small, furry humans. (p. 72)

Interesting fact

Scientists have only recently come to realize that the sex of the person who handles the mice can also make a dramatic difference. “Mice are so afraid of males that it actually induces analgesia,” a pain-numbing reaction that screws up all sorts of studies, Garner said. Even a man’s sweaty T-shirt in the same room can trigger this response. (p. 80)

Human trials are better...

Back in the decades when drug development was progressing rapidly, doctors weren’t trying to create new drugs based on a deep understanding of biology. They just experimented on people—not mice—to see what worked. “I wouldn’t necessarily seek to defend the historic approach,” Scannell said. “I think today people would be horrified if they knew how drug discovery really worked in the fifties and sixties. But I also think it is a historical fact that it was an efficient way to discover drugs. It may be an ethically unpalatable fact and something you would never wish to revisit, but I think probably bits of it could be revisited with not huge risk.” (pp. 89-90)

So science could deliver us a lot more useful drugs and cures to diseases if we did more human trials. The reason of course that we do all these animal trials - giving cute little mice cancer and then pumping them full of weird drugs to try to cure cancer - is because it would be unethical to do these experiments with humans. Violating the informed consent principle in human scientific experimentation is why we sentenced 16 Nazi doctors to dangle from short ropes at the Nuremberg trials after World War II. Since the bad science done in animal studies is so costly in billions of dollars and years of work wasted while cures to disease remain undiscovered we have to ask, could there be an ethical way to do risky human experimentation? Well, in the past scientists did a lot of human experimentation on society's unwanted; prisoners and the mentally ill. While I'd prefer society dealt more harshly with and executed a lot more of its criminals, I don't think any of us want to see these sordid episodes of medical history repeated. A free market for human experimentation would go a long way toward solving the furry problem of bad animal studies...

• Big pharma wants to test a new cancer drug but they don't want to waste 5 years doing misleading animal studies.
• Big pharma instead offers a half-million dollars to people who volunteer to be injected with a specific type of cancer that will metastasize and then undergo an experimental treatment.
• Scientists doing risky experiments on humans would probably take their work a lot more seriously than bored postdocs poking their thousandth rat with a syringe.

How many people out there would risk their lives for a half-million dollars? How many people out there are suicidal and are looking for a way to kill themselves? How many people are serving life sentences in jail and would risk their lives in exchange for a fortune paid to their families? More than enough to make some scientific progress. Big pharma has billions and billions of dollars to invest in research and development as does the NIH, it seems like there's enough money to convince people to risk their lives for the sake of science.

Metformin, a controversial yet renowned anti-aging drug, is an interesting case study of science succeeding by way of experimentation with a traditional European herbal medicine...

Some of the most successful drugs are the result of serendipity, as is the case for metformin, the most widely used drug for type 2 diabetes. Decades ago a researcher in the Philippines studying an obscure compound to treat flu and malaria reported that it also seemed to lower blood sugar... The original compound was discovered as an herbal extract, and to this day scientists don’t understand the biological mechanism. But it doesn’t matter. It works. (p. 90)

Metformin

Category: Nootropic Ingredients

In Vitro Invaders

Too often scientists reach conclusions based on experiments with disembodied cells suspended in glass containers, this methodology is seemingly hopelessly problematic.

This immortal cell line, labeled HeLa, was just the first of many. And because HeLa cells grew so quickly, they became rapacious weeds in the world of biomedical research labs. The slightest lapse in hygiene can transfer a HeLa cell from one dish to another that’s harboring a different line. The fast-growing HeLa cells quickly crowd out the other cells and simply take over. (p. 95)

A 2007 study estimated that between 18 and 36 percent of all cell experiments use misidentified cell lines. That adds up to tens of thousands of studies, costing billions of dollars. About a quarter of those misidentified lines are actually HeLa, (p. 96)

You may have heard the fascinating story of Henrietta Lacks' immortal cancer cell line, this woman's cells have just kept growing, conquering the world in their own microscopic way. This cell line contaminates and renders useless many cell line studies.

All told, he figured perhaps 12,000 papers are based on bogus cell lines. But that’s not the end of it. He estimates that, on average, each of those papers was cited in other papers thirty times. “When you start doing the multiplication, we’re talking about billions of dollars that have been spent using a cell line inappropriately.” (p. 103)

It’s not clear how much value comes from research that relies on cell lines in the first place. Much as scientists appreciate the convenience of studying a disease in a petri dish, the results are often hard to apply to human illness. (pp. 107-108)

Chinese studies are (often) crap

He’s found that 70 percent of this substandard genomics work is taking place in China. The studies are being published in English-language journals, he said, “and almost all of them are wrong.” (p. 133)

Chinese scientists get cash bonuses for publishing in Science, Nature, or Cell, and Schekman said they sell coauthorships for cash. That practice would fail the test of scientific integrity in the United States. (p. 178)

The Absurdistan of Academia

The book confirms the stereotype of academia being a byzantine maze of pretend productivity, which hamstrings actual intellectual progress...

Once young biomedical scientists finish their PhDs, they go into a twilight world of academia: postdoctoral research. This is nominally additional training, but in fact postdocs form a cheap labor pool that does the lion’s share of the day-to-day research in academic labs. Nobody tracks how many postdocs are in biomedicine, but the most common estimate is that there are at least 40,000 at any given point. They often work for five years in these jobs, which, despite heavy time demands, usually pay less than $50,000 a year—a rather modest salary for someone with an advanced degree and quite possibly piles of student debt. All this would be worth the sacrifice if a research job were waiting at the end of the process. But the job market in academic research is bad and has been getting far worse. (p. 173)

Giving geniuses the space that they need to invent is the opposite of what modern academia does.

Academia used to encourage that by granting academic scientists sabbaticals once every seven years. That’s still an option, on paper at least, but for many in biomedical research, “that doesn’t work anymore because everyone’s writing grants and everyone’s too stressed out.” It’s just too risky to leave your lab for an academic year, given the struggle to fund labs these days. (p. 166)

Philosophically, I don't have my mind made up on institutions but they seem to mostly be a problem, a big problem. Institutions, like the government or universities, or the NIH seem to be magnets for psychopaths, sociopaths, egomaniacs, and the incompetent who can't cut it in the free market. Institutions are where anti-social narcissists can build political fiefdoms for themselves and get fat off the backs of the tax livestock (that would be us!) providing their federal funding. On the other hand, NASA got us to the moon, the British Empire banished human chattel slavery and CERN invented the internet, to name a few examples of big wins for humanity from institutions.

The Science Publishing Racket

In science, there is this middle-man media industry of science journals, like Nature, Cell, The Lancet, and PLOS One. The incentives in publishing are problematic, to say the least...

“If you think about the system for incentives now, it pays to be first,” Veronique Kiermer, executive editor of the Public Library of Science (PLOS) journals told me. “It doesn’t necessarily pay to be right. It actually pays to be sloppy and just cut corners and get there first. That’s wrong. That’s really wrong.” This perverse incentive is warping biomedical science. (p. 172)

Retractions of studies occur, but not nearly as much as they should.

70 percent of the retractions they studied resulted from bad behavior, not simply error. They also concluded that retractions are more common in high-profile journals—where scientists are most eager to publish in order to advance their careers. (p. 182)

They were flabbergasted to find that some journals demanded payment—up to $2,100—just to publish their letter pointing out someone else’s error. (p. 182)

Consider the case of the recent (hilarious!) hoax paper...
The conceptual penis as a social construct
...that actually passed the peer review process and was published in a mainstream social "science" journal, Cogent Social Sciences.

Many journals like Cogent Social Sciences operate with a pay-to-publish model. The authors of The conceptual penis paid $625, they published under fake names as part of a fake social research group, which Cogent Social Sciences did not catch. So for $625, approximately the same amount you would have to pay to score some cocaine, get drunk on champagne, and bang a hooker in a chintzy hotel room in Vegas, anyone's nonsensical opinions can be transformed into "science".

How many other papers that have appeared within the pages of Cogent Social Sciences are similar pseudo-scientific nonsense? I would wager probably most of them.

The Losing War on Cancer

Looking at the stark statistics western medicine is doing a remarkably bad job of finding a cure for cancer.

A glaring example of the failure of the Western approach to health and disease is the so-called war on cancer in America, where this deadly disease flourishes more than anywhere else on earth. Over the past 35 years, the National Cancer Institute has spent more than $ 20 billion researching a cure for cancer, while suppressing all homoeopathic and preventive approaches to the disease. Since 1962, the number of cancer deaths per 100,000 people in America has risen from 170 to 185, and in 1986 close to 1 million new cases of cancer were reported. (4492-4496).

It would seem to be less of a race for the cure and more of a perverse game of musical chairs, that costs billions of dollars and millions of lives. The risks of "wuwu" holistic cancer treatments look a whole lot better in the shadow cast by the towering mountain of bodies after the 45-year War on Cancer.

“The government, and actually the American people, have suddenly realized that they’re spending a lot of money and cancer isn’t yet cured, so to speak. We bragged that we would cure cancer, and then it turns out we didn’t.” Bourne worries that “everyone suddenly thinks research is terrible and it’s not worth anything.” (p. 195)

To quote author Levi Garraway 

“The reality for most cancer drugs is that most patients don’t respond,” (p. 206)

But there is some reason for hope...

The best clinical studies are now designed and carried out with great care (and at great expense). They involve many patients and often multiple research centers to make the findings more robust. And those improvements have helped push medical science forward, providing credible evidence for treating and managing disease, while gradually driving bad ideas out of practice. To cite one example, a careful study determined that hormone replacement therapy was deadly for many women taking the combination of estrogen and progestin, claiming many lives during the years doctors prescribed those drugs together. By one estimate, that corrective study triggered a change in medical practice that averted 126,000 breast cancer deaths and 76,000 heart disease fatalities between 2003 and 2012. (p. 222)

The Coming Decline

“Humanity is about to go through a couple of really rough centuries. There is no way around this,” he said, looking out on a future with a burgeoning population stressed for food, water, and other basic resources. Over the previous few centuries, we have managed a steadily improving trajectory, despite astounding population growth. “The scientific revolution has allowed humanity to avoid a Malthusian crisis over and over again,” he said. To get through the next couple of centuries, “we need to have a scientific enterprise that is working as best as it can. And I fundamentally think that it isn’t.” (p. 17)

This echoes the message of At Our Wits’ End, that we're at the precipice of a new dark age as global IQs are in decline mostly for the reason brilliantly and humorously portrayed in the first few minutes of the classic movie, Idiocracy.

As I read more of these books that credibly prophesize the devolution of civilization, I take note of how fortunate I was to be born when I was in the year 1985, historically late enough to enjoy the tremendous opportunity and prosperity created by Western civilization but early enough to see it start to break down and do something about it! I was born at just the right time to enjoy extraordinarily good health, if I was born a little earlier I would have spent decades eating toxic Wonder Bread, if I was born today, as a growing baby I'd be injected with 50-60 toxic vaccines, irradiated 24/7 by 5G EMFs and probably addicted to internet porn by age 9.

 Biohacker ethos and practice

Are validated by this book...

• Biohackers love science but more importantly, we are rigorous practitioners. We do numerous N-of-1 personal trials to see what works for us instead of just placing faith in the conclusions of a study (or what Google says).
• Ever since I began writing about Nootropics, I've done anecdotal analysis of public sentiment about a given supplement. If 10 studies on Pubmed say that a drug or supplement is great but I can find 50 people on the internet reporting non-response or some kind of concerning side effect then, in my view, it's a problematic thing to consume. If there's consistent alignment between what the science says and what dozens (or hundreds) of people online are saying that gives me a lot of confidence about it.
• Biohackers view animal and in vitro studies with heightened skepticism. We know that the gold standard of science is the double, blind placebo-controlled human trial of a statistically significant number (usually over 50) of humans.
• This book has pushed me further into the natural, organic, homeopathic camp. Adaptogenic herbs, for example, have been used for thousands of years by different cultures around the globe. Which I'd contend, gives them more credibility than the double-blind, placebo-controlled studies done with them. There's an argument to be made that we are evolutionarily intertwined with herbal medicine as our ancestors have been using them for thousands of years to stay alive in a brutal and dangerous world. Your ancestor was the village chieftain who could afford Ginsing so he could get an erection to impregnate his fourth wife and have his 16th child. You're the result of thousands of years of stark evolutionary selection bias favoring those who responded positively to natural medicine.
  

Rigor Mortis gives a lot of ammunition to anti-vaxxers, climate change "deniers", flat-earthers, and other "conspiracy theorists" who are skeptical of the "settled science" preached about by the makeup-embellished talking heads on the mainstream media. The firehose of federal funding corrupts the inquisitive minds of scientists and undermines institutions' commitment to the truth. If I tried to design a system to encourage conspiracy, coverup, collusion, and demands for dogmatic consensus I couldn't have done better.

To harken back to the metaphor in the first paragraph of this review, science is like politics, or maybe more accurately like democracy, it's badly broken and works the way it's supposed to only about half the time, but it's still better than the alternative. The good news is that not all science is crap, it's increasingly easy to spot the red flags of bad science, see my deep-dive article and documentary on the topic...

I rate Rigor Mortis 4 stars

It's a crucial warning about the shoddiness of much of medical science and an important call to action to do better because our collective health depends on it. If you're a scientist or considering a career in science or academia you should read it - otherwise, you can pass on it. Minus one star because it's just not that readable of a book, it was a chore to get through, luckily I had a lot of Nootropics and a modicum of righteous indignation to keep me wide awake as I was flipping the pages.