Researchers Show Parachutes Don't Work, But There's A Catch

Dec 22, 2018
Originally published on December 26, 2018 1:25 pm

Research published in a major medical journal concludes that a parachute is no more effective than an empty backpack at protecting you from harm if you have to jump from an aircraft.

But before you leap to any rash conclusions, you had better hear the whole story.

The gold standard for medical research is a study that randomly assigns volunteers to try an intervention or to go without one and be part of a control group.

For some reason, nobody has ever done a randomized controlled trial of parachutes. In fact, medical researchers often use the parachute example when they argue they don't need to do a study because they're so sure they already know something works.

Cardiologist Robert Yeh, an associate professor at Harvard Medical School and attending physician at Beth Israel Deaconess Medical Center, got a wicked idea one day. He and his colleagues would actually attempt the parachute study to make a few choice points about the potential pitfalls of research shortcuts.

They started by talking to their seatmates on airliners.

"We'd strike up a conversation and say, 'Would you be willing to be randomized in a study where you had a 50 percent chance of jumping out of this airplane with — versus without — a parachute?' " Yeh says.

Only a few people said yes to this outrageous invitation, and they were excluded for reasons of questionable mental health.

The scientists had much better success asking members of their own research teams from Harvard, University of California, Los Angeles (Where Yeh's brother is a surgery professor), and University of Michigan (where a buddy works) about volunteering to participate in the experiment on other aircraft.

In all, 23 people agreed to be randomly given either a backpack or a parachute and then to jump from a biplane on Martha's Vineyard in Massachusetts or from a helicopter in Michigan.

Relying on two locations and only two kinds of aircraft gave the researchers quite a skewed sample. But this sort of problem crops up frequently in studies, which was part of the point Yeh and his team were trying to make.

Still, photos taken during the experiment show the volunteers were only too happy to take part. "I think people are laughing all of the way to the ground," Yeh says.

Oh, there's one important detail here. The drop in the study was about 2 feet total, because the biplane and helicopter were parked.

Nobody suffered any injuries. Surprise, surprise. So it's technically true that parachutes offered no better protection for these jumpers than the backpacks.

"But, of course, that is a ludicrous result," Yeh says. "The real answer is that that trial did not show a benefit because of the types of patients who were enrolled."

If they had enrolled people at high risk for injury, that is people in flying aircraft, the results would have been quite different (not to mention unethical).

But something like this happens in everyday medical research. It's far too easy for scientists who have already anticipated the outcome of their research to cherry-pick patients and circumstances to achieve the results they expect to see. This research paper carried that idea to the ridiculous extreme.

The study's findings were published in the traditionally lighthearted Christmas issue of the medical journal, BMJ.

"It's a little bit of a parable, to say we have to look at the fine print, we have to understand the context in which research is designed and conducted to really properly interpret the results," Yeh says. Scientists often read just the conclusion of a study and then draw their own conclusions that are far more sweeping than are justified by the actual findings.

This is a real problem in science.

"I know that people often don't look detailed enough into what is being investigated to know how to interpret the results of a trial," says Cecile Janssens, an epidemiology professor at Emory University.

Janssens was delighted to come across the paper on Twitter. She says like a lot of research, its results are accurate as far as they go, but "the results can only be generalized to situations where people jump out of an aircraft within a few feet above the ground."

She plans to give this paper to her students with a straight face and see how long it takes for them to get the deeper points about scientific methodology buried in this absurd experiment.

"It will be unforgettable," she says — far better than assigning a straight-ahead scientific study.

Yeh is pleased to see that the fun he had with his colleagues is turning into a teaching tool. He also savors some of the more subtle lessons buried in the paper.

For example, the scientists attempted to submit it to a government registry of research studies, which is required for many studies involving human subjects. They chose one in Sri Lanka to reduce the risk that it would be discovered in advance, spoiling the joke. It was rejected.

"They thought that a trial conducted in this manner could not lead to scientifically valid evidence," he said.

"They're right!" he adds with a laugh.

In fact, the paper acknowledges that the research team members cracked themselves up so much that "all authors suffered substantial abdominal discomfort from laughter."

"Our greatest accomplishment from all of this was we felt very good that we were able to cite Sir Isaac Newton in the paper," he says. They referred to Newton's classic 1687 paper establishing the law of gravity.

Yes, gravity is a law. Mess with it at your own risk.

You can reach NPR science correspondent Richard Harris by email: rharris@npr.org.

Copyright 2019 NPR. To see more, visit https://www.npr.org.

SCOTT SIMON, HOST:

This just in - research published in a major medical journal concludes that a parachute is no more effective than an empty backpack. But before you leap to any conclusions, hear the whole story from NPR science correspondent Richard Harris.

RICHARD HARRIS, BYLINE: The gold standard for medical research is a study that randomly assigns volunteers to try an intervention or to be in a control group. For some reason, nobody has ever done a randomized controlled trial of parachutes. In fact, medical researchers often use the parachute example when they argue they don't need to do a study because they're so sure they already know the answer. Dr. Robert Yeh at Harvard Medical School got a wicked idea one day. He and his colleagues would actually attempt the parachute study to make a few choice points about the potential pitfalls of research. They started by talking to their seatmates in airliners.

ROBERT YEH: We'd strike up a conversation and say, you know, would you be willing to be randomized in a study where you had a 50 percent chance of jumping out of this airplane with versus without a parachute?

HARRIS: Did you get any takers?

YEH: (Laughter) For the most part, we did not on those commercial airlines.

HARRIS: And those who said yes were excluded for reasons of questionable mental health. They had much better success asking members of their own research teams, from Harvard, UCLA and the University of Michigan, about volunteering to participate in the experiment on other aircraft.

YEH: Either a small biplane - that was our group that - we went to Martha's Vineyard and did that, and the group in Michigan went and jumped out of a small helicopter in fact.

HARRIS: The 23 people who agreed to jump from the smaller aircraft represented quite a skewed sample, which was part of the point Yeh and his team were trying to make. Photos taken during the experiment show the volunteers were only too happy to take part.

YEH: I think people are for the most part laughing all the way down to the ground.

HARRIS: Right, which is about two feet or something.

YEH: Exactly right (laughter).

HARRIS: Oh, a minor detail here. The biplane and helicopter were parked, so it's not surprising that nobody suffered any injuries. It's literally true that parachutes offered no better protection for these jumpers than the backpacks.

YEH: And, of course, the top line of results would be that parachutes don't work. But, of course, that is a ludicrous result. The real answer is that that trial did not show a benefit because of the types of patients that were enrolled.

HARRIS: If they had enrolled people at high risk for injury - that is, people up in the sky - the results would have been quite different, not to mention unethical. Their study is published in the traditionally lighthearted Christmas issue of the medical journal BMJ.

YEH: It's a little bit of a parable to say that we have to look at the fine print. We have to understand the context in which research is designed and conducted to really properly interpret the results.

HARRIS: Emory University epidemiology professor Cecile Janssens says via Skype this is a real problem in science.

CECILE JANSSENS: I know that people often don't look detailed enough into what is being investigated to know how they can interpret the results of a trial.

HARRIS: Janssens was delighted to come across the paper on Twitter. She says, like a lot of research, the results are accurate as far as they go.

JANSSENS: Results of this trial can only be generalized to situations where people jump out of an aircraft that's within a few feet above the ground.

HARRIS: She plans to give this paper to her students with a straight face and see how long it takes for them to get the deeper points about scientific methodology buried in this absurd experiment.

JANSSENS: It'll be unforgettable.

HARRIS: Yeh is pleased to see that the fun he has had with his colleagues is turning into a teaching tool.

YEH: And our greatest, I think, accomplishment from all of this is that we felt very good that we were able to cite Sir Isaac Newton in the paper.

HARRIS: Yes, I noticed that. It was in fact his original publication from - what was it? - 1687.

YEH: (Laughter) That's right.

HARRIS: Yes, gravity is a law. Mess with it at your own risk. Richard Harris, NPR News. Transcript provided by NPR, Copyright NPR.