I dropped my wife’s new iPad. Dropping things is one of the many curses of oafdom, a defining condition for me. And this promised to be costly: The impact cracked the corner of the glass surface leaving the screen portion unaffected, but marring the machine’s techno-loveliness, and sending my better half into a funk.
Through a period of deep contrition and not-quite absolution I searched frantically for solutions, cheap solutions. And soon I found nirvana. In a recent Apple online ‘forum’ a gentleman with a penchant for exclamation points wrote that the Genius Bar in the Apple store had replaced(!) his broken glass surface (!!) for free (!!!!). Too good to be true? Probably. But the exclamation points had me excited.
I marched in to the store, showed the goateed Genius our baby’s blemish, and flashed a conspiratorial smile. He looked at the crack, sucked the air in through his teeth—the way one does when a tourist in New York City asks directions to the Eiffel Tower—and looked at me.
“That’s gonna be about $400.”
“Huh.” I nodded blankly and stared at him, then the iPad, for a long time. I considered pointing out the fairy tale from the Apple forum, but he shook his head slowly. Must have heard that one before. Defeated, I picked up the hideous, deformed machine and turned to leave.
“Looks like I’ll have to live with it,” I offered. “Thanks anyway.”
“Um, you need to replace this glass.” To my surprise, he wasn’t ready to let go.
I turned back. “Really?”
“Of course, dude. This crack is gonna grow. It’ll get bigger and spread across the screen.”
I looked at the web, which had no radiating fissures, no dead-end lines, and nothing that looked like it could spread away from the affected corner.
“How can you tell?” I asked.
“Dude.” He leaned in. “We replace all of these, because if we don’t the crack just grows.”
“Well, okay. But… how do you know it will grow?”
“That’s what I’m saying, man. We just have to replace them all. So that’s what we do.”
It didn’t seem like the right place to get pedagogical about incorporation bias, or begging the question, but hacky sack’s logic was heroically unscientific. This wasn’t even ‘bad science’, the kind of inductive reasoning that leads to faulty generalizations from isolated observations. The Genius hadn’t even observed the phenomenon he was asserting. He simply ‘knew’ it.
Oddly, in the world of cardiovascular research, a nearly identical logic error pervades virtually everything that cardiologists do in daily practice. In a research field flooded with money and dominated by earnest men and women who believe in science, a surprisingly conspicuous flaw has driven three decades of data misinterpretation. The core of the problem is a research outcome measure known as ‘revascularization’.
In cardiology research, revascularization is any mechanical attempt to provide blood flow to an area of the heart, either by percutaneous measures (stents to open narrowed arteries) or surgical means (operations to ‘bypass’ the arteries with new ones). For years cardiovascular researchers have presumed that a patient only undergoes revascularization when it is necessary, a last ditch effort to prevent death or heart attack. So if a patient in such studies doesn’t have a heart attack and doesn’t die, but does undergo revascularization, that patient is considered, for the purposes of interpreting the study, to have suffered a fate equal to death or heart attack. As a result, many drugs have not been shown to significantly reduce heart attacks or deaths, but they have been shown to reduce revascularization, which occurs far more often than heart attacks or death ever did. And many cardiologic tests have not been shown to be robust predictors of heart attacks or deaths, but they do have a knack for predicting who will have revascularization.
This logic might be reasonable if everyone who underwent revascularization was indeed saved from an impending heart attack or death. But stent revascularizations are most often not performed on patients in the throes of a major heart attack—the only ones known to benefit. Studies and meta-analyses have proven repeatedly that except for patients actively having a heart attack, placing stents neither saves lives nor prevents future attacks. This is a discomfiting fact that even the American Heart Association has recently conceded.(1) Furthermore, the most optimistic trial data suggest that, at best, 3 to 5% of bypass surgery patients live longer because of the operation.(2) Thus at least 19 of 20 people who have bypass surgery will not experience a life-saving benefit.
What’s more, decisions about who undergoes revascularization are disturbingly whimsical. A substantial body of research demonstrates that who gets revascularization in the U.S. is often based on where they live, how ‘good’ their insurance is, their race, and their socioeconomic status, rather than their medical condition.(3)(4)(5)
To summarize: patients undergoing revascularization will endure an invasive procedure, anesthesia, and the body stressors, hospitalization, side effects, complications, and rehabilitation that follow. But precious few of them will be saved from death or a future heart attack, which makes using ‘revascularization’ as a substitute for death or heart attack absurd, and profoundly misleading.
What’s more, the problem is compounded in certain types of studies. Most glaringly, when revascularization is used to examine how accurately tests diagnose or predict heart attacks or deaths, the result is total obfuscation. In such studies there are typically more subjects who undergo revascularization than who have a heart attack or death. So the test ends up not as a predictor of heart attacks and deaths, but as an indicator of who is likely to have an unnecessary, and likely unhelpful, procedure.
It gets worse: A large percentage of people who have a stress test that is positive (i.e. suggests narrowed coronary arteries) will have a revascularization procedure because the stress test was positive. Thus the stress test didn’t ‘predict’ the revascularization, it caused the revascularization. But in the study results the stress tests will appear to have been ‘an excellent predictor of the need for revascularization’. This is a textbook example of ‘incorporation bias’, a crippling flaw in study design that allows the test result to influence (i.e. to be incorporated into) the judgment of whether the test is right. This would be like fixing my wife’s iPad because goatee said we should, and then calling the repair job ‘proof’ that he was right.
The great tragedy of flawed study design in research is its legacy. Instead of stress tests being a predictor of what patients care about (having heart attacks or dying), they are a predictor of an invasive, expensive, and largely fruitless techno-bangle. Instead of modern cardiac drugs being proven effective against heart attacks and death, many are effective only at preventing bad tests and unnecessary procedures. In a twisted way this seems useful too, but it’s a dishonest and ineffective approach to preventing real heart problems.
As we untangle the legacy of past research errors, it is critical to understand and learn from them in order to prevent them in the future. Allowing revascularization to be reported as an outcome equivalent to heart attack or death in any study is a huge mistake that should no longer be tolerated, because modern cardiac patients are paying a heavy price in the form of unnecessary procedures, unnecessary side effects, and sometimes fatal complications. Not to mention the colossal cost and resource burden that unnecessary procedures place on both health systems and individuals.
And so it was that on my walk home from the Apple store I learned to love a blemish, and to accept that it doesn’t ‘need’ to be repaired because someone says so. I also enjoyed a moment of clarity about preventing future mistakes: don’t drop the iPad.
1. Writing Committee Members, Levine GN, Bates ER, Blankenship JC, Bailey SR, Bittl JA, Cercek B, Chambers CE, Ellis SG, Guyton RA, Hollenberg SM, Khot UN, Lange RA, Mauri L, Mehran R, Moussa ID, Mukherjee D, Nallamothu BK, Ting HH. 2011 ACCF/AHA/SCAI Guideline for Percutaneous Coronary Intervention: Executive Summary. Circulation 2011 Dec;124(23):2574 -2609.[cited 2012 Feb 4 ]
2. Yusuf S, Zucker D, Passamani E, Peduzzi P, Takaro T, Fisher L., Kennedy J., Davis K, Killip T, Norris R, Morris C, Mathur V, Varnauskas E, Chalmers T. Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists Collaboration. The Lancet 1994 Aug;344(8922):563-570.[cited 2012 Feb 4 ]
3. Hannan EL, Wu C, Chassin MR. Differences in per capita rates of revascularization and in choice of revascularization procedure for eleven states. BMC Health Serv Res 2006 Mar;6:35.
4. Writing Group Members, Lloyd-Jones D, Adams R, Carnethon M, De Simone G, Ferguson TB, Flegal K, Ford E, Furie K, Go A, Greenlund K, Haase N, Hailpern S, Ho M, Howard V, Kissela B, Kittner S, Lackland D, Lisabeth L, Marelli A, McDermott M, Meigs J, Mozaffarian D, Nichol G, O’Donnell C, Roger V, Rosamond W, Sacco R, Sorlie P, Stafford R, Steinberger J, Thom T, Wasserthiel-Smoller S, Wong N, Wylie-Rosett J, Hong Y, for the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart Disease and Stroke Statistics—2009 Update. Circulation 2009 Jan;119(3):e21-e181.[cited 2012 Feb 4 ]
5. Wennberg D, Dickens J Jr, Soule D, Kellett M Jr, Malenka D, Robb J, Ryan T Jr, Bradley W, Vaitkus P, Hearne M, O’Connor G, Hillman R. The relationship between the supply of cardiac catheterization laboratories, cardiologists and the use of invasive cardiac procedures in northern New England. J Health Serv Res Policy 1997 Apr;2(2):75-80.[cited 2012 Feb 1 ]