Delusions of Benefit in the International Stroke Trial

Results of the largest and arguably most important trial ever of thrombolytics (clot-busting drugs) for acute stroke were published last week in The Lancet, and the study’s conclusions are breathtaking. Not because of the study results, which are unsurprising, but because the authors’ conclusions suggest that they have gone stark, raving mad.

The International Stroke Trial 3 (‘IST-3’) was a remarkable achievement. The study enrolled 3100 patients, nearly four times that of any previous stroke trial, randomly assigned either to treatment with intravenous thrombolytic drugs, or treatment without the drugs. But unlike earlier studies the ‘pragmatic’ design of IST-3 was unblinded, used no placebos, and included the elderly, the non-elderly, and those with strokes of all severities. In other words, it enrolled common stroke patients having common strokes, a real-world test of the drugs.

Thrombolytics have remained controversial for acute stroke partly because nine of the eleven major trials to date have demonstrated either no benefit or else harm. Supporters argued, however, that early treatment (0 to 3 hours from symptom onset) in the famous NINDS trial made the study unique (other trials went up to 4 or 6 hours), and justified recommendations for use in the early time period. The 3-hour cutoff faded, however, when a 2005 trial used the drug successfully between 3 and 4.5 hours, and again when a respected review group argued that the data suggest similar effects up to 9 hours. It is now apparent to all that NINDS was not unique, and the stroke world has been waiting with bated breath for a large, high quality effort to retest the fundamental question: do thrombolytics decrease death and disability in acute stroke?

In IST-3 the drug was given between 0 and 6 hours, and the data generated two clear findings: First, the drugs failed to reduce death or dependence at six months. In the thrombolytic group 36.57% were alive and independent, while in the control group the number was 35.13%, a difference of about 1%. The difference would have had to be roughly 5% or more to be considered anything other than a wash.  

And second, there was no discernible relationship between timing of administration and drug effect. The drug looked good in the first three hours, but then harmful for the next 90 minutes, and then good again for the next 90. This is a biologically nonsensical (i.e. random) distribution, suggesting that time differences are not a likely mediator of drug effect.

Thus, in one fell swoop every important argument in favor of thrombolytics for acute stroke was dashed. Worse still, the trial was unblinded, and as part of the protocol patients were enrolled only if both they and their doctor considered the drug to be “promising, but unproven.” This is a distinct, and marked, advantage for the drug group. Patients and doctors tend to be more hopeful when a “promising” drug is given. Doctors and staff may treat more aggressively, or more attentively, and patients are inspired to work harder toward recovery. Non-blinded trials are known to significantly enhance the effect of any intervention in comparison to control groups. Given the results, it is thus quite possible that the unblinded design of IST-3 has hidden significant harms of thrombolytics.

Whence, then, the authors’ claim of benefit? The authors describe a “secondary exploration” of their data using ordinal analysis. This uncommon method of measurement examined if thrombolytics may have ‘shifted’ some patients toward better categories of outcome, despite not shifting them toward being alive or independent. Lo and behold, it appeared to be so (though only with the help of an unexplained statistical “adjustment”). Of course, in any group of exploratory analyses some will appear favorable by random chance—which is why there’s only one primary outcome in any trial—because if you keep flipping the coin and moving the goal posts, eventually you’ll hit . . . something. Most damningly, in a moment of clarity, the authors themselves have described ordinal analysis (in a separate paper about IST-3) as “not appropriate for the primary analysis of outcome.” And yet they write their conclusions as if the illusory “shift” were the primary outcome.

With advances in scientific literacy it has been years since I have seen a top journal allow authors to proclaim a conclusion in direct conflict with their own primary study results. And yet the authors blithely conclude that thrombolytics “improved functional outcome.” Worse, an accompanying editorial trumpets that “the role of stroke and emergency physicians is now not to identify patients who will be given rt-PA, but to identify the few who will not.” Welcome to Wonderland.

These statements feel not just forced, but frankly delusional. Has neuro gone psycho? The results of IST-3 indicate, at best, a profound disappointment (even the hallucinated benefit would be tinier than any previously claimed) and at worst the beginning of the end for thrombolytics in stroke. In either case, reality may be tough to handle, but it is not a matter of debate, or interpretation, or perception. The primary outcome failed. We have a phrase for that: no benefit.  

 

User login