(Reprinted with permission from American Journal of Psychiatry 2011; 168:770–775)
Everyone said, loud enough for the others to hear: “Look at the Emperor’s new clothes.” —The Emperor’s New Clothes, by Hans Christian Anderson
It has been over 10 years since the initiation of the National Institute of Mental Health (NIMH) Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study and 5 years since the first publication of its primary results (1). In this period, the initial report has been cited in the literature over 1,600 times (2) while more than 80 articles from the study’s extensive database (Table 1), as well as a book serving as an archive of the study’s results and implications (3), have been published. In the meantime, several more randomized trials comparing the effectiveness of antipsychotics have been completed (4–6), meta-analyses that bear on the findings of the CATIE study have been performed (7, 8), and commentaries on CATIE’s findings and critiques of its methodology have been published (9–11). All of these help us to view the CATIE study in a broader context and enable us to determine what we really learned from it.
When the CATIE study was designed in 1999–2000, the prevailing opinion of researchers and clinicians alike was that the newer (second-generation) antipsychotic drugs were vastly superior to the older (first-generation) antipsychotic drugs in efficacy and safety. This largely reflected the results of studies sponsored by the manufacturers of the new drugs (12, 13), marketing messages of pharmaceutical companies and the hopes of many who wanted better treatments. Indeed, the hypothesis and expectation of the CATIE study investigators was that the first-generation antipsychotic perphenazine would be inferior to the newer agents. Consequently, the finding that perphenazine was similar in effectiveness to most other medications had a profound effect that extended beyond the scientific and psychiatric communities to the lay public and various stakeholder groups. Somewhat sensational news reports decried the preferential use and greater cost of the newer medications and the marketing practices that led to them. For example, the September 21, 2005, editorial page of The New York Times opined, “A government-financed study has provided the strongest evidence yet that the system for approving and promoting drugs is badly out of whack. The study compared five drugs used to treat schizophrenia and found that most of the newest, most heavily prescribed drugs were no better than an older drug that is far cheaper. The nation is wasting billions of dollars on heavily marketed drugs that have never proved themselves in head-to-head competition against cheaper competitors” (14).
But what did we really learn from the CATIE study? In this commentary, we summarize its major implications and their relevance to clinical practice. We will also address some of the study’s most relevant critiques.
The most striking result of the CATIE study, which enrolled almost 1,500 individuals with chronic schizophrenia, was the high rate of treatment discontinuation (up to 74%) over the 18-month period of the trial and the short median time to discontinuation of treatment (about 6 months) in all phases of the trial. Treatment discontinuation in CATIE reflected the desire of patients and their clinicians to switch their medications or patients’ nonadherence to treatment. These findings were surprising to some but are consistent with those observed in administrative databases that document antipsychotic prescribing patterns in state Medicaid programs (15, 16), in Québec (17), and in the U.S. Veterans Health Administration (18). In fact, the mean duration of treatment for persons with schizophrenia spectrum disorders who started new antipsychotics in these analyses of administrative data was shorter than that for participants in CATIE.
The most controversial finding of the CATIE study was the lack of significant differences in effectiveness between most of the second-generation antipsychotics and perphenazine, the proxy for the first-generation antipsychotics. It has been argued that olanzapine was the most effective antipsychotic medication in the first phase of the study in spite of the lack of a statistically significant advantage over perphenazine or ziprasidone (19). However, olanzapine had the most adverse metabolic effects and highest discontinuation rate as a result of intolerability. Moreover, the other second-generation antipsychotics were similar to perphenazine in effectiveness. In addition, there were no advantages in efficacy for any of the second-generation antipsychotics with regard to negative symptoms or cognitive impairment. The most robust differences observed between drugs were in the rates of side effects, particularly weight gain and laboratory measures of cholesterol, triglycerides, and prolactin. Extrapyramidal symptoms were similar across treatment groups, although more patients receiving perphenazine discontinued treatment because of this side effect.
The CATIE study showed that each drug might be most useful in particular situations. For patients whose symptoms did not improve with first-line treatment, clozapine was most effective. Olanzapine was effective in all phases of the study, but it and clozapine had the greatest side effect liabilities. For patients who switched medications because of side effects, the best alternative depended on the type of the individual side effects and the severity of the patient’s illness. Risperidone was effective overall for people who discontinued prior medications as a result of intolerability (and is now available as a generic). Quetiapine worked well for people who did not tolerate perphenazine. Ziprasidone demonstrated the most favorable metabolic profile. Perphenazine, because it was priced as a generic, was the most cost-effective drug in the study’s main phase.
The essential import of the CATIE study can be summarized as follows. Antipsychotic drugs, both old and new, are clearly effective and have been a boon to the treatment of schizophrenia. However, they have substantial limitations in efficacy and safety, which lead clinicians and consumers to seek better results by switching or adding medications. The numerous antipsychotic drugs, however they might be classified, are more similar to than different from each other. To the extent that antipsychotics differ, it is more in their side effects than therapeutic effects. Nevertheless, there is variation in the effectiveness of antipsychotic drugs, which for individual patients can be substantial, and what works for one person may not work for another. Consequently, treatments for schizophrenia must be individualized.
The CATIE study has not suffered from lack of attention. Of all the issues raised in the commentary and critiques of the CATIE study, we believe that three are most salient. CATIE used an innovative outcome measure to capture the overall effectiveness of the medications and to reflect the input of patients and clinicians on their efficacy and tolerability: time to “all-cause treatment discontinuation.” It is important to emphasize that discontinuation did not mean that patients stopped treatment and left the study but that they and/or their clinicians elected to switch or stop the medication to which they had been randomly assigned. This measure was criticized as being not sufficiently specific or clinically valid (11). However, treatment discontinuation is a discrete event that may have many clinically important causes that are not mutually exclusive or specifically identified. For example, in everyday practice when patients “drop out” of treatment or are “noncompliant,” this is often because of problems with psychotic symptoms and/or adverse effects. The measure’s simplicity and comprehensiveness make it an attractive primary outcome for effectiveness studies. Patients in CATIE who discontinued treatment for any cause had lower quality of life scores than those who completed the study, and their quality of life scores at the time of discontinuation were decreased from baseline (20).
A second criticism was that the dose ranges of the study drugs were not equivalent. However, the doses selected were based on those used in clinical practice. Moreover, no studies at the time of the trial, or subsequently, have demonstrated clear differences in dose response from those used in the trial. In addition, the dose of the first-generation drug, perphenazine, was administered at the low end of the recommended dose range. This was done to minimize the potential extrapyramidal side effects, but the drug still proved to be therapeutically comparable to the second-generation medications.
A more cogent criticism is that the study was not powered for noninferiority. This is accurate but does not negate the results. The fact that the study was powered for superiority reflects the investigators’ a priori belief that the second-generation drugs would prove superior. The fact that the new drugs did not show statistical superiority (or even numerical superiority in all cases except olanzapine) over perphenazine indicates that if there were an effectiveness difference, which the study did not reveal because of power limitations, the magnitude of the effect must be small. In addition, the confirmatory pattern of results from subsequent studies and meta-analyses further supports the validity of the CATIE results.
Given its startling results and the extraordinary attention that it attracted, one might have expected the CATIE study to have had a profound effect on clinical practice. However, prescribing patterns have not markedly changed in the ways suggested by the CATIE study’s results. For example, since 2006, among New York State Medicaid recipients with schizophrenia or schizoaffective disorder, clozapine use is flat, olanzapine use has declined, quetiapine use is up, and risperidone use has declined—even though it became generic during this time—while its branded metabolite, paliperidone, has gained considerable use. Meanwhile, use of perphenazine and all other mid- and low-potency first-generation antipsychotic drugs remains rare (Figure 1).
Figure 1.Antipsychotic Prescriptions Filled for New York Medicaid Recipients With Schizophrenia Spectrum Disorders (2006–2009)a
a Antipsychotic prescriptions filled indicates the number of patients who filled at least one prescription for a specified drug in the calendar year.
On the other hand, the CATIE results have clearly affected the debate about the relative effectiveness of antipsychotic drugs and our understanding of the true value and real role of the different types of antipsychotics. Moreover, the CATIE study has dramatically demonstrated the value and importance of independently sponsored and conducted comparative effectiveness trials to inform clinicians, consumers, and policy makers of the relative value of marketed treatments for medical disorders. In particular, policy makers need information to make rational decisions about whether to adopt expensive new treatments that have not been compared with cheaper existing ones. The importance of comparative effectiveness research is evident in recent legislation. In 2009, the American Recovery and Reinvestment Act provided for the development of an infrastructure for the ongoing generation and dissemination of information on comparative effectiveness. In 2010, the Patient Protection and Affordable Care Act established the Patient-Centered Outcomes Research Institute to identify national priorities for research and to establish, update, and carry out a national comparative outcomes research project agenda.
CATIE helped to demonstrate that, although the introduction of second-generation antipsychotic drugs brought new options for the treatment of psychosis, the major advance many had hoped for remains elusive. By revealing the truth about the emperor’s new clothes, CATIE has helped to refocus efforts on the need for truly innovative treatments and strategies that can make significant advances for persons with schizophrenia and related psychoses.