Adjunctive Atypical Antipsychotic Treatment for Major Depressive Disorder: A Meta-Analysis of Depression, Quality of Life, and Safety Outcomes

Introduction

Atypical antipsychotic medications are widely used in the treatment of major depressive disorder. In the United States in 2007 and 2008, there were an estimated 3.9 million treatment visits per year in which an antipsychotic medication was prescribed for depression, and nearly all of these (96%) involved prescription of an atypical antipsychotic medication [1]. Although aggregate statistics mask the specific indications for use (i.e., monotherapy versus adjunctive therapy), this represents a substantial increase in antipsychotic treatment of depression over time, as there were just over 2 million such visits annually during 1995 and 1996, of which 405,000 involved prescriptions for atypical antipsychotic medications. These data are also consistent with market reports from industry [2]. Three atypical antipsychotic medications have approval from the US Food and Drug Administration (FDA) as adjunctive therapies in depression for adults, while none are approved for monotherapy. These approvals (and subsequent marketing efforts), along with the volume of prescriptions, suggest that a large number of prescriptions for atypical antipsychotic medications written for the treatment of depression are being used for adjunctive therapy [3–5].

The efficacy of adjunctive atypical antipsychotic therapy in reducing depression symptom severity in major depressive disorder is summarized in two previous systematic reviews, but neither comprehensively summarized data on both efficacy and safety [6,7]. Both reviews analyzed efficacy only in terms of dichotomous response and remission outcomes derived from clinician-rated depression measures and did not assess changes in terms of symptom severity on the underlying continuous rating scales. Safety was only assessed by examining dropout rates due to adverse events; the authors of these meta-analyses [6,7] and of a relevant narrative review noted that a comprehensive summary of safety data is lacking [8]. A Cochrane review provided a more thorough assessment of both efficacy and safety outcomes but did not include data on important patient-centered efficacy outcomes such as patient-rated depression, functional impairment, or quality of life [9]. The Cochrane review assessed the frequency of several relevant adverse events, but some critical adverse events of interest, such as elevated cholesterol or triglyceride levels, were not included. Further, and most importantly, effect size estimates presented in these reviews may have been inflated because the authors did not summarize unpublished data, such as those from FDA New Drug Applications (NDAs) or manufacturers’ clinical trial registries [10–13]. Given the importance of functional status, quality of life, and drug-related side effects to the overall assessment of well-being and recovery from depressive mood episodes [14–16], we conducted this meta-analysis to provide a comprehensive estimate of the efficacy and safety profiles of atypical antipsychotic medications for the adjunctive treatment of major depressive disorder.

Methods

Ethical Review.

Because this was a study-level systematic review and meta-analysis of trials, and did not involve collection and analysis of any individual-level data, ethical approval was not sought for this study.

Search Strategy.

This systematic review was reported using PRISMA guidelines; the PRISMA checklist is provided as Text S1. To identify both published and unpublished studies for review, we searched Medline, PsycINFO, ClinicalTrials.gov, and the Cochrane Central Register of Controlled Trials using the terms depression AND (aripiprazole OR asenapine OR clozapine OR iloperidone OR lurasidone OR olanzapine OR paliperidone OR quetiapine OR risperidone OR zjprasidone). Medline search results were restricted to the following article types: clinical trial, controlled clinical trial, or randomized controlled trial. Our literature search was conducted in December 2011 and updated on December 14, 2012. In addition, we searched the American Psychiatric Association Annual Meeting New Research Abstracts for 2001–2010 using each of the generic drug names as a search term, then winnowed the results down to abstracts that appeared to possibly meet the inclusion criteria. We also examined all references in a previously published meta-analysis [6] as well as those contained in each published study obtained through our literature search.

To obtain additional unpublished data, we searched the drug manufacturers’ online clinical trial registries as well as FDA NDAs for the atypical antipsychotic medications that have received an indication for the adjunctive treatment of major depressive disorder (aripiprazole, olanzapine-fluoxetine combination [OFC], and quetiapine). For published studies, we supplemented published data with data available in NDAs or clinical trial registry reports whenever such data were available.

Study Selection.

Trials were included if they were acute-phase (i.e., not for relapse prevention or maintenance treatment [17,18]), placebo-controlled trials in which participants treated with antidepressant medications were randomly assigned to additionally receive an atypical antipsychotic medication or placebo. In order to meet our definition of treatment-resistant depression, participants must have been diagnosed with current major depressive disorder and must have been determined to have had an inadequate response to at least one course of antidepressant medication treatment prior to enrollment in the study. Furthermore, data for at least one outcome measure must have been reported in a manner that allowed calculation of an effect size. No language exclusions were applied.

Data Extraction.

Four study authors (G. I. S., A. P., M. I. B., and E. L.) coded study descriptor data. To establish consistency, all coders first coded the articles reporting outcomes from the aripiprazole studies. Then two study authors (G. I. S. and A. P.) jointly coded the OFC and risperidone articles, and two study authors (M. I. B. and E. L.) jointly coded the quetiapine articles. Disagreements were resolved by consensus. Coders were not blind to the results of the coded studies.

Several descriptor variables were coded for each study. (1) Flexible dosing versus fixed dosing regimen. (2) Dosage range. (3) Mean dosing achieved at end point. (4) Number of participants in each group of trial. (5) Duration of acute-phase treatment (weeks). (6) Number of prior failed trials of antidepressant medications, where the number of failed trials prior to study enrollment (historical) and the number of failed trials during the study (prospective) prior to initiation of the study drug for adjunctive treatment were recorded separately. (7) Procedures employed to evaluate for major depressive disorder (structured interview or otherwise). (8) Use of a structured instrument versus open-ended questioning to elicit adverse events [19,20] (with the latter assumed if no details were reported). (9) Adverse events scale(s) used to systematically assess for any particular adverse event(s), if any. (10) The criterion used to establish a minimum level of occurrence for adverse events reporting in the trial (e.g., if only those adverse events occurring in at least 5% of participants were reported in the associated journal article, the adverse events reporting threshold was coded as 5%). (11) Extent to which the random-sequence generation procedures were adequate versus inadequate or unclear [21]. Adequate sequence generation procedures included use of a computer program, random number table, coin tossing, randomly drawing envelopes, throwing dice, or similar methods. Merely describing the trial as randomized was considered an unclear method of sequence generation. (12) Whether or not the study eliminated placebo responders prior to randomization. (13) Whether or not persons who had a prior nonresponse to the study drug were excluded. (14) Whether or not the placebo was described as identical to the study drug in terms of at least two of the following three criteria: taste, appearance, and smell [22]. (15) Use of blinded raters, coded as affirmative if the following two conditions were met: (a) it was explicitly stated that blinded raters were used, and (b) it was explicitly stated that different personnel were used to rate efficacy measures and adverse events [23–25]. (16) Funding sponsor.

Efficacy and safety outcome data were independently extracted by two authors (G. I. S. and A. P.) and then checked for agreement. Disagreements were resolved by checking the original data source.

Outcome Measures.

Remission was defined variably across studies. We recorded the most stringent definition of remission utilized in each trial while also recognizing that the Montgomery–Asberg Depression Rating Scale (MADRS) [23] was the most commonly used outcome measure in the included trials. One end-point remission measure was selected from each trial according to the following order of priority: MADRS ≤8, then Hamilton Depression Rating Scale (HAM-D) ≤7 [24], then MADRS ≤10. Some trials of OFC defined remission as MADRS ≤8 at two consecutive visits during the study even if these two consecutive visits did not necessarily occur at study end point [25–27]. The clinical trial registry reports of these trials also provided the number of participants who met remission criteria at an interim time point but then relapsed. For these studies, we calculated the number of participants in remission as the number of participants who achieved interim remission minus the number of patients who subsequently relapsed.

Response was defined across studies as a 50% improvement from baseline to end point on either the MADRS or HAM-D [28]. When studies provided response rates for both measures, we used the MADRS as the response measure, as it was the most commonly reported measure of response.

We recorded data from any continuous measure of depression, quality of life, or functioning but opted not to analyze single rating scale items from larger scales (e.g., individual MADRS items) separately because they were infrequently reported. When data were reported on both the MADRS and HAM-D, we included data from the MADRS, as it was the most commonly used measure of depressive symptoms. The only continuous self-report measure of depression used in these trials was the Inventory of Depressive Symptomatology Self Report [29]. Continuous measures of quality of life included the Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q) [30] and the Short Form 36 Health Survey (SF-36) [31]. The only continuous measure of functional impairment employed in these trials was the Sheehan Disability Scale (SDS) [32]. As measures of quality of life and functional impairment varied across studies, we pooled such measures together to create an omnibus effect size for each drug, and across all drugs.

We aggregated conceptually similar adverse events into the following categories. (1) Sedation-related: asthenia, fatigue, lethargy, sedation, somnolence, or feeling tired. (2) Akathisia-related (either self-reported or observer-rated): akathisia or restlessness. (3) Extrapyramidal symptoms (EPS), other than akathisia-related (either self-reported or observer-rated): dyskinesia, dystonia, extrapyramidal disorder, EPS, muscle spasms, muscle twitching, parkinsonism, or tremor. (4) Abnormal metabolic laboratory results: elevated fasting or nonfasting total cholesterol, low-density lipoprotein (LDL) cholesterol, or triglycerides; low high-density lipoprotein (HDL) cholesterol; or elevated fasting or nonfasting glucose, glycated hemoglobin; or hyperglycemia. (5) Elevated prolactin. (6) Edema or peripheral edema. (7) Significant weight gain, defined across various trials as weight gain of ≥7%, ≥10%, or >10% from baseline to end point.

We also coded events that were reported in the categories of pain, psychiatric events, nausea, and infection. However, because no sign of elevated risk was gleaned from these data, these analyses are not reported (data available from authors on request).

Statistical Analysis.

The quality of data reporting varied across studies. For continuous outcomes, effect sizes were computed from means and standard deviations when possible. When these were not provided, effect sizes were computed based on means and p-values, or p-values only. In some studies, three or more treatment groups were compared, thereby creating a structural dependency that could affect our estimates. For example, two fixed doses (A and B) of an adjunctive atypical antipsychotic medication might be compared to one group that received adjunctive placebo (C), in which case the estimated efficacy of A and B would be defined relative to the same comparison group. To maintain independence, we pooled these comparisons and utilized their average (e.g., the average of A versus C and B versus C).

Each effect size was weighted by its inverse variance in order to provide a pooled effect size estimate that most accurately approached the true population effect size [33]. We calculated odds ratios (ORs) for categorical measures and used Cohen’s d for continuous measures. We converted continuous effect sizes to Hedges’g, which corrects for a small bias in Cohen’s d [33]. We reported both efficacy and safety data for each drug individually and across drugs. An OR presents a relative measure of treatment effect; to also provide a measure of absolute benefit/harm, we calculated the number needed to treat (NNT) for treatment benefits and the number needed to harm (NNH) for adverse events [34]. The NNT represents the number of participants who would need to be treated with an adjunctive antipsychotic to gain one additional beneficial response over what would have been obtained had all patients received adjunctive placebo. NNH represents the number of patients who would require treatment to generate one additional adverse event relative to placebo. NNT/NNH values were calculated based on the pooled OR rather than from the risk difference in each study, as the risk difference is associated with more between-study heterogeneity than the OR [35]. Conversions from OR to NNT were performed in Visual Rx software [36]. The baseline risk (required for calculating NNT) was estimated by using the pooled rate of events occurring among placebo-treated patients weighted by each study’s total sample size. The baseline risk was calculated separately for each drug, so that placebo participants in one drug’s trials were not used to calculate baseline risk for a different drug. As in any meta-analysis, our estimates of NNT and NNH generalize only to situations in which patients receive a similar dosage for a similar treatment duration; further, estimated NNH and NNT apply only when generalizing to patients similar to those in the included trials. Because of various study inclusion and exclusion criteria, patients in the placebo groups in our meta-analysis may not be representative of patients seen in some clinical practice settings.

We performed homogeneity analyses using the Q statistic. Because the Q test of homogeneity often lacks power to detect heterogeneity when the number of trials in a meta-analysis is small, we also calculated the I² statistic [37]. To pool estimates across studies while incorporating potential heterogeneity, we employed a random effects model in all analyses [38]. Confidence intervals for I² were calculated using Method III as described in Higgins and Thompson [39] using a spreadsheet. When performing such calculations in pooled analyses based on only two comparisons when Q≤k, we added the number 1 to both Q and k in order to avoid the mathematical problem of dividing by zero; this generally resulted in a slight shrinking of the confidence intervals under these conditions. Unless specified otherwise above, all analyses were performed using Comprehensive Meta-Analysis software [40]. We lacked adequate statistical power to perform subgroup analyses.

We examined the potential existence of publication bias by performing trim and fill analysis for pooled continuous depression outcomes. Trim and fill procedures examine potential asymmetry of effect sizes. Based on the assumption that effects are distributed symmetrically, trim and fill analysis imputes the number and likely effect size of missing studies, then recalculates the pooled analysis with imputed data from missing studies [41].

Results

Study Characteristics.

The evidence search flow is described in Figure 1. We obtained one controlled trial of aripiprazole that used low doses (2 or 5 mg); we did not include this trial because the starting dose of 2 mg was administered for 30 d prior to participants switching to the dose of 5 mg that falls within the recommended 5–10 mg range set by the FDA [42]. Characteristics of the 14 included studies are provided in Table 1. The definition of treatment-resistant depression differed somewhat across trials. The process by which diagnoses were made was described clearly in six trials, and the number of prior failed trials varied across studies. Only three studies clearly described their random-sequence generation procedures, and only one trial clearly described using clinical raters who were blind to both treatment assignment and participants’ reports of adverse events. While most trials used rating scales to assess for EPS and akathisia, and a minority of trials used a measure of sexual functioning, no trial reported using a structured instrument for eliciting a broad range of adverse events. All studies were funded by the study drug manufacturer except for one trial that was funded jointly by the study drug manufacturer and the US National Institute of Mental Health [27].

Efficacy.

In terms of remission, adjunctive treatment with each antipsychotic was associated with a statistically significant benefit, with ORs ranging from 1.42 to 2.37 (Table 2). ORs for response were also statistically significant for aripiprazole, quetiapine, and risperidone—but not for OFC (Table 2). The NNT for remission was nine for aripiprazole, quetiapine, and risperidone but was a substantially higher 19 for OFC (Table 2). NNTs for response were seven (aripiprazole), eight (risperidone), and ten (quetiapine). Pooled ORs are displayed visually in Figures 2 and 3 [43]. Among participants who achieved remission during treatment, participants assigned to OFC were less likely to remain in remission than participants assigned to placebo. Only two of 56 placebo participants relapsed, compared to 18 relapses among 99 participants on OFC (OR, 0.27; 95% CI, 0.08–0.90).

Pooled effect sizes for continuous outcomes are provided in Table 3. Adjunctive aripiprazole, quetiapine, OFC, and risperidone were all more efficacious than adjunctive placebo based on clinician-rated measures of depression severity (MADRS/HAM-D). Effect sizes were as follows: aripiprazole: g = 0.35 (95% CI, 0.23–0.48); OFC: g = 0.26 (95% CI, 0.04–0.45); quetiapine: g = 0.40 (95% CI, 0.26–0.53); and risperidone: g = 0.48 (95% CI, 0.22–0.73). The effects of risperidone may have been exaggerated by the reliance on post hoc analysis rather than a priori analysis in the largest study of the drug, as the effect of the drug was greater at 6 wk (g = 0.46) than at the prespecified primary end point of 4 wk (g = 0.32) [44]. According to convention, these effect sizes would be considered ‘‘small’’ or ‘‘small to moderate’’ in magnitude [45]. Effect sizes on depression severity measures did not differ significantly between drugs (Q_B=1.93, p = 0.59), though there was limited power to detect such differences. The pooled difference in mean change on the MADRS in the 11 trials that reported such data was 2.69. In these 11 trials, the mean effect size was g = 0.31, which differed only slightly from the overall mean effect size when including both the HAM-D and MADRS; thus, the 11 trials reporting MADRS mean change data seem representative of the entire sample of included trials. Only the trials of adjunctive aripiprazole reported self-reported depression symptom severity, yielding a very small effect size of g = 0.15. The effects observed on the Clinical Global Impressions–Severity Scale were either small or small-to-moderate, with the exception of risperidone, for which a moderate effect was generated.

With regards to quality of life and functioning, adjunctive quetiapine, aripiprazole, and OFC produced effect sizes that were either not statistically significant or small and clinically negligible in magnitude. Adjunctive risperidone was more efficacious than adjunctive placebo on quality of life/functioning, with a small-to-moderate effect size. The pooled effect across quality of life/functioning measures varied significantly across treatments (Q_B = 6.88, p = 0.003), with risperidone (g = 0.49) yielding a higher effect than the other three drugs combined (g = 0.11), which did not differ significantly from each other (Q_B = 4.02, p = 0.13). However, the effect of aripiprazole on quality of life/functioning was small (g = 0.22) and statistically significant (p = 0.001), whereas the effects of OFC (g = 0.04, p = 0.74), and quetiapine (g = 0.05, p = 0.53) were both not statistically significant and of quite small magnitude. The effect of aripiprazole on quality of life/functioning should be interpreted with caution, as the effect for the drug on the SDS was very small and no longer statistically significant when patients who violated study protocol were excluded from analysis (g = 0.12, p = 0.08). Similarly, the effect of risperidone on quality of life/functioning should be interpreted tentatively since it is largely driven by post hoc analyses.

Adverse Events.

Atypical antipsychotic medications differed substantially in their reported adverse event profiles. Table 2 reports adverse events that showed increased risk (p≤0.10). A more detailed listing of adverse events and pooled ORs for each event category are provided in Table 4.

Adjunctive aripiprazole was frequently associated with akathisia (NNH, 4; 95% CI, 3–6) and also linked to a statistically significant elevation in the occurrence of sedation (NNH, 14; 95% CI, 8–33) and significant weight gain of ≥7% during trials (NNH, 29; 95% CI, 10–119). Adjunctive OFC was often associated with significant weight gain of >10% or ≥10% (NNH, 9; 95% CI, 5–20), sedation (NNH, 5; 95% CI, 3–12), abnormal metabolic laboratory results (NNH, 10; 95% CI, 5–29), and elevated prolactin (NNH, 6; 95% CI, 4–11). Adjunctive quetiapine had a very high rate of reported sedation (NNH, 3; 95% CI, 2–3) and was also linked to abnormal metabolic laboratory results (NNH, 6; 95% CI, 4–9) and significant weight gain of ≥7% (NNH 37; 95% CI, 12–594). Adjunctive risperidone was not associated with an increased rate of any spontaneously reported adverse events.

All four drugs resulted in statistically significant weight gain (Table 3): mean weight gain in trials of adjunctive aripiprazole, quetiapine, and risperidone was approximately 1 kg, while the average weight gain resulting from adjunctive OFC was 4.20 kg (95% CI, 3.79–4.61). OFC was associated with more weight gain than the other drugs (Q_B = 58.46, p<0.001), which did not differ significantly from each other (Q_B = 0.66, p = 0.72).

The thresholds for adverse event reporting in the included publications are shown in Table 1. Adverse events were typically listed in a table and were reported only if a certain proportion of study participants reported that event. For example, if only those adverse events reported by 5% or more of participants in either group were reported in the published journal article, we describe it in Table 1 as ‘‘≥5% in any group.’’ In general, little to no additional information was provided in the study publications regarding adverse events beyond that which was presented in such tables. Meta-analysis of effects on sexual functioning rating scales was not performed because of the often unclear reporting of these measures (see Table 3).

Publication Bias.

The trim and fill procedure suggested the existence of three unpublished trials, bringing the overall effect on depression measures to 0.32. A funnel plot showing the results of this analysis can be seen in Figure 4.

Discussion

In this meta-analysis of 14 randomized trials of atypical antipsychotic medications used for the adjunctive treatment of major depressive disorder, we found that all included atypical antipsychotics were more efficacious than adjunctive placebo in terms of their effects on depressive symptom severity and remission. However, the effect sizes were small or small-to-moderate in magnitude, and OFC did not generate a statistically significant benefit on treatment response. All of the studied drugs except risperidone demonstrated substantial risk of several adverse events. Our findings have clinically important implications for comprehensively understanding the risk–benefit profiles of these adjunctive treatments for major depressive disorder.

The overall effect size on depression severity was g = 0.34, an effect conventionally deemed as small. In a meta-analysis of antidepressants versus placebo, Kirsch et al. found an effect size of 0.32, which they interpreted as not clinically relevant [46]. This was in line with the recommendations of the National Institute for Health and Clinical Excellence in the United Kingdom, which deemed effect sizes of g<0.5 as clinically insignificant, though no evidence was cited for this cutoff [47]. However, Turner et al.’s meta-analysis of antidepressants versus placebo found an effect size of 0.31 [11], which was interpreted as ‘‘measurable and significant’’ [48]. These differing interpretations are understandable given that Cohen noted that his original categorization of effect sizes (0.2 = small, 0.5 = medium, and 0.8 = large) was arbitrary [45]. We interpret the effect of adjunctive antipsychotic treatment on depression measures as of questionable clinical relevance. In addition, sole reliance on depression rating scales to determine treatment benefit is likely inadequate in understanding overall treatment efficacy.

The pooled difference in mean change across 11 trials was 2.69 points on the MADRS. The MADRS consists of ten items, each rated on a 0–6 scale, assessing sadness, inner tension, reduced sleep, loss of appetite, concentration, difficulty with starting daily activities, inability to feel, pessimism, and suicidal thoughts. A small difference favoring an atypical antipsychotic over placebo on the MADRS may thus reflect small differences across several dimensions, or perhaps a sizable difference on one or two dimensions combined with nil differences on other items. For instance, a pooled analysis of the two large quetiapine trials included in our meta-analysis found that quetiapine at 150 mg/d and quetiapine at 300 mg/d were superior to placebo by 2.50 and 2.85 points on the MADRS at study end point, respectively [49]. The treatment advantage in terms of the items ‘‘apparent sadness’’ and ‘‘reported sadness’’ appears to be about a third of that for the ‘‘reduced sleep’’ item (Figure 3b of [49]), suggesting that quetiapine’s sedative effect on sleep may account for a substantial degree of the observed improvement in depression scores. Thus, improvement in overall depression rating scales should be interpreted cautiously.

Response and remission rates are often used to convey the magnitude of treatment benefit; however, these categorical measures are created arbitrarily from underlying continuous rating scale data [50]. In some circumstances these categorical measures may inflate treatment differences relative to mean change on the continuous scale [51]. While response and remission rates are potentially useful outcome measures, they should be considered only in the context of a wider set of outcome data.

With the exception of risperidone, nearly all of the included trials estimated small or minimal benefits with regards to quality of life and functional impairment. Quetiapine and OFC generated no benefit on such measures, whereas the benefits of aripiprazole were statistically significant yet quite modest. Although risperidone appeared to possess the strongest risk–benefit profile in our analyses, our findings about risperidone were based on the smallest sample size of any of the included drugs. We also have concerns about data reported in the largest risperidone trial. The published version of the study emphasizes outcomes at the end of the 6-wk trial [44]. However, in its discussion section and the trial’s associated ClinicalTrials.gov registry entry [52], it is mentioned that the primary study end point was actually 4 wk; this is mentioned neither in the paper’s methods section nor in the abstract. The effect size on the HAM-D is 30% smaller at the 4-wk end point relative to the 6-wk end point. Effects on the Q-LES-Q and SDS were reported only at week 6, but it seems likely that these effects would be smaller at the primary study end point. Given that this study included 69% of the total participants in risperidone trials, our pooled estimate of risperidone efficacy is therefore driven by the inclusion of post hoc analyses. Further, a previously published relapse prevention study (not included in our meta-analysis due to its study design) found no benefit for risperidone over placebo, suggesting that risperidone-related gains may be transient [53,54].

Taken together, our findings raise significant concerns regarding the impact of these medications in improving overall well-being. Although improvements in quality of life or functional status commonly co-occur with improvements in depression symptom severity, this cannot automatically be assumed. One comprehensive literature review estimated only a moderate degree of correlation between these constructs [55]. It has been argued that changes on quality of life measures may lag changes on depressive symptom measures and that short-term trials may not be an appropriate setting in which to estimate changes on quality of life measures [15]. Contrary to this argument, however, four of five recently published short-term antidepressant medication trials found that benefits of medication over placebo were similar on measures of (1) quality of life or functional impairment (e.g., as measured by the Q-LES-Q and SDS) and (2) depression symptom severity (e.g., as measured by the HAM-D and MADRS) [56–60]. Our findings highlight the fact that reporting data only on symptom response and resolution may provide an incomplete and perhaps overly optimistic summary of a medication’s overall effects on well-being [15,16,61]. More robust assessments of quality of life and functional impairment should be incorporated into the design of clinical trials of any putative antidepressant.

Without longer-term data on not only depression symptom severity but also quality of life and social functioning, it is difficult to assess the risk–benefit profile of these medications prescribed over the long term. None of the included trials provided data on long-term (i.e., ≥6 mo) outcomes comparing adjunctive antipsychotic medication treatment to adjunctive placebo. Our failure to find long-term outcome data is consistent with that of previous research teams [17,62]. For example, one systematic review of long-term, two-arm parallel randomized controlled antidepressant trials initially identified 2,693 abstracts, only to ultimately include six trials [62]. This limitation is shared with other treatments; there is very little understanding of how adjunctive treatments for depression influence long-term well-being.

In addition to providing a thorough assessment of efficacy outcomes, our meta-analysis departs from the literature in a second notable way by comprehensively summarizing the available safety information on these medications. Such safety data have not been included in prior quantitative reviews, but our conclusions echo concerns raised in previous meta-analyses and a narrative review regarding potential treatment-related harms associated with use of atypical antipsychotic medication in the adjunctive treatment of depression [6–8]. Overall, we found that treatment was linked to several adverse events, including akathisia (aripiprazole), sedation (quetiapine, OFC, and aripiprazole), abnormal metabolic laboratory results (quetiapine and OFC), and weight gain (all four drugs, especially OFC). Measures of absolute benefit and harm (NNT and NNH) provide an intuitive metric for understanding treatment-related benefits and harms. However, these measures are dependent on baseline control group risk, which may vary substantially across clinical subgroups [63]. Thus, our findings in terms of NNT and NNH should be interpreted as estimates of effects for each drug relative to control participants who may differ from participants treated in clinical practice.

Our ability to provide an adequate safety profile of these medications was limited in two respects. First, while 11 of 14 included trials used a structured instrument to elicit adverse events, these measures were limited to assessing potential EPS- and akathisia-related events, and, in five studies, sexual functioning. No study reported using a structured checklist to elicit adverse events outside of EPS, akathisia, or sexual functioning, which is a substantial limitation given that adverse events are reported with as much as 20 times greater frequency when elicited through structured checklists versus being recorded in response to patient complaints [19,20]. The importance of measuring adverse events systematically was demonstrated historically in the case of selective serotonin reuptake inhibitors: in registration trials, sexual dysfunction was neither systematically assessed nor found to be frequently spontaneously reported by patients [64,65]. Further investigation indicated, however, that sexual side effects on selective serotonin reuptake inhibitors are actually quite common [19]. While the collection of adverse event data via structured checklists is a more sensitive method of collecting adverse event data, it may result in many common (mostly minor) health problems being endorsed even if they are not due to treatment, potentially leading to decreased specificity in differentiating medications from placebo [66]. To bridge the differences between the systematic and open-ended assessment of adverse events in clinical trials, some sort of hybrid method of collecting adverse event data could be performed, such as randomly assigning some participants within both the active treatment and placebo groups to complete a structured checklist while assigning others to complete an open-ended assessment of adverse events.

A second constraint on our ability to adequately summarize the drugs’ safety profiles is that many adverse events were not reported in journal articles and that some of the data were incomplete or reported in a fashion that may have obscured treatment-related harms. We agree with the Cochrane reviewers that ‘‘data on side effects were often very poorly described’’ [9]. Conceptually similar events such as sedation, fatigue, and somnolence were sometimes reported separately, often with no attempt to pool them together. This is in direct contradiction to FDA guidance, which states that events that ‘‘represent the same phenomenon (e.g., somnolence, sedation, drowsiness) should ordinarily be grouped together as a single adverse reaction to avoid diluting or obscuring the true effect’’ [67].

Given the notable side effect profiles of the studied drugs, it is likely that the double-blind was significantly compromised; however, none of the included trials tested the integrity of blinding. For example, patients who rapidly gained weight in an OFC trial, who complained of akathisia in an aripiprazole trial, or who reported sedation in a quetiapine trial would likely cue the awareness of study personnel that they were assigned to the active drug condition. Assuming that proper informed consent was obtained, participants were also likely to accurately guess their treatment assignment based on side effect cues [68,69]. This could have led to inflated efficacy ratings by clinical raters and participants [70,71]. The lack of protocols assessing the integrity of the double-blind in the trials included in our meta-analysis is consistent with the wider clinical trials literature [72]. The potential for unblinding to cause inflated efficacy ratings among clinical raters could be substantially limited if efficacy outcomes were assessed by different personnel than those who assessed adverse events [70]. Yet the use of separate raters to assess efficacy and safety outcomes was reported in only one trial [73].

The FDA statistical reviewer for aripiprazole [74] wrote regarding Berman et al. [75] that ‘‘the medical reviewer is concerned about the considerable number of protocol violations in the study primarily due to usage of opiates/barbiturates’’ [74]. Regarding the Marcus et al. [76] trial, the FDA reviewer wrote that the difference between groups in the number of participants who used prohibited medications was ‘‘huge’’ [74], with nine patients in the placebo group doing so compared to 24 in the aripiprazole group. The reviewer thus performed a separate analysis, excluding patients in the two trials who violated the study protocol, the results of which indicated a minimal, non-statistically significant effect of aripiprazole on functional status. In the journal articles, this potentially important issue is not mentioned. The FDA reviewer reported results only from reanalysis of the MADRS and SDS, so it is unknown to what extent these protocol violations may have impacted results on other outcome measures [74].

Our results differ somewhat from those of Nelson and Papakostas, whose meta-analysis concluded that augmentation with atypical antipsychotics was effective and, further, that ‘‘this body of evidence is considerably larger than that for any other augmentation strategy in the treatment of major depressive disorder’’ [6]. There are seven differences in our analyses that provide reasons why we reached different conclusions. The greatest divergence in our results was regarding OFC, for which we found a lower OR favoring OFC for remission (1.42) than did Nelson and Papakostas (1.83). In this first instance, Nelson and Papakostas utilized whatever definition of remission was provided by the authors of each study, whereas we used a more restrictive definition. Three OFC trials defined remission as achieving a MADRS score of ≤8 at two consecutive visits—even if patients relapsed during the trial [25–27]. We found that after meeting criteria for remission, OFC-treated participants were more likely to relapse than placebo-treated participants; this contributed to our finding a less favorable result for OFC in terms of remission. Second, we extracted data from all comparison groups that received adjunctive placebo treatment, whereas Nelson and Papakostas excluded one comparison group from each of two OFC trials [25,26]. Third, Nelson and Papakostas estimated a significant treatment effect for OFC on response, whereas we did not. This difference seems due to a combination of our inclusion of all adjunctive placebo comparison groups and our use of random effects analysis as opposed to their use of a fixed effects model [38]. Our fourth point of difference was that Nelson and Papakostas included data from two conference presentations on quetiapine that showed positive findings; we were unable to obtain data from these authors despite three emailed requests over a span of 4 wk. Additionally, we attempted to contact one author via phone; the attempt did not result in the release of any data. Nonetheless, the pooled ORs generated in our analyses for quetiapine in terms of response (1.53) and remission (1.79) were quite similar to those published in Nelson and Papakostas’s meta-analysis (1.60 and 1.89, respectively). Our fifth difference was the use of different definitions of remission in one risperidone trial [73], and the sixth difference was Nelson and Papakostas’s inclusion of data from one small risperidone trial from which we were unable to extract remission data [77], leading to our finding a slightly lower rate of remission (OR of 2.37 versus 2.63). Lastly, and most importantly, the primary point of difference is that our analysis provides a more comprehensive appraisal of treatment efficacy and safety, which, as discussed above, presents a more accurate assessment of the comparative risks and benefits of treatment.

Our review adds to the Cochrane review on this topic [9] by filling in three important data gaps: (1) unpublished data from the FDA and clinical trial registry reports, (2) data on functioning and quality of life outcomes, and (3) data on metabolic laboratory parameters. Thus, our dataset contained more outcomes and often provided a more comprehensive assessment of included outcomes than the Cochrane review. For instance, the Cochrane review included data from one trial that reported data on clinically significant weight gain for patients on OFC, whereas we included data on both mean weight changes and binary measures of clinically significant weight gain from four such trials. We included laboratory data for several metabolic parameters for both quetiapine and OFC. Despite some differences in methodology, we agree with the Cochrane review that the evidence supporting the use of adjunctive atypical antipsychotics for depression is modest.

Several methodological issues also bear mention. First, while all trials were described as randomized, double-blind trials, only three trials clearly described adequate sequence generation procedures; in the remaining studies, such procedures were unclear. A lack of appropriate randomization or differences in the taste, smell, or appearance of the medication and placebo may allow study personnel and/or participants to guess their treatment assignment. As purportedly double-blind trials with unclear or inadequate randomization are associated with larger effects than trials in which adequate randomization is clearly described, this leads to the possibility that the current set of efficacy ratings were inflated to an unknown extent [21,78]. Second, the design of some of the included trials may have compromised their validity. In each of the aripiprazole trials, patients were treated with an antidepressant plus adjunctive placebo for 8 wk; at that point, those who showed a treatment response were eliminated from the study, and the remaining patients were assigned to either remain on the same treatment or receive adjunctive aripiprazole in place of adjunctive placebo. Thus, all patients taking placebo during the randomized trial had clearly demonstrated poor response to placebo treatment and were likely predisposed to perform poorly during the randomized portion of the trial, thereby possibly inflating the estimated efficacy of the study drug [79].

In any systematic review, publication bias is a potentially serious problem [10,80]. To incorporate as much data as possible, we conducted a thorough literature search and included unpublished data. We did not uncover the existence of any additional unpublished negative trials in our search, but this does not mean that such trials do not exist. Given the small number of trials for each drug in our study, we lacked statistical power to conduct a formal analysis of publication bias for each drug. However, when pooling across drugs, we detected that publication bias may have slightly enhanced the overall effect size on depression measures. Our results likely represent an upper boundary for the efficacy of these compounds (as demonstrated in prior meta-analyses), assuming that relevant unpublished data are more negative than positive in terms of efficacy [10,11].

We are aware of no trials that have directly compared adjunctive atypical antipsychotic medication treatment to other adjunctive treatments such as psychotherapy or lithium, or to other treatment strategies such as switching the antidepressant medication initially used for treatment. Further study may answer critical outstanding questions regarding the safety profiles and longer-term outcomes associated with these medications. Taken together, our meta-analysis found evidence of (1) some improvement in clinician-assessed depressive symptoms, (2) little evidence of substantial benefit in overall well-being, and (3) abundant evidence of potential treatment-related harm. Our comprehensive evaluation of safety and both relative and absolute efficacy provides critical insight that may be useful for clinicians attempting to thoroughly understand the risk–benefit profiles of these adjunctive treatments for major depressive disorder.