Assessment of Selective and Universal Screening for Suicide Risk in a Pediatric Emergency Department

INTRODUCTION

Suicide is the 10th leading cause of death in the United States and the 2nd leading cause of death among youth aged 10 to 19 years.¹ Suicide incidence among adolescents has increased in the United States over recent decades, with pronounced increases among girls aged 10 to 14 years² and black children aged 5 to 12 years.³ To successfully identify suicide risk, public health efforts must reach a broad proportion of those at risk for suicidal behavior.^4,5 One approach is to screen for suicidal thoughts and behaviors in settings that may be accessed by individuals during high-risk periods,^6,7 such as following emergency department (ED) discharge.^8,9 The Joint Commission (TJC) released a Sentinel Event Alert¹⁰ recommending suicide risk screening in all individuals being treated or evaluated for behavioral health conditions as their primary reason for care in TJC-accredited hospitals and behavioral health care organizations and requires all health care organizations to use validated screening tools and other procedures for patients at risk of suicide as part of its accreditation process as of July 1, 2019. Despite the Sentinel Event Alert, many psychiatric EDs rely on clinical judgment of risk of harm to self and/or others^11,12; medical EDs are even less objective and systematic.¹³ In addition, there are limited data on the predictive ability of the suicide risk screening tools recommended by TJC.

The Ask Suicide-Screening Questions (ASQ), a screening instrument that can be rapidly administered by hospital staff without specialized training in less than 2 minutes,¹⁴ could facilitate more widespread selective and universal screening approaches if shown to be associated with suicide-related outcomes compared with current procedures for identifying suicide risk. We previously found that the ASQ, implemented as a selective screening tool for youth presenting with behavioral or psychiatric presenting problems to a pediatric emergency setting, identified suicide risk among youth who had not otherwise reported suicidal thoughts or behaviors and that positive screens were associated with subsequent suicide-related return visits.¹⁵ Furthermore, the ASQ has been shown to identify high rates of undetected suicidal thoughts among youth aged 10 to 12 years.¹⁶ The ASQ has not yet been tested as a universal screen, although other universal screening methods have been found to be feasible with adult patients.¹⁷ While the ASQ has been screened in small convenience samples of patients with medical and psychiatric problems,¹⁴ to our knowledge, no study has reported on the association between results of universal suicide risk screening as routine care in a pediatric population and suicide-related outcomes.

The present study tested the association of positive ASQ screens with subsequent suicide-related outcomes among both a selective screening sample (ie, patients aged 8 to 18 years with a behavioral or psychiatric presenting problem) and a universal screening sample (ie, to patients aged 10 to 18 years with a medical presenting problem in addition to those aged 8 to 18 years presenting with a behavioral or psychiatric concern) in a pediatric ED. The aims were to (1) determine the association of ASQ screening results with subsequent return to the ED for suicide-related reasons or deaths by suicide; (2) evaluate whether the ASQ statistically improves on the association between the presenting problem and suicide-related outcomes; (3) determine whether the ASQ identifies risk among youth with no known risk of suicide; and (4) compare the ASQ’s effectiveness between universal and selective screening conditions.

Methods

This was a retrospective cohort study of a consecutive case series of patients in the Johns Hopkins Hospital Pediatric ED from March 18, 2013, through December 31, 2018. Implementation was sequential, starting with selective screening of patients presenting to the ED for psychiatric or behavioral concerns in March 18, 2013, to December 31, 2016, and then broadened to universal screening in January 1, 2017, to December 31, 2018. Selective screening included patients aged 8 to 18 years with a behavioral or psychiatric presenting problem and the universal approach screened everyone aged 10 to 18 years in addition to those aged 8 to 18 years with behavioral or psychiatric problems. Nurses in the ED administered the ASQ as standard of care during triage (eAppendix 1 in the Supplement). The Johns Hopkins Pediatric ED is part of an urban academic pediatric medical center with approximately 30000 patient visits per year. No patients were excluded on the basis of gender, minority status, or insurance type. If a patient screened positive on the ASQ, the ED physician was notified and patients were given additional evaluations and referrals as deemed appropriate, including emergency psychiatric evaluations when necessary, consistent with treatment as usual for potential suicide risk. The medical record review and evaluation of impact of suicide risk screening with the ASQ as routine care was approved by the Johns Hopkins School of Medicine institutional review board. Because screening was implemented as routine care, informed consent was not necessary. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies. For further details on ED screening procedures, see eAppendix 2 in the Supplement.

Screening Assessment

The ASQ is a 4-item nonproprietary suicide risk screening instrument that can be administered to patients in the pediatric ED by nurses, regardless of psychiatric training.¹⁴ A positive response to any of the questions is considered a positive screen, whereas a negative screen requires negative responses to all questions. In the initial development study, the ASQ was found to have a sensitivity of 96.9%, a specificity of 87.6%, and a negative predictive value of 99.7% for patients with medical and surgical concerns and 96.9% for patients with psychiatric concerns.¹⁴

Measures

The primary exposure variable was binary, indicating a positive or negative ASQ screen at the index visit. Suicide-related presenting problem, based on the presence of suicidal ideation or attempt, was indicated with a binary variable. Presenting problems were mutually exclusive and limited to 1 problem per visit. Suicide-related presenting problems at the index visit were included in regression and survival models as an independent variable. Suicide-related presenting problems at follow-up visits were used as the outcome variable, combined with suicide deaths. To determine death by suicide, the ASQ database was matched with state death records of individuals aged 8 to 24 years between 2013 and 2018.

Additional measures abstracted from the electronic health record and included as covariates were demographic characteristics (age, gender, and race/ethnicity) and disposition at initial visit, which was collapsed into (1) discharged, (2) admitted or transferred, and (3) other, including leaving against medical advice. Race/ethnicity was coded as a single variable indicating non-Latino white, non-Latino black, Latino, or other.

Statistical Analysis

Baseline Characteristics

To identify baseline characteristics associated with positive ASQ screens, respondents with positive and negative screens were compared on demographic variables (age, race/ethnicity, gender) and clinical variables (presenting problem, disposition) using χ² analyses and t tests. These analyses were conducted separately for the universal and selective subsamples.

Survival Analyses

Universal and Selective Subsamples.

Cox proportional hazards regression analyses were used to test for the risk of subsequent suicide death or ED visits for suicidal thoughts or behavior, defined based on electronic health record documentation of the presenting problem, using all available follow-up data. Follow-up period was determined based on number of days between the initial ED visit and either suicide death or subsequent ED visit due to suicidal ideation and/or behavior; in the absence of either of these events, data were censored as of December 31, 2018, the last date of available death record and clinical follow-up data. Survival analyses were conducted separately for the selective and universal subsamples, with adjustments in the first block of independent variables for sociodemographic characteristics (race/ethnicity and gender) and adjustments in the second block for baseline clinical variables (suicide-related presenting problem at index visit, disposition at index visit). Adjustment was made for the presenting problem to test whether the ASQ was associated with suicide-related outcomes above and beyond the presenting problem. An additional Cox regression of the entire sample with the inclusion of independent variables indicating universal vs selective screening, as well as an interaction term for universal screening × screening outcome, was run to test whether there was a significant difference in the association of positive ASQ screens with outcomes between the universal and selective screening conditions. All measures of association are presented as hazard ratios (HRs) with 95% confidence intervals and were considered significant at 2-tailed α = .05. Analysis was conducted using SPSS statistical software version 25 (IBM).

Unadjusted Kaplan-Meier survival curves are presented for data visualization, comparing time to subsequent suicidal behavior for positive vs negative screens. To illustrate the unique contribution of screening to the identification of risk for subsequent suicidal behavior, additional Kaplan-Meier survival curves were constructed to compare participants who (1) had negative screens and did not present with suicidal thoughts or behavior, (2) had negative screens but presented with suicidal thoughts or behavior, (3) had positive screens but did not present with suicidal thoughts or behavior, and (4) had positive screens and also initially presented with suicidal thoughts or behavior.

Psychiatric and Nonpsychiatric Universal Subsamples.

An additional set of survival analyses were run to test the association of ASQ results with suicide-related outcomes among the universal screening subsample, divided into participants with and without a psychiatric presenting problem. The analyses among participants without psychiatric problems were intended to clarify whether the ASQ would be of value among participants presenting with a medical problem who would likely not be selectively screened for suicide risk.

Short-term Relative Risk

To be consistent with past literature,^7-9,18 we focused on the risk period of 3 months following the initial visit. Among the 13746 respondents with at least 3 months of follow-up, we calculated relative risk and risk differences associated with (1) positive ASQ screens, (2) suicidal thoughts or behavior as the initial presenting problem, and (3) the presence of either of the first 2 indicators (ie, to determine the relative risk based on an either-or scenario in which a respondent presented with either of the 2 indicators of suicide risk at baseline). Risk was calculated separately for the selective and universal groups and in the total composite sample, along with positive and negative predictive value, sensitivity, and specificity.

Exploratory Analyses

Exploratory analyses included the following: (1) overall study procedures; (2) relative risk calculations for death by suicide; (3) demographic comparisons between selective and universal subsamples; (4) full frequency data for the short-term relative risk calculations; and (5) demographic comparisons between respondents who did vs did not present to the ED with suicidal behavior as their presenting problem, among those who then screened positive on the ASQ. In addition, relative risk was calculated using only death by suicide as an exploratory analysis, given the low prevalence of this outcome.

Results

Sample Characteristics

The ASQ was administered to a total of 15003 participants (7044 [47.0%] male; 10209 [68.0%] black; mean [SD] age, 14.5 [3.1] years at baseline) across the entire study period, with 4666 participants screened during the selective screening phase (2013-2016) and 10337 screened during the universal screening phase (2017-2018). In the selective screening group, 2089 participants (44.8%) identified as male, and 3188 (68.3%) identified as black; mean (SD) age was 14.0 (3.1) years. Similarly, in the universal group, 4955 (47.9%) identified as male and 7021 (67.9%) as black; mean (SD) age was 14.7 (3.2) years. The selective and universal subsamples were temporally distinct and varied on several demographic factors (eFigure in the Supplement). Patients were followed up for a mean (SD) of 1133.7 (433.3) days in the selective screening condition and a mean (SD) of 366.2 (209.2) days in the universal condition. In the selective condition, there were 275 suicide-related ED visits and 3 deaths by suicide. In the universal condition, there were 118 suicide-related ED visits and no deaths during the follow-up period.

The protocol compliance rate (ie, percentage of eligible participants who were actually screened) ranged from 59% to 81% during the selective screening phase and 80% to 86% during the universal screening phase. Of all screened participants, 806 (7.8%) screened positive in the universal subsample, and 1435 (29.9%) screened positive in the selective subsample. Among both subsamples, positive screens were significantly more common among female respondents, respondents who were transferred or admitted, and those whose presenting problem was suicidal ideation or behavior (Table 1). The distribution of positive screens varied by race/ethnicity for the selective sample only, with white respondents most likely to screen positive and Asian respondents least likely (Table 1). There were no age differences in screening outcome for either sample (Table 1). Frequencies of positive screens and suicide-related presenting problems are shown in Table 2. In the combined sample, 150 out of 1012 youths (14.8%) who had a suicide-related presenting problem screened negative on the ASQ. Notably, 1229 of 2241 patients (54.8%) who screened positive on the ASQ did not report suicidal ideation or behavior as their presenting problem. This group, whose suicide risk may have otherwise been undetected, were disproportionately more likely to be male (61.0% of male patients with undetected risk vs 51.4% of female patients; χ²₁ = 19.3; n= 2241; P < .001) or black (57.5% undetected) or Latino (54.5% undetected) compared with white (49.9%), other race (49.5%), or Asian (44.4%) (χ²₄ = 11.7; n= 2241; P = .02). Undetected risk did not vary by age (detected: mean [SD], 14.3 [2.5] years; undetected: mean [SD], 14.2 [2.9] years;t₂₂₃₉ = 0.8; P = .45).

Survival Analyses for Universal and Selective Subsamples

Both positive screens on the ASQ and suicide-related presenting problems at the index visit were significantly associated with subsequent visits for suicidal ideation or behavior (including death by suicide) in both the universal (HR, 6.8 [95% CI, 4.2-11.1]) and selective (HR, 4.8 [95% CI, 3.5-6.5]) screening conditions (Table 3). For the selective screening condition only, the likelihood of subsequent suicide-related ED visits varied significantly by race/ethnicity (HR for Latino vs non-Latino respondents, 1.7 [95% CI, 1.1-2.8]) and was higher for those who were admitted during their baseline visit (HR vs discharged, 1.4 [95% CI, 1.1-1.8]). Notably, the HR for a positive ASQ screen was numerically higher in the universal subsample compared with the selective subsample, and this subsample × screen interaction was statistically significant when tested in the full sample (Wald χ²₁ = 6.2; P = .01). Kaplan-Meier curves illustrate the duration of follow-up until the first visit to the ED for suicide-related reasons, comparing both (1) respondents with positive vs negative screens on the ASQ and (2) all 4 combinations of presenting problem and screening outcome (Figure 1). When the universal subsample was divided based on whether the presenting problem was psychiatric vs nonpsychiatric, positive screens on the ASQ were associated with subsequent visits for suicidal ideation and behavior for both the psychiatric subsample (HR, 2.2 [95% CI, 1.2-4.1]) and the nonpsychiatric subsample (HR, 7.1 [95% CI, 3.2-15.8]) (eTable 1 in the Supplement).

Short-term Relative Risk

Relative risk was calculated based on the likelihood of returning to the ED for suicide-related problems within 3 months of the initial index visit. For the selective subsample, universal subsample, and combined total sample, a positive ASQ screen combined with a suicide-related index problem yielded the greatest relative risk, followed by a positive ASQ screen alone, and finally by a suicide-related index presenting problem with a negative ASQ screen (Table 4). Positive and negative predictive values were similar between screening and presenting problem approaches to identifying suicide risk, although screening consistently yielded higher sensitivity but lower specificity (Table 4). For complete cross-tabulated data, see eTable 2, eTable 3, and eTable 4 in the Supplement.

Death by Suicide

There were 3 deaths in the selective screening subsample and none in the universal subsample. The relative risk for confirmed death by suicide in the selective screening subsample was 4.5 (95% CI, 0.4-49.6). For further detail, see eAppendix 3 in the Supplement.

Discussion

Main Findings

Our study found that positive screens on the ASQ in a pediatric ED were associated with subsequent suicidal behavior, defined as either suicide-related ED visits or death by suicide, and that this association was both statistically independent of, and of greater magnitude than, the association of the index presenting problem alone with suicide-related outcomes. The ASQ was particularly effective at identifying risk for subsequent suicide-related ED visits among patients who had no psychiatric presenting problems during their index visit. While there were only 3 confirmed deaths by suicide in this cohort, 2 of the participants who died had screened positive on the ASQ, translating to a statistically nonsignificant relative risk of 4.5 (95% CI, 0.4-49.6).

The ASQ identified 1229 patients (54.8% of positive ASQ screens) whose suicidal thoughts or behaviors would have been undetected through standard hospital procedures, particularly male patients and black patients. A previous study of universal screening in adult EDs yielded a similar rate and demographic distribution of undetected suicide risk.¹⁷ The improved detection of suicide risk in black youth is especially important given that the suicide rate has increased among elementary school–aged black youth over the past few decades.³ Conversely, 14.8% of those with an index presenting problem related to suicide risk screened negative on that same visit, which may represent the subset of youth who come to the ED for suicide-related reasons but either are no longer experiencing these thoughts by the time the ASQ is administered or chose not to speak about their suicidal ideation or behavior.

Implementation Considerations

The direct comparison of selective vs universal screening and the analysis of psychiatric and nonpsychiatric subgroups within the universal subsample can inform best practices around suicide risk screening in pediatric EDs. Our finding that the HR was greatest for universal screening, particularly for universal screening of patients with nonpsychiatric presenting problems, takes The Joint Commission’s 2016 Sentinel Decree¹⁰ that health care professionals should use selective screening procedures 1 step further to show the value of universal screening. While universal screening is more labor intensive and can seem to be not clinically indicated or possibly irrelevant to medical patients, suicide risk screening was contextualized by nurses at triage as a patient safety effort and, overall, most clinicians in this ED were able to include the ASQ in their triage routines. Our results indicate that the ASQ was strongly associated with suicide risk among youth with medical or surgical presenting problems, which is particularly important as these patients may be more likely to slip through the cracks when detecting suicide risk and connecting to mental health care. Notably, our finding that the effect size for suicide-related presenting problems was very small after adjusting for ASQ results suggests that index presenting problems are a weak and insufficient indicator of subsequent suicidal behavior compared with suicide risk screening, at least when used in isolation.

Strengths and Limitations

Strengths of this study include real-world comparison of the consecutive implementation of selective and universal screening approaches within the same setting as routine care. The availability of death records to supplement the hospital-based outcome measures was also a strength. However, one limitation was that the death records yielded an insufficient number of cases for reliable analyses and were limited to in-state deaths. We intend to revisit this outcome pending a longer follow-up period. We did not have access to data on mental health treatment after screening and thus could not study whether identification of suicide risk by screening resulted in greater engagement in mental health services and reduced future risk for suicidal behaviors.

An additional limitation is that medical records indicate that some medical and surgical patients were given the ASQ in the selective screening condition even though the protocol specified that it was to be given only to patients with presenting psychiatric problems. This may have inflated the HR (4.8) for the selective screening condition (based on comparison with the HR for the psychiatric-only subsample of the universal screening condition [2.2]), although this may more accurately reflect the real-world implementation of such screening tools, which may not be done with complete fidelity, particularly in busy ED settings. Furthermore, at least some of this variance from protocol was due to clinical judgment, as nursing staff sometimes administered the ASQ in the selective screening condition to medical and surgical patients who they felt to be at risk, although this was not consistently documented. Some youths were not screened even though they met inclusion criteria, which likewise reflects our use of a real-world ED setting for this study; youth sometimes cannot be fully assessed in ED triage due to aggressive behavior, cognitive issues, lack of responsiveness, or urgent need for medical attention, while other missed screens may have simply been due to oversight. Follow-up data were not available for those who were not screened, who may have varied from the screened samples in terms of subsequent risk for suicidal behavior.

This is also a single-site study, and we do not have data on patients who went to another hospital for subsequent ED visits. Because the selective and universal protocols were implemented consecutively as routine care based on a hospital policy change, the selective screening group had a longer follow-up period. We chose to include all participants rather than set a minimum follow-up period for the survival analyses, although a sensitivity analysis limiting the sample to individuals with at least 3 months of follow-up yielded similar results. Also, real-world implementation factors, such as the parents being present for some ASQ assessments, may have negatively affected the reliability of patient reports while simultaneously enhancing the external validity of the study.

Conclusions

These data provide convincing evidence that implementing standardized suicide screening in a pediatric ED is an effective means of identifying otherwise undetected risk for suicidal behavior. Although screening with the ASQ was associated with future suicidal behaviors, enhanced follow-up, care coordination, or other community-based prevention approaches are needed to prevent future suicidal behaviors in our patient population.^19-23 While the benefit in detection was greater for universal screening, both universal and selective screening resulted in improvements in detection of risk compared with treatment as usual. The increased detection of suicidal behavior among black youth may help address some emerging racial disparities in childhood suicide rates.³