A Scoping Review of Measures Used in Early Intervention Services for Psychosis

Psychotic disorders usually begin in late adolescence or early adulthood and have profound personal, family, social, and economic impacts (1, 2). Consensus has emerged over three decades that high-quality care in early intervention services (EISs) and reduced treatment delay may improve outcomes, especially in the first episode of psychosis (3–5). The EIS model has been widely implemented in Australia, Canada, Denmark, Hong Kong, Singapore, and the United Kingdom (6). Similar models have also been rolled out in the United States (7–10).

EIS guidelines identify elements and evidence-based standards of care essential to consistent high-quality services (11, 12). These standards include early and rapid access to care; use of low-dose second-generation antipsychotic medications; and a customizable package of evidence-based, recovery-oriented interventions focused on social and occupational functioning and the mitigation of symptoms and distress. The patient-oriented vision central to this model focuses on patient agency and responsibility for care, recovery, and individual life prospects.

Despite superior outcomes in EIS for first-episode psychosis (FEP) (5, 13), achieving improvements such as sustained service engagement or (re)integration in employment or school for all patients remains challenging (14, 15). Multidisciplinary research has also revealed gaps between actual implementation and evidence-based guidelines and best practices related to EIS (16–18).

Substantial and varied research in EIS has focused on measuring treatment delay (19, 20); pathways to care (21); symptomatology and clinical, functional, or long-term outcomes (22, 23); predictors and durability of post-EIS outcomes (24, 25); and service performance (4, 26, 27). Although this research has advanced understanding of progression, outcomes, and treatments, measurement-based care has not been consistently deployed in the everyday practice of EIS. Furthermore, measurement-based care itself remains an underresearched topic in the EIS field (28, 29), undermining the potential to identify patients who are making suboptimal progress or to improve interventions and outcomes.

We began by adopting Aboraya and colleagues’ (30) definition of measurement-based care: “The use of validated clinical measurement instruments to objectify the assessment, treatment and clinical outcomes, including efficacy, safety, tolerability, functioning, and quality of life, in patients with psychiatric disorders.” In this review, we broadened this definition to refer to the use of systematic data collection to monitor individual-level treatment and directly inform both care-related decisions (29, 31–35) and the standard of care in services (30, 36) by allowing aggregation of patient data (37). Furthermore, service-level measures, such as fidelity monitoring (11), can be pertinent in measurement-based care because they can promote individual- and service-level adherence to best practices and clinical standards in EIS.

The benefits of measurement-based care for clinical decision making and quality improvement have been increasingly recognized (29, 31, 33, 35). However, to support clinicians with evaluation of and ongoing adjustments to treatment, measurement-based care and assessments must be administered and analyzed systematically and continuously for every patient at the start of treatment. Such sustained measurement can also enrich research on illness progression, outcomes, long-term recovery, and service design (31, 33, 35, 38–40).

Patient-reported outcome measures (PROMs) (37, 41–43) and patient-reported experience measures (PREMs) are rich sources of information on care outcomes and experiences from the patient’s perspective (44). PROMs and PREMs, together with similar family- and clinician-reported measures and more traditional information sources such as clinical-administrative data, can yield a comprehensive assessment of outcomes, quality of care, and service provision (37, 41). Especially in EIS, which emphasizes engagement of patients and families (45, 46), the inclusion of PROMs, PREMs, and family-reported measures is crucial (37, 41).

As an important first step in advancing comprehensive measurement-based care in EIS, we conducted a scoping review to identify and synthesize measures reported in the EIS literature with respect to their purpose (i.e., to assess patients, families, or programs), the domains covered (e.g., symptoms, functioning, quality of life), and reporting perspectives targeted (i.e., those of the patient, family, or clinician). These three categories were selected a priori as effective and efficient ways to describe, synthesize, organize, and assess the types of data gathered for each measure. In addition to informing measurement-based care, this review sought to facilitate the standardization of measures used in EIS services.

Methods

Study Design

This scoping review was part of a larger knowledge synthesis and implementation project, the Meaningful Assessment Protocol (MAP) study. We followed a five-stage methodological framework for scoping reviews (47) and organized the review on the basis of the PRISMA Extension for Scoping Reviews (PRISMA-ScR) checklist (48). (The checklist is available in the online supplement to this review.)

Research Questions

The following questions guided the research: What measures are used in EIS on the basis of peer-reviewed studies published between 2000 and 2020? What are their purposes (i.e., to assess patients, families, or programs)? What domains (i.e., foci of data collection) are assessed? From whose perspectives (patient, family, or clinician) are these measures reported?

Identification of Eligible Studies

A health sciences librarian (J.B.) developed the search strategy and performed searches in MEDLINE, EMBASE, PsycINFO, CINAHL, Cochrane Trials, and Proquest Dissertations and Theses from January 1, 2000, to September 1, 2020, without language restrictions. The MEDLINE strategy was developed with input from the project team to identify pertinent literature on EIS for psychosis and was peer reviewed by a second librarian by using the PRESS (Peer Review of Electronic Search Strategies) standard (49). Once finalized, the MEDLINE strategy was adapted to other databases (the full search strategies are available here: https://doi.org/10.5683/SP3/DUGJMW). Results from each database were exported to EndNote, and duplicates were removed. Publications were required to be in English, peer reviewed, published between 2000 and 2020, and pertinent to EIS for psychosis and to include one or more measures. Conference proceedings, dissertations, commentaries, letters to the editor, non–full text publications, evaluations, and descriptions of single interventions (e.g., cognitive-behavioral therapy [CBT]) or scales (e.g., psychometric properties) were excluded. Articles on comorbid general medical conditions, pharmacological management, and neurocognitive assessments were excluded because previous reviews and guidelines have been published on these topics (50–54).

Data Extraction

Four reviewers, working independently in pairs, performed different stages of the review process. Title and abstract as well as full-text screening were conducted with Covidence software (55). Reviewers screened titles and abstracts for each retrieved citation and then reviewed the full texts of articles for inclusion criteria. Data extraction was performed on all included articles by using a standardized form (see Key Items Forming Part of the Data Extraction Templates in the online supplement). Disagreements regarding data screening or extraction were resolved by discussion and consensus between reviewers or, failing consensus, by two authors (M.F. and S.N.I.) who trained and closely supervised the review team.

Articles included in this review comprised clinical and services research (trials and programs or service evaluations), study protocols, economic evaluations, guidelines, and fidelity assessments. Standardized data extraction forms were created for each type of article by using categories covered in pertinent critical appraisal tools and checklists: SQUIRE (Standards for Quality Improvement Reporting Excellence) (56) for program evaluations, SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) (57) for study protocols, and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) (58) for economic evaluations. This process involved the creation of data extraction forms sensitive and relevant to each type of study and standardization of the extraction process, providing methodological rigor. Although the checklists were heterogeneous and some had inconsistent scoring information, no studies were excluded on the basis of quality because our primary goal was not quality assessment but the identification of measures used in EIS.

The extracted data were entered into a Microsoft Excel spreadsheet. Items extracted included study characteristics (title, authors, year, and journal); location and description of EIS; study aims, methods, and measures used; purposes and covered domains of the measures; stakeholders (patient, family, or clinician) from whose perspectives the measures were reported; administrative procedures; and psychometrics. Measures of general medical health, medication adherence, and adverse effects as well as neurocognitive assessments were excluded because articles dealing with these topics were also excluded from this review.

Summary and Reporting

Study characteristics were summarized as frequencies and percentages for categorical variables. All descriptive statistics and plots were generated with Microsoft Excel.

An alphabetically arranged archive was created for the extracted measures, including their citations. Measures were classified and organized as clinician-reported outcome measures (CROMs), PROMs, or PREMs, or family-reported outcome or experience measures (FROMs and FREMs). CROMs involved an external observer, either a clinician or research staff member, who rated the interviews or scales.

Results

After we removed duplicates, 4,260 articles remained. Titles and abstracts were screened, and 611 articles were retained for full-text review (see PRISMA flow diagram in the online supplement). Of these, 91 studies met the inclusion criteria. A hand search of 26 reviews (examined for studies mentioning the use of measures; these studies were further evaluated for inclusion) yielded 24 additional articles. To avoid repeat counting of the measures used in a single study, we included only assessments and measures reported in study protocol papers and excluded any publications reporting results that were linked to the protocol study already included.

In total, 115 studies were selected for data extraction. (See the online supplement for a list of the studies, measures used, intended assessment domains, and time points during treatment when measures were administered [Characteristics of Studies Included]; an inventory of all measures, listed alphabetically by their acronyms [Alphabetized Inventory of Measures]; and a tabulation of measures by purpose, assessment domain [e.g., diagnoses, symptom assessment, and functioning], and reporting perspective assessed [Assessment Domains in Early Intervention Services].)

Study Locations

The reviewed studies were from North America (N=32), Western and Southern Europe (N=25), Scandinavia (N=24), the Asia-Pacific region (N=16), Oceania (N=14), Central and Eastern Europe (N=2), and intercontinental (Indo-Canadian) (N=2), representing 36 countries (Figure 1). Twenty percent (N=23) of the studies were published in 2019 or 2020 (see Number of Publications by Year in the online supplement). The measures favored in EIS reflected certain regional preferences. (Cited studies are identified henceforth by study number from Characteristics of Studies Included in the online supplement [e.g., S1, S2, etc.]) For example, the Role Functioning Scale (RFS) was used in nine studies, seven of which came from the Hong Kong Early Assessment Service for Young People With Psychosis (EASY) and Jockey Club Early Psychosis programs (references S55, S58, S59–62, S64), whereas the Service Utilization and Resources Form for Schizophrenia was used exclusively in North America (S25, S106, S109, S111, S114).

Study Objectives and Design

Sixty-seven studies (58%) assessed treatment outcomes in EIS. Ten (9%) were economic evaluations, and 10 (9%) undertook quality improvement or fidelity assessments. Twenty-eight studies (24%) involved program descriptions and protocols (see Characteristics of Studies Included in the online supplement). Excluding 11 studies (10%) without data collection (e.g., program descriptions and knowledge syntheses), study designs included descriptive and correlational designs (N=45, 39%), randomized controlled trials (N=20, 17%), quasi-experimental designs (N=17, 15%), cost-effectiveness studies (N=10, 9%), cross-sectional cohort designs (N=4, 3%), surveys (N=3, 3%), pre-post implementation interventions using chart audits or administrative data (N=2, 2%), consensus methodology (N=1, 1%), guideline development (N=1, 1%), and retrospective chart audit (N=1, 1%).

Measurement Timing and Purpose

Measures for seven studies were recorded at service entry or project baseline, and 79 studies using entry and baseline measures had at least one additional time point: year 1 (N=62), year 2 (N=32), and year 5 or beyond (N=17). Outcome assessments included patient symptom remission, recovery, housing, hospitalization, and functioning at 5 years in studies set in Canada (references S17, S34, S35), Denmark (S39, S44, S45), Norway (S80), Sweden (S89, S92), Italy (S70), and Japan (S75). Additionally, 7.5-year outcomes were assessed in one Australian study (S5) and 10-year outcomes in the Danish OPUS–1 studies (S38, S48), Danish-Norwegian Treatment and Intervention in Psychosis Study (TIPS) (S77), and Hong Kong EASY programs (S55, S57, S58). The longest follow-up durations were reported for a 13-year Swedish study on employment (S92) and a Hong Kong EASY program 12-year study on suicide (S56).

Administration and Psychometrics

Forty-five studies (39%) reported rater training on at least one measure, and 36 (31%) reported interrater reliability for specific measures. Raters were blind to the experimental condition in four studies, and two required rater consensus. Although many studies referenced well-established measures, only 15 provided reliability or validity information on any of the measures used (references S4, S9, S15, S16, S31, S36, S67, S72, S76, S85, S89, S94, S103, S113, S115). Even though the studies represented 36 countries, only 12 reported using translated measures, five of which identified validated translations (see Alphabetized Inventory of Measures in the online supplement).

Measurement Domains and Typology

This section describes the use of CROMs versus PROMs and PREMs or FROMs and FREMs for each domain and the number of studies using each measure (see Assessment Domains in Early Intervention Services in the online supplement). Overall, 60% (N=103 of 172) of all measures in all domains were CROMs, 30% (N=52) were PROMs/PREMs, and 10% (N=17) were FROMs/FREMs.

Program assessment.

Program assessments included three domains and involved 19 measures. Program fidelity was assessed by using seven measures in eight studies (references S16, S24, S25, S39, S42, S46, S48, S95), typically by using data from multiple sources, such as interviews, observation, chart reviews, and documents. Patient engagement was assessed by using a CROM, the Service Engagement Scale (SES), in two studies (S53, S95), and two studies used PREMs: the Working Alliance Inventory (WAI) (S26) and the Scale to Assess Therapeutic Relationships–Patient (STAR-P), Public and Patient Engagement Evaluation Tool (PPEET), and the Patient-Centered Outcomes Research Institute Engagement Activity Inventory (PECI) (S25). Satisfaction with services was assessed during follow-up at multiple time points by using five measures in 12 studies (S6, S25, S41, S45, S46, S53, S84, S86, S87, S99, S107, S110). The Client Satisfaction Questionnaire (CSQ), a PREM, and a modified family version were most frequently cited (S41, S45, S46, S53, S84, S86, S87, S107, S110).

Program fidelity was assessed by using CROMs exclusively, whereas studies used CROMs and PREMs to assess patient engagement with services and PREMs or FREMs to assess satisfaction. Remarkably, no studies, including program or service model evaluations, evaluated clinician attitudes, knowledge, skills, or related competencies.

Patient assessment.

Most measures (N=136 of 172, 79%) were used to assess patient presentation at treatment or study entry and outcomes during follow-up. Characteristics assessed included premorbid functioning, duration of untreated psychosis (DUP), diagnosis, symptomatology, comorbidity, risk, functioning, and quality of life, among others (see Assessment Domains in Early Intervention Services in the online supplement).

Thirty studies estimated DUP, with seven measures administered by clinicians or external observers; 23 of these studies used the interview-based (Early Recognition) Inventory for the Retrospective Assessment of the Onset of Schizophrenia (ERIRAOS and IRAOS), or Circumstances of Onset or Relapse Schedule (CORS) (references S19, S26–28, S30–32, S35, S37–39, S43, S45, S46, S50, S59–61, S64, S66, S67, S71, S73). DUP was usually assessed in studies reporting outcomes (N=21 of 30, 70%) (see Characteristics of Studies Included in the online supplement).

In 72 studies, diagnoses or diagnostic categories were established by using 15 measures, with the Structured Clinical Interview for DSM Disorders (SCID) used in 51 (71%) of these studies. The Schedule for Clinical Assessment in Neuropsychiatry (SCAN) was used in 10 studies (S39, S43–46, S49, S50, S99, S100, S102), and multiple other instruments were used in 11 studies (S15, S19, S37, S52, S69, S71, S72, S93, S94, S99, S112). Two studies used four patient-report measures (S3, S72), one study a family-reported measure (S52), and the others used CROMs, including interviews and checklists. Studies often established diagnoses at baseline and reassessed them after 1 year.

Psychosis symptomatology was the most frequently assessed domain (N=95 of 115 studies, 83%). Symptoms were assessed at single time points and at follow-up to determine change in clinical status by using 12 measures. Among studies assessing psychosis symptomatology, the Positive and Negative Symptom Scale (PANSS) was used most frequently (N=57 studies, 60%), followed by the Scale for the Assessment of Negative Symptoms (SANS, N=30, 32%), the Scale for the Assessment of Positive Symptoms (SAPS, N=25, 26%), and the Brief Psychiatric Rating Scale (BPRS, N=21, 22%). Symptom measures for psychosis were commonly used in combination (see Characteristics of Studies Included in the online supplement), and only three were PROMs: the Brief Symptom Inventory (BSI), measuring paranoia, positive symptoms, anxiety, and depression (S2, S10); the Frankfurt-Pamplona Subjective Experiences Scale (FPSES) on subjective experiences (S72); and the Community Assessment of Psychic Experiences–Positive 15 (CAPE-P15) for psychotic-like experiences (S113).

Comorbid symptomatology was also assessed at multiple time points during treatment. Depression was assessed in 36 studies with six measures, including three PROMs and three CROMs. The Calgary Depression Scale, a CROM, was most frequently used (N=25 studies, 69%), likely because of its specific focus on depression in schizophrenia (59) and its ability to distinguish depression from negative symptoms (60, 61). Five studies (S1, S8, S72, S101, S104) used the Beck Depression Inventory, a PROM. Anxiety and mania were each measured in five studies by using two PROMs (S112, S113) and one CROM (S7, S30, S32) for anxiety as well as one CROM for mania (S2, S51, S64, S88, S95). Substance use and related problems were assessed in 28 studies by using 17 measures—12 CROMs and five PROMs (see Assessment Domains in Early Intervention Services in the online supplement). Only seven of these studies used the following PROMs: Alcohol Use Disorder Identification Test (AUDIT) (S22, S33, S85, S89), Adolescent Drug Abuse Diagnosis Instrument (ADAD) (S6), Adolescent Alcohol and Drug Involvement Scale (AADIS) (S25), Drug Abuse Screening Test (DAST) (S22, S33), and Compulsive Internet Use Scale (CIUS) (S53). Suicidality and other risks (e.g., violence and dangerousness) were assessed in six studies using seven measures, two of which—the Adolescent Suicide Questionnaire–Revised (ASQ-R) (S6) and Inventory of Suicide Orientation 3D–Adolescent version (ISO-3D) (S2)—were PROMs.

“Functioning” was a broad term used by many researchers (see Characteristics of Studies Included in the online supplement) that included overall functioning (both symptomatology and general functioning), social and occupational functioning, and premorbid functioning. Sixty-eight studies used 14 measures of overall functioning, including eight CROMs, four PROMs, and two FROMs (see Assessment Domains in Early Intervention Services in the online supplement). The Global Assessment of Functioning (GAF) was used in 52 (76%) of these studies, whereas 22 reported using more than one measure of functioning, including PROMs such as the 12–Item Short-Form Health Survey (SF-12) for health status (S107, S110), the Mental Health Inventory (S3), the Warwick-Edinburgh Scale measuring well-being (S64), and the World Health Organization Disability Assessment Schedule (WHO-DAS) 2.0 (S25). FROMs used to assess patient functioning included the WHO-DAS 2.0 and Life Skills Profile (LSP-20) (S25).

Social and occupational functioning was assessed in 34 studies using 10 measures, including eight CROMs. One-fourth of studies under this domain utilized more than one measure. The Social and Occupational Functioning Assessment Scale (SOFAS) (S2, S3, S5, S12, S22, S33, S34, S52, S54, S55, S57–62, S64, S66, S93, S94) or SOFAS Personal and Social Performance Scale (SOFAS-PSP) (S7, S26) were used in 22 studies, the RFS in nine (S55, S58–62, S64, S107, S115), and the Social Functioning Scale (SFS) in two (S111, S114). Three studies used the SFS as a PROM (S10, S106, S112), and one (S95) asked patients to report on their use of time on a list of activities over the previous month. Nearly all studies assessing premorbid functioning (N=25 of 26) used the Premorbid Adjustment Scale, a CROM.

In 36 publications, quality of life was assessed with 12 measures, most commonly Heinrichs’ Quality of Life Scale, a CROM (S2, S5, S7, S14, S17, S18, S19, S25, S33, S98, S109, S114). Six instruments measuring quality of life were PROMs: the World Health Organization Quality of Life Brief or 26 (WHO-QOL-BREF/26) (S5, S7, S51, S72, S75, S85), Wisconsin Quality of Life (WQOL) (S26, S29), Subjective Satisfaction With Life Scale (SSWL) (S89), EuroQOL–5 Dimension (EQ-5D) (S53, S95), Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q-18) (S53), and SF-12, a general health measure (S62, S64). Quality of life was assessed at multiple time points during follow-up, frequently in conjunction with symptom assessments (N=31 of 36 studies). Recovery was assessed with four PROMs (S25, S64, S107, S110).

Additional domains assessed were service use and cost (S23, S25, S68, S75, S95, S101, S102, S106, S109, S111, S114); life events and trauma (S2, S107, S110); personality, self-esteem, coping, and expectations (S2, S51, S71, S72); insight (S36, S51, S88, S95, S99); needs and strengths (S101, S108, S112), and family factors (S2). Of 21 measures used to assess these domains, 12 were CROMs (one was both a CROM and PROM) (see Assessment Domains in Early Intervention Services in the online supplement). Ten studies (S2, S3, S5, S9, S12, S32, S34, S35, S39, S97) used multidimensional measures, such as illness and symptom history, by using three CROMs and one PROM.

In summary, patient presentations, symptomatology, functioning, and recovery were overwhelmingly measured with CROMs (N=88 of 136, 65%). Very few measures were reported by patients (N=45, 33%), and few studies used PROMs to measure psychotic and comorbid symptoms (S1, S2, S6, S8, S10, S25, S72, S101, S104, S112, S113) (see Assessment Domains in Early Intervention Services in the online supplement). Recovery and quality of life were exceptions, because an equal or higher number of PROMs, compared with CROMs, were used to evaluate these domains.

Family and caregiver assessment.

Family factors were assessed in 19 publications with 17 measures, including 59% (N=10) FROMs, which is a considerably higher percentage compared with that for PROMs used in evaluating patient domains. Caregiving, coping, and burden were assessed in eight studies by using five measures, four of them FROMs, including four studies that used the Experience of Caregiving Inventory (ECI) (references S6, S12, S18, S101). Expressed emotion and family functioning (S41, S73) and family background (e.g., genetic and obstetric histories) (S2, S52) were assessed with CROMs. Other family factors (e.g., depression, needs, quality of life, health, well-being, and functioning) were assessed in seven studies (S3, S10, S18, S25, S41, S45, S101), with six of eight measures used being FROMs.

Discussion

This comprehensive review evaluated measures used in EIS for psychosis to monitor multiple aspects of patient and family presentations and recovery over the course of treatment and examined the effectiveness of early intervention treatment models. As evinced by our review, research on EIS for psychosis has evolved considerably over the past 20 years. Well-designed studies, including several longitudinal investigations, have been informed by, and have iteratively informed, service innovations. Most of the reviewed studies were conducted in Australia, Canada, Denmark, Hong Kong, Norway, Singapore, or the United Kingdom, countries that pioneered EIS for psychosis. By identifying, describing, and evaluating measures used worldwide in EIS for psychosis, we have created a valuable, synthesized archive of measures that can support future research into measurement-based care and services.

EISs for psychosis are avowedly concerned with continually improving service quality and outcomes. Measurement science has long recognized the value of multi-informant, multilevel assessments (62) to enhance comprehensiveness and accuracy. Measurement-based care, an important component of evidence-informed care, may further this goal by helping programs and services or governments and regulatory bodies closely monitor program performance compared with clinical standards and guidelines. Systematically administering measures throughout treatment may also help to assess progress and generate data that will inform treatment decisions and personalize care for patients and families. In the assessment of psychosis, this process should translate into the routine use of patient-, family-, and clinician-reported measures of multiple domains.

CROM Dominance and Neglect of PROMs/PREMs and FROMs/FREMs

Use of PROMs is promoted as a way of empowering patients to self-manage care and voice their opinions on service providers’ decisions (41, 63, 64). PROMs, PREMs, FROMs, and FREMs show promise for improving outcomes and are favored by patients and their families (41, 64). Use of PROMs also aligns with early intervention guidelines in emphasizing patient and family engagement (65–68), yet this review revealed that PROMs/FROMs and PREMs/FREMs are used far less frequently than CROMs. Whereas 60% of measures were CROMs in the reviewed studies, only 30% were PROMs and 10% FROMs.

The predominant use of CROMs may stem from the strong focus on assessing symptomatology in psychosis research and may reflect the priorities set by clinicians and researchers, who were the original leaders in creating these assessments. Assessments by clinicians and external observers may also have been prioritized for psychosis, a condition often characterized as involving loss of contact with reality, poor insight, and metacognitive difficulties (69). Limited use of PROMs may also reflect attitudes toward psychosis that underestimate or undervalue patients’ capacity to provide useful information, which may be rooted in a deficits-based model. Even social and occupational functioning was assessed primarily with CROMs, which may have precluded a patient-centered examination of what are now considered to be key dimensions, such as job satisfaction, perceived performance, and aspirations.

Although CROMs were mainly used to assess patient characteristics and outcomes, relatively more FROMs than CROMs were employed in assessments of family factors. However, families were still rarely involved in rating patient functioning and symptoms, even though many young persons with FEP live with or are in close contact with their families (70). Families remain an untapped resource for patient assessment, particularly in relation to the question of patient functioning outside the clinic (71).

The results of our synthesis thus suggest that measurement-based care in EIS stands to benefit from greater inclusion of PROMs and FROMs, which can complement clinician-reported measures to provide a more comprehensive understanding of treatment outcomes and effectiveness.

Carlier et al. (72) identified two complementary theories describing the operation of PROMs: the feedback intervention theory, which suggests that feedback on interventions and progress through PROMs informs providers’ treatment decisions, and the therapeutic assessment theory, which considers the positive therapeutic effects of PROMs in involving patients in reviewing their own data and scores. Use of PROMs may also engender virtuous cycles of improvement in patient-provider communication; shared understanding of goals; enhanced clinician understanding of patient views and personal circumstances; and improved patient adherence, outcomes, and satisfaction with services. Patient and family reports on experience with services may also support quality improvement efforts.

An important direction of future research would be to evaluate the impact of the routine use of PROMs on patient outcomes in early psychosis. PROMs used in schizophrenia, psychosis, and associated conceptual and methodological issues were the subject of literature reviews (40, 44) that cited evidence for the use of short, psychometrically sound, and clinically relevant PROMs. In the broader mental health field, evidence on the benefits of patient-reported measures is mixed but still emerging. A Cochrane review (67) found insufficient evidence for the benefits of routine outcome monitoring with PROMs in treating common mental disorders. This finding was uncertain, however, given a high risk of bias in the included studies. Two more recent studies of PROMs in psychotherapy reported small-to-moderate effects on outcomes compared with treatment as usual, with one study reporting twice the levels of clinical improvement among clients who were predicted to have poor outcomes (73) and the other identifying positive effects in community settings but not in psychiatric settings (74).

Overall, the increased use of PROMs and FROMs would align with the EIS philosophy of emphasizing patient and family engagement and principles of patient-oriented research (75). Ideally, this alignment should be situated in the closer involvement of patients and families in treatment decision making and in services and research design and implementation. Patient and family involvement in service design and treatment delivery can identify new domains to assess and allow for the creation of new PROM and FROM measures.

Greater Focus on Coproduction and Implementation

Studies in this review focused mainly on the assessment of patient characteristics and outcomes, and patient data were frequently pooled in evaluations of service models. Pooled patient-level DUP data, for instance, were often used as an indicator of treatment delay and a proxy for program accessibility. Given increased concern with fidelity to standards and implementation gaps in EIS, program-level implementation processes may need more direct and consistent assessment. Such assessment may include, but also extend beyond, fidelity measures to account for the fit between the EIS model (or model components) and local contexts and for whether the EIS model can be adapted without compromising its inherent logic and core evidence-based features (69). Furthermore, higher adoption of PROMs and FROMs should emerge from more extensive involvement of patients and families, which could lead to wider uptake of evidence-based practices as well as innovation within the EIS model.

Remarkably, only 12 of the 115 studies assessed patient and family satisfaction, and no study used a satisfaction measure tailored to EIS, thereby underlining an important gap and priority for measures development research in early psychosis. Measures are also needed to assess patient and family perceptions of key features of the EIS model, such as promoting hope and optimism as well as sensitivity to developmental and illness phases. Many assessments from our review, including the BPRS and PANSS for symptom assessment, the Lehman Quality of Life Inventory (LQOLI) and Heinrichs’ Quality of Life Scale for quality of life, and the Strauss-Carpenter Scale for assessment of functioning, have long histories in schizophrenia research. Well-established measures are a strength but may also be problematic in that they were not specifically designed for younger individuals in the early phases of the illness. Some conventional measures may also use prescriptive framing and language that do not reflect the subjective experiences of patients and service users nor represent the empowering, hopeful philosophy of early intervention.

Along with the foci of this review (purpose, domain, and perspective), many additional aspects are important to consider in any efforts to implement measurement-based care more widely and meaningfully. Critical among these aspects is the level of stakeholder engagement (e.g., patients, family members, or caregivers) in the selection and implementation of measures. Even more fundamentally, their participation is vital in arriving at the constructs (i.e., domains) that are the focus of research and measurement-based care and in creating measures that assess such constructs. In our review, only one included study (reference S25) provided details about the engagement of persons with lived experience in selecting or implementing measures.

Future studies should also systematically identify the domains in need of development of measures (particularly PROMs and FROMs). Our study reported on the number of measures per domain (further categorizing them as patient or family reported vs. clinician reported), which may be a crude indicator of gaps in the field. For some domains, it may also be reasonable that measures be assessed only from certain reporting perspectives (e.g., satisfaction cannot be assessed from the clinician’s perspective). Notwithstanding these nuances, the predominance of clinician-reported measures indicates the development or use of PROMs as a worthwhile direction of future research.

Future efforts should also evaluate the psychometric and other properties of measures, such as reliability (test-retest and internal consistency), validity (content validity, face validity, and construct validity), responsiveness (sensitivity to change), costs, length, cross-cultural validity, and availability in diverse languages, by using, for example, the Consensus-Based Standards for the Selection of Health Measurement Instruments checklist (76).

To make such information easily accessible to researchers, clinicians, patients, family members, policy makers, and others, our team developed an online database (MAP-PRO [an acronym that reflects a meaningful assessment protocol, patient-reported outcomes, and tools for professional use]), which presents each of the measures identified in this review, providing information such as reliability, validity, cost, length, language availability, and training and categorizing each measure as a CROM, PROM, or FROM. The filter options available on the Web platform (https://www.mcgill.ca/mappro/information-hub/measures-library) allow users to quickly and effectively identify the information they are most interested in accessing. Finally, the promise of measurement-based care can be realized only if those in the field also acknowledge and navigate challenges such as those related to the implementation and performance of research-based measures in community settings and associated time and cost burdens.

From Measurement-Based Care to Data Harmonization

This review of measures can help researchers identify the measures most commonly used in EIS to assess symptoms, functioning, and other domains—an important step toward the harmonization of measures in EIS at both local and global levels—which may open opportunities for future research studies. Such harmonization would also facilitate big-data platforms, data science, and rapid learning health care systems (77), thereby improving outcomes and service quality (78–80).

Pioneering work on data harmonization for outcome measures in EIS has been led by two important initiatives: the International Consortium for Health Outcomes Measurement (ICHOM) (in the area of psychotic disorders [psychosis, schizophrenia, and bipolar disorder]) and the Core Assessment Battery (CAB) for the U.S. Early Psychosis Intervention Network (EPINET). The ICHOM used multiple methods (focus groups, systematic literature reviews, and the Delphi consensus process) and approaches involving international stakeholders (professionals, service users, and caregivers) to identify four outcome domains (symptoms, recovery, functioning, and treatment) and 14 core patient-reported outcomes (81). The EPINET National Data Coordinating Center is a research initiative focused on determining best treatments in EIS. EPINET developed a CAB (EPINET CAB) on the basis of standardized measures and individual items that assess key domains of EIS, such as psychosis psychopathology, recovery, contextual factors, and treatment. The CAB was developed through a consensus process by the EPINET steering committee and is available on the EPINET portal (81–83).

We note that research from low- and middle-income countries, where most individuals with FEP live, was underrepresented in our synthesis. This underrepresentation reflects the limited reach of EIS in these contexts and may be partly due to the exclusion of non-English publications. Future efforts to identify suitable measures and their purposes and effectiveness will need a more global scope. Such measures should be developed and selected with respect to a given epistemic context and by adhering to principles of cultural sensitivity (84).

Strengths and Limitations

To our knowledge, this is the first comprehensive review and synthesis of measures used in EIS for psychosis. Some eligible studies may have been overlooked because the search was limited to published English-language studies. We largely excluded studies that used critical appraisal tools for quality assessment, given that we wished to prioritize identification of measures used in EIS. Restriction of the search to published, peer-reviewed literature may have introduced a bias toward studies involving CROMs, which may have been viewed as more amenable to publication.

A second limitation of this review stemmed from heterogeneity in the ways that data are collected for CROMs and who is involved in rating CROMs. Some of the CROMs identified in this review may have been implemented with an interview format, in which patients reported their satisfaction or perceived mental state and symptoms in response to questions. Most of the included studies that reported on CROMs did not provide specific details on whether ratings were entirely based on responses directly provided by patients, informed by responses provided by patients but interpreted or judged by clinicians, or based on other information (e.g., information given by patient’s primary clinician). Furthermore, some CROMs are designed to be completed by interviewers with specific training (e.g., SCID), and others require raters to have no specific training (e.g., SES). The articles included in this review, however, did not always provide information on the level of training received by raters.

Interestingly, we found some literature indicating that CROMs (rather than PROMs) more accurately reflected symptoms (85, 86), whereas other evidence suggested that patients’ reports of symptoms are more accurate. The accuracy of patient-reported symptoms was particularly true in research with adolescents (87), and the authors speculated that PROMs gave adolescents the feeling that their “concerns are being heard directly,” offering them a sense of agency (88) and autonomy (89).

Similarly, patient and family measures could have been used in EISs for psychosis to drive treatment decisions, but they were not reported in publications that focused on EIS outcomes. It is also possible that EISs regularly use such important measures, a practice that strongly aligns with EIS philosophy, but that these programs do not have proper resources to engage in research activities or publish their assessment protocols or outcome data on services. We also recognize that some of the studies in this review may have used the same protocol and data, which may have resulted in an overestimation of the use of some measures. We recognize that this review did not integrate the perspectives of persons with lived experience in its planning, data synthesis, or writing. Finally, this review may have missed some measures by excluding studies focused on specific interventions (e.g., CBT) or broad therapeutic approaches and orientations (e.g., recovery). Although this exclusion was a limitation, it also reflects the reality that such assessments do not have mainstream acceptance.

Conclusions

This review evaluated the use of measures in EIS for psychosis in terms of meaningful dimensions—particularly representing domains of interest and stakeholder perspectives—revealing important practice and research gaps. Many influential policy statements recommend routine monitoring of clinical care by using PROMs (90, 91); more widespread use of PROMs and FROMs is critically needed to complement traditional clinician-reported outcomes and would have important implications for ensuring that EISs for psychosis meet the needs and realities of patients and their families. Widespread use of PROMs and FROMs would also be consistent with core principles of early intervention and would support patient and family engagement and the ideals of patient-oriented research and measurement-based care.