Psychiatrists, like other medical professionals, are confronted by a need to maintain specialty specific knowledge despite an explosion in the amount of new information and the ongoing demands of clinical practice. Given these challenges, it is not surprising that researchers have consistently found gaps between actual care and recommended best-practices (1—10). In attempting to enhance the quality of delivered care, a number of approaches have been tried with varying degrees of success. Didactic approaches, including dissemination of written educational materials or practice guidelines, produce limited behavioral change (11—19). Embedding of patient-specific reminders into routine care can lead to benefits in specific quality measures (11, 13—16, 20—23) but these improvements may be narrow in scope, limited to the period of intervention or unassociated with improved patient outcomes (24—27). Receiving feedback after self or peer-review of practice patterns may also produce some enhancements in care (13—15, 23, 28—30). Given the limited effects of the above approaches when implemented alone, the diverse practice styles of physicians and the multiplicity of contexts in which care is delivered, a combination of quality improvement approaches may be needed to improve patient outcomes (14, 19, 28, 29, 31—34).
With these factors in mind, the American Board of Medical Specialties and the American Board of Psychiatry and Neurology are implementing multi-faceted Maintenance of Certification (MOC) Programs that include requirements for self-assessments of practice through reviewing the care of at least 5 patients (35). As with the original impetus to create specialty board certification, the MOC programs are intended to enhance quality of patient care in addition to assessing and verifying the competence of medical practitioners over time (36, 37). Although the MOC phase-in schedule will not require completion of a Performance in Practice (PIP) unit until 2014 (35), individuals may wish to begin assessing their own practice patterns before that time. To facilitate such self-assessment related to the treatment of depression, this paper will discuss several approaches to reviewing one's clinical practice and will provide sample PIP tools that are based on recommendations of the American Psychiatric Association's Practice Guideline for the Treatment of Patients with Major Depressive Disorder (38).
Traditionally, most quality improvement programs have focused on retrospective assessments of practice at the level of organizations or departments (39). The Healthcare Effectiveness Data and Information Set (HEDIS) measures of the National Committee for Quality Assurance (NCQA) (40) are a commonly used group of quality indicators that measure health organization performance. When used under such circumstances, quality indicators are typically expressed as a percentage that reflects the extent of adherence to a particular indicator. For example, in the quality of care measures for bipolar disorder (41) derived from the American Psychiatric Association's 2002 Practice Guideline for the Treatment of Patients with Bipolar Disorder (42), one of the indicators is that "Patients in an acute depressive episode of bipolar disorder who are treated with antidepressants, [are] also receiving an antimanic agent such as valproate or lithium." In this example, to calculate the percentage of patients for whom the indicator is fulfilled, the numerator will be the "Number of patients in an acute depressive episode of bipolar disorder, who are receiving an antidepressant, and who are also receiving an anti-manic agent such as valproate or lithium." and the denominator will be the "Number of patients in an acute depressive episode of bipolar disorder who are receiving an antidepressant" (41).
As in the above example, most quality indicators are derived from evidence-based practice guidelines, which are intended to apply to typical patients in a population rather than being universally applicable to all patients with a particular disorder (43, 44). In addition, practice guideline recommendations are mainly informed by data from randomized controlled trials. Patients in such trials may have significant differences from those seen in routine clinical practice (45), including clinical presentation, preference for treatment, response to treatment, and presence of co-occurring psychiatric and general medical conditions (43, 46, 47). These differences may result in treatment decisions for individual patients that are clinically appropriate but not concordant with practice guideline recommendations.
When quality indicators are used to compare individual physicians' practice patterns, quality measures can be influenced by practice size, patients' sociodemographic factors and illness severity as well as other practice-level and patient-level factors. For example, when small groups of patients are receiving care from an individual physician, a small shift in the number of individuals receiving a recommended intervention could lead to large shifts in the resulting rates of concordance with evidence-based care. Without appropriate application of case-mix adjustments, across-practice comparisons may result in erroneous conclusions about the quality of care being delivered (48, 49). For patients with complex conditions or multiple disorders receiving simultaneous treatment, composite measures of overall treatment quality may yield more accurate appraisals than measurement of single quality indicators (50—52).
With the above caveats, however, use of retrospective quality indicators can be beneficial for individual physicians who wish to assess their own patterns of practice. If a physician's self-assessment identified aspects of care that frequently differed from key quality indicators, further examination of practice patterns would be helpful. Through self-assessment, the physician may determine that deviations from the quality indicators are justified, or he may acquire new knowledge and modify practice to improve quality. It is this sort of self-assessment and performance improvement efforts that the MOC PIP program is designed to foster.
app1 and app2 provide sample PIP tools, each of which is designed to be relevant across clinical settings (e.g., inpatient, outpatient), straightforward to complete and usable in a pen-and-paper format to aid adoption. Although the MOC program requires review of at least 5 patients as part of each PIP unit, it is important to note that larger samples will provide more accurate estimates of quality within a practice. app1 provides a sample retrospective chart review PIP tool that assesses the care given to patients with major depressive disorder. Although it is designed as a self-assessment tool, this form could also be used for retrospective peer-review initiatives. As with other retrospective chart review tools, some questions on the form relate to the initial assessment and treatment of the patient whereas other questions relate to subsequent care. app2 provides a prospective review form that is intended to be a cross-sectional assessment and could be completed immediately following a patient visit. As currently formatted, app2 is designed to be folded in half to allow real-time feedback based upon answers to the initial practice-based questions. This approach is more typical of clinical decision support systems that provide real-time feedback on the concordance between guideline recommendations and the individual patient's care. In the future, the same data recording and feedback steps could be implemented via a web-based or electronic record system enhancing integration into clinical workflow (53). This will make it more likely that psychiatrists will see the feedback as interactive, targeted to their needs and clinically relevant. Rather than relying on more global changes in practice patterns to enhance individual patients' care, such feedback also provides the opportunity to adjust the treatment plan of an individual patient to improve patient-specific outcomes (54—56). However, data from this form could also be used in aggregate to plan and implement broader quality improvement initiatives. For example, if self-assessment using the sample tools suggests that signs and symptoms of depression are inconsistently assessed, consistent use of more formal rating scales such as the PHQ-9 (57—59) could be considered.
Each of the sample tools attempts to highlight aspects of care that have significant public health implications (e.g., suicide, obesity, use of tobacco and other substances) or for which gaps in guideline adherence are common. Examples include underdetection and undertreatment of co-occuring substance use disorders (5) and the relatively low concordance with practice guideline recommendations for use of psychosocial therapies and for treatment of psychotic features with MDD (4). Table 1 summarizes specific aspects of care that are measured by these sample PIP tools. Quality improvement suggestions that arise from completion of these sample tools are intended to be within the control of individual psychiatrists rather than dependent upon other health care system resources.
After using one of the sample PIP tools to assess the pattern of care given to a group of 5 or more patients with major depressive disorder, the psychiatrist should determine whether specific aspects of care need to be improved. For example, if the presence or absence of co-occurring psychiatric disorders has not been assessed or if these disorders are present but not addressed in the treatment plan, then a possible area for improvement would involve greater consideration of co-occurring psychiatric disorders, which are common in patients with MDD.
These sample PIP tools can also serve as a foundation for more elaborate approaches to improving psychiatric practice as part of the MOC program. If systems are developed so that practice-related data can be entered electronically (either as part of an electronic health record or as an independent web-based application), algorithms can suggest areas for possible improvement using specific, measurable, achievable, relevant and time-limited objectives (60). Such electronic systems could also provide links to journal or textbook materials, clinical practice guidelines, patient educational materials, drug-drug interaction checking, evidence based tool kits or other clinical materials. In addition, future work will focus on developing more standardized approaches to integrating patient and peer feedback with personal performance review, developing and implementing programs of performance improvements and reassessment of performance and patient outcomes.
Sample Retrospective Chart Review Performance-in-Practice Tool for the Care of Patients with Major Depressive Disorder
Instructions: Choose 5 patients with a primary diagnosis of major depressive disorder. If the answer to a given question is "Yes", place a check mark in the appropriate box. If the answer to the question is "No" or "Unknown", leave the box unchecked. After reviewing the charts of all 5 patients, complete the final column to determine the relative proportion of patients to whom the recommendation was followed. Any rows for which the total is <2 may be a useful focus for quality improvement efforts.
Sample "Real-Time" Performance-in-Practice Tool for Patients with Depression
This "real time" PIP tool is intended to be a prospective cross-sectional assessment that could be completed immediately following a patient visit. As currently formatted, the tool is designed to be folded in half to allow real-time feedback based upon answers to initial practice based questions. Up to 5 hours additional CME credit can be earned through use of the PIP tool and completion of the survey.
Sample "Real-Time" Performance in Practice Tool for Patients with Depression Survey Form and CME Certification Begin date February 2008, End date February 2010.
To earn CME credit for this Survey Program, psychiatrists should use the Sample Real Time Performance in Practice Tool as indicated. After using the performance in practice tool, participants should fully complete this survey and send it by mail to APACME 1000 Wilson Boulevard, Suite 1825 Rosslyn VA 22209, or fax to 703 907 7849, or send by email to email@example.com.
Objective: After completion of this activity psychiatrists will have the foundation for subsequent performance improvement initiatives aimed at enhancing outcomes for patients with major depressive disorder.
The APA is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians. APA designates this educational activity for a maximum of 5 AMA PRA Category 1 credits. Physicians should only claim credit commensurate with the extent of their participation in the activity.
List the most helpful aspects of this PIP tool:
List the least helpful aspects of this PIP tool:
How do you plan to use the information gained from this self-assessment in your practice?
How might we improve upon this PIP tool in the future?
Please evaluate the effectiveness of this CME activity by answering the following questions.
1. Achievement of educational objectives: YES____ NO____
2. Material was presented without bias: YES____ NO____
American Psychiatric Association CME 1000 Wilson Blvd., Suite 1825Arlington, VA 22209-3901Telephone: (703) 907-8637, Fax: (703) 907-7849 To earn credit, complete and send this page. Retain a copy of this form for your records.
Number of hours you spent on this activity ______________ (understanding & using the tool and completing the survey up to 5 hours)
APA Member: Yes_____ No_____
Last name First name Middle initial Degree
City State Zip code Country
I would like to receive my certificate by: