""

General Psychiatry Residency Program QI

""

About us

The General Psychiatry Residency Program at the University of Toronto is the largest in North America with 200 residents distributed across our full and associate affiliated hospitals in the Toronto Academic Health Sciences Network (TAHSN). The residency program is supported by the over 900 faculty members within the Department of Psychiatry.

""

Our Mission

To provide outstanding psychiatry residency training in a culturally safe, diverse, and inclusive environment that inspires lifelong learning and professional identity formation.

Our graduates will meet the needs of our communities and lead scholarship and innovation in mental health care.

Our values

  • Excellence
  • Accountability
  • Justice
""

Program Evaluation and Quality Improvement Strategy

The General Psychiatry Residency Program is committed to continuous improvement of the educational experiences, to improve the residency program and ensure residents are prepared for independent practice.

The Program Evaluation Subcommittee oversees continuous improvement processes in the residency program through systemically and regularly reviewing a range of data to improve the residency program. Based on data reviewed:

  1. Strengths and areas for improvement are identified.
  2. Action plans are established to address areas for improvement.
  3. Interventions are iteratively evaluated to determine the effectiveness of actions taken and to determine whether further action is required.

The Program Evaluation Subcommittee reports to the Psychiatry Residency Program Committee

Data Sources

A surface covered in papers featuring different type of charts and graphs, all in shades of blue. A white calculator and a silver pen sit on top of the papers.
  1. Residency Program and Site Surveys conducted every 6-months.
  2. Postgraduate Medical Education data: Rotation Evaluation Scores (RES) and regular review of “flagged” Learner Assessment of Clinical Teacher (LACT) forms (see Teacher Feedback and Assessment below).
  3. Feedback from other subcommittees and working groups.

The Program Evaluation Subcommittee is also informed by the residency program’s other subcommittees and working groups, in particular the Learning Environment Working Group.

Program QI Highlights 2023-2024

""

Call Experience

Creation of Call Working Group in 2022-23 to review residency program documents relating to on-call experiences and inform residency program support of residents in 2023-24 with the double cohort sitting for the Royal College Exam.

Accomplishments

  • Conversion of Call Working Group to a Subcommittee given learner experience feedback provided via Residency Program and Site Surveys and Resident Leadership Council.

Workplan 2023-24

  • Review of 6-month call schedules.
  • Assess supports at hospitals sites for Chief Residents to create and revise call schedules.
  • Review of impact call accommodations on call pools (with the Subcommittee on Resident Wellbeing and Assessment Subcommittee).
""

Curriculum

Review and revamp of rotation structure and curriculum in response to November 2020 RCPSC External Review. See PG Newsletter Volume 4 (Fall 2022) - Accreditation for details.

Past Editions of PG Newsletter also provide regular Accreditation and Program QI updates.

Rotation Evaluation Scores (RES)*

Rotation Evaluation Scores (RES)*

   
Rotation Overall RES              Number of Evaluations     
PGY1 Addictions 4.04 33
PGY1 CL Psychiatry 3.93 34
PGY1 Family Medicine/Hospitalist            3.93 31
PGY1 Psych ER 4.04 72
PGY1 PLEX Clinical 4.26 6
PGY1 PLEX Research 4.73 9
PGY1 U/MPS 4.08 18
PGY1 Outpatient 4.19 72
PGY2 Inpatient 4.43 73
PGY2 Outpatient 4.06 6
PGY2 Longitudinal SPMI 4.05 52
PGY3 Child N/A N/A
PGY3 Geriatric 3.98 38
PGY3 SMI 4.24 41
PGY3 U/MPS 4.05 29
PGY4 Addictions 4.12 34
PGY4 CL 4.11 44
PGY5 IMHC 4.52 55

 *Data for 2022-23 as of October 11, 2023.

Workplan 2023-24

  • Review of RES and narrative feedback at Curriculum and Program Evaluation Subcommittees, with a report back to PRPC for higher-level review and recommendations for any rotations and sites with RES < 4.
  • Triangulation of RES data with residency program and site surveys.

Competence by Design (CBD)

""

EPA completion report

Every 6-months, EPA completion reports by hospital and faculty member are shared with hospital sites. Psychiatrist-in-Chiefs/Heads (PIC/Hs) are tasked with working with faculty members on improving EPA completion rates as needed, with support of residency program leadership.

General Psychiatry Program Faculty EPA Completion Report Department Overview

Legend:

 

Assessors at site has completed <50% of initiated EPAs

 

Assessors at site has completed 51-79% or more of initiated EPAs

 

Assessors at site has completed 80% or more of initiated EPAs

 

Site

# of Assessors

# Completed

# Incomplete

Completed + Incomplete

Completion Rate (%)

Entrustable EPAs

% Entrustable

Baycrest

6

74

7

81

91%

64

86%

CAMH

152

1145

333

1478

77%

1098

96%

MGH

1

7

0

7

100%

7

100%

NYGH

8

18

5

23

78%

15

83%

OSCMHS

10

23

1

24

96%

23

100%

Other

2

6

2

8

75%

6

100%

SCCMH

1

2

0

2

100%

2

100%

Sinai Health - BPH, MSH

32

343

80

423

81%

309

90%

SHSC

43

467

72

539

87%

427

91%

Surrey Place

1

3

3

6

50%

3

100%

Trillium Health Partners

3

5

0

5

100%

5

100%

Unity Health - SJHC

19

46

17

63

73%

45

98%

Unity Health - SMH

47

463

110

573

80%

442

95%

UHN - PMH, TGH, TRI, TWH

50

372

99

471

79%

354

95%

UofT Health & Wellness

3

6

5

11

55%

6

100%

WCH

7

52

7

59

88%

40

77%

TOTAL

385

3032

748

3780

80%

2842

94%

Notes

1.       Includes EPAs completed by psychiatry faculty, allied health and clinical fellows.

2.       Includes EPAs initiated by residents and faculty.

3.       Data for CAMH includes EPAs completed during PGY1 Family Medicine/Hospitalist rotation.

4.       Data for CAMH, SHSC, and SMH includes EPAs completed by Family Medicine physicians during PGY1 & PGY4 Addictions rotations.

5.       Does not include EPAs completed by other residents including subspecialty residents.

6.      Does not include data from PGY1 off-service (medicine) rotations.

7.       If a site is not listed = no data during this time period (i.e. GHCCF, HRRH, YTC)

 

Previous Overall Completion Rates for EPAs

Period

# Completed

# Incomplete

Completed + Incomplete

Completion Rate

Entrustable EPAs

% Entrustable

July 1, 2021 – January 31, 2022

4549

1352

5901

77%

N/A

N/A

February 1, 2022 – June 30, 2022

2824

858

3682

77%

N/A

N/A

July 1, 2022 – December 31, 2022

3789

871

4660

81%

3489

92%

January 1, 2023 – June 30, 2023

3032

741

3773

80%

2846

94%

 

""

CBD Coaching

As a part of Competence by Design (CBD), all residents are assigned a non-evaluative CBD Coach who supports a resident from Transition to Discipline to Transition to Practice. In June 2023, with the first full cohort of residents from PGY1-5 completing CBD as part of the program’s CBD pilot, an evaluation of the CBD coaching model was conducted. Data continues to be analysed.

Initial analysis has revealed the following:

  1. Benefits from the faculty perspective include establishing longitudinal relationships with residents to support their growth and development and enhanced knowledge of the residency program’s assessment strategy.
  2. From the resident perspective, a desire for greater clarity regarding the role of CBD Coaches in their development (see link above for CBD coach description along with creation of Coaching Checklist developed in 2021-22).

Results from further analysis will be shared with the Program Evaluation Subcommittee, PRPC, and updated on our Program QI website as the data becomes available.

Communication and Program Navigation

Regular updates

2022-23 Department of Psychiatry Analysis

Our Current Residents and Postgraduate Education webpages are frequently used by residents and are among our Top 10 most-visited webpages.

Learning Environment and Hidden Curriculum

This working group launched in 2023.

Learning Environment Working Group published their recommendations in Fall 2023.

Residency Program and Site Surveys

Hospital Site Responses are available to residents on Quercus

Flagged Learner Assessment of Clinical Teacher (LACT)* Analysis 2022-23

*any item on assessment with a score lower than 3/5

LACT Forms – Questions

Amount of contact (Brief, Moderate, Extensive) 

5 questions  

  1. Effective clinical teaching 
  1. Responsive relationships with effective feedback 
  1. Positive role model 
  1. Learning climate  
  1. Overall rating out of 5 (Unsatisfactory/Weak/Acceptable/Good/Superior)  
Severity of Learner Experience Concern Examples Number Action

Mild

Supervisor difficult to reach at times but overall available.

Paucity of clinical teaching.

Feedback that was experienced as incongruent with learner’s perceived performance.

5-10

Expectation Setting by Chief/Head

  • Awareness building conversations with further close monitoring
  • Review of supervisor guidelines
  • Discussion with a mentor and/or education leader at the site

Moderate

Repeated flagged LACTs of with mild/moderate learner experience concerns with previous expectation setting by Chief/Head.

Inadequate supervision or times when learner unable to reach supervisor for more urgent patient care issues.  Little to no clinical teaching from supervisor.

Feedback that was experienced as harsh or publicly shaming. Reported microaggressions, racist or sexist comments or behaviour.

5-10

Education Interventions

  • Coaching on education and/or professionalism issues
  • Remediation plan
  • Mandated Centre for Faculty Development courses
  • Assigned readings

Severe

Persistent learner experience concerns (e.g. lack of supervision, feedback experienced as undermining safety of learning environment) that have not improved with expectation setting or education interventions.

Serious boundary crossings (e.g. alleged sexual harassment or assault)

0

Disciplinary Interventions

  • Temporary or permanent loss of learners
  • Temporary or permanent loss of a supervisory/teaching/leadership role
  • Surrender or revocation of hospital privileges or university appointment

 

Key Themes (from narrative comments)

  1. Supervisor availability and service to education ratio rotation. 
  2. Feedback – absence of, experienced as incongruous with performance or insensitive.
  3. Flexibility in approach to teaching, supervision, and clinical care – e.g. supervisor experienced as rigid or harsh.

Voice of the Resident

To ensure robust integration of resident perspectives in key residency program initiatives, there is resident representation across residency program committees, subcommittees, and working groups. Resident leaders have also been identified by residency program surveys as key sources of support and guidance. As such, residency program leadership have worked with resident leaders in the following quality improvement initiatives to enhance resident leader development:

  1. Fortifying the role of the Psychiatry Residents’ Association of Toronto (PRAT) via Terms of Reference.
  2. Establishment of the Resident Leadership Council (RLC).