Download the full-sized PDF of Using the Ottawa Surgical Competency Operative Room Evaluation (O-SCORE) in a Canadian Plastic Surgery ProgramDownload the full-sized PDF



Permanent link (DOI):


Export to: EndNote  |  Zotero  |  Mendeley


This file is in the following communities:

Graduate Studies and Research, Faculty of


This file is in the following collections:

Theses and Dissertations

Using the Ottawa Surgical Competency Operative Room Evaluation (O-SCORE) in a Canadian Plastic Surgery Program Open Access


Other title
Surgical education
Generalizability theory
Type of item
Degree grantor
University of Alberta
Author or creator
Budden, Curtis R
Supervisor and department
Gierl, Mark (Educational Psychology)
White, Jonathan (Medicine and Dentistry)
Examining committee member and department
Gierl, Mark (Educational Psychology)
White, Jonathan (Medicine and Dentistry)
Zhu, Jay (Medicine and Dentistry)
Department of Educational Psychology
Measurement, Evaluation and Cognition
Date accepted
Graduation date
2016-06:Fall 2016
Master of Education
Degree level
Competency-based medical education has overtaken the traditional apprenticeship model of medical education. This shift toward competence-based training demands a higher number of assessments per trainee. Current assessment methods in Plastic Surgery have not been well studied. Numerous assessment tools in surgery exist but do not apply to the specialty of Plastic Surgery because of the nature of procedures being assessed. One tool which combines a focus on competency as well as possible applicability to Plastic Surgery is the Ottawa Surgical Competency Operative Room Evaluation (O-SCORE). This tool uses a unique entrustability scale which is construct aligned with competency-based training. The purpose of this study was to examine the validity evidence and reliability of assessments made using the O-SCORE in a Canadian Plastic Surgery training program. Ten residents at the University of Alberta participated in this research and 41 evaluations were performed by supervising Plastic Surgeons. Residents were asked to be evaluated on three common Plastic Surgery procedures (a) breast reduction mammoplasty, (b) mandibular fracture open reduction internal fixation, and (c) and hand fracture fixation. The psychometric properties of the items were assessed using item-total correlation and Cronbach’s alpha. Validity evidence was examined through the differentiation of trainees based on level of training. Reliability testing was performed in two ways. The first was a video analysis of technical skills by Plastic Surgeons. The second was a generalizability study. This G study was a mixed, two facet design. Facets were occasion and item. Occasion in this study was nested within resident as trainees were evaluated on different occasions. The items on the O-SCORE were found to have high item-total correlations (range 0.74 to 0.88). Cronbach’s alpha for technical items based on video analysis was 0.91. It was found that assessments using the O-SCORE significantly differentiated junior and senior trainees on all items and overall score, Λ = 0.08, F(4,36) = 2.77, p < 0.001. The O-SCORE contains a dichotomous item assessing competency and this was found to be significantly different between junior and senior trainees, χ2 = 29.73, ν = 1, p < 0.001. The dependability coefficient from the generalizability study was 0.91. A decision study was conducted and showed that a minimum of two occasions were necessary to obtain a reliability coefficient of greater than 0.85. The O-SCORE and its entrustability-scaled anchors offer a construct-aligned assessment tool for operative skills. At the University of Alberta’s Plastic Surgery residency program, it has been shown that this tool offers high internal consistency of technical items, high overall reliability with three occasions, and validity evidence. Assuring competence is the goal of CMBE and using the O-SCORE as part of an assessment armamentarium is a step closer to this objective.
This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for the purpose of private, scholarly or scientific research. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
Citation for previous publication

File Details

Date Uploaded
Date Modified
Audit Status
Audits have not yet been run on this file.
File format: pdf (PDF/A)
Mime type: application/pdf
File size: 1532031
Last modified: 2016:11:16 15:23:28-07:00
Filename: Budden_Curtis_R_201606_MEd.pdf
Original checksum: d74ef879c3e34e7aef6c09b3d829faa6
Activity of users you follow
User Activity Date