Oldfield, Zaita (2018) Making the cut: an evaluation of selection into the Royal Australasian College of Surgeons’ surgical training program. Other Degree thesis, Victoria University.

Abstract

Selection of trainees for surgical training is widely acknowledged to be both complex and important. Doctors embarking on surgical careers expend considerable time and resources in training, as do those responsible for facilitating their learning. The aim of surgical training is to prepare surgeons to be competent, effective, ethical practitioners in unsupervised clinical practice as consultant surgeons (Carroll, Kennedy, Traynor & Gallagher, 2009; Elfenbein, Sippel, McDonald, Watson, Scarborough & Migaly, 2015). The aim of selection must be to admit those who are most likely to succeed in surgical training and beyond. Admitting candidates who are inadequately suited to surgical training can result in trainees who unduly struggle or are unable to satisfy training requirements, and may ultimately jeopardise patient safety. In contrast, not accepting candidates who are well-suited to surgical training may be unfair to both candidates and communities that miss out on skillful surgeons. These aspects are among those that make selection to surgical training an extremely ‘highstakes’ activity. Those responsible for selection grapple with assessing candidates’ current skills and attributes, and with ascertaining their likely future performance. Limitations of, and tensions between interacting elements—human agency, requirement specifications, instruments, processes, influences—in selection to surgical training, mean that the long-term outcomes of selection can never be assured. This study identifies principal elements in selection to surgical training, discusses influences on, and interrelationships between the elements, and reviews connections between selection and surgical training assessments. This study has appraised the current instruments used for selection into the Royal Australasian College of Surgeons’ (RACS) General Surgery (GS) training program in Australia and New Zealand, to establish their effectiveness in predicting trainees’ performance in assessments during the first two years of training. Data were considered for selection and assessment items for trainees for three yearly cohorts, selected in 2008, 2009, and 2010. The study compared the performance of trainees in three selection instruments—a structured CV, a structured referee report, and a multi-station interview—to their performance in three examinations and three work-based assessments. Firstly, Pearson product-moment correlations were calculated to examine the extent of relationships within each set of variables to determine intra-relationships of the selection items, of the examination items, and of the work-based assessment items. Secondly, Pearson productmoment correlations were conducted to determine degrees of association between selection items and performance in each of the subsequent assessment items. Thirdly, multiple regression analysis was conducted to determine the extent to which trainees’ scores in the selection items (independent variables) predicted scores in each of the assessments during training (dependent variables). The model fit and strength was assessed using the analysis of variance (ANOVA) step within the regression analysis. The relative strength of the associations between dependent variables and the independent variables were assessed using regression coefficients. The findings of this study have shown that performance in RACS GS selection partially predicts performance in assessments during training. In general, candidates’ performance varied across each of the selection items, performance in all examinations was highly consistent and performance in major end of term work-based assessments was also consistent. Most correlations between performance in selection and performance in assessments during training were positive, with the exception that performance in the CV was usually inversely correlated with subsequent assessments. The referee reports and total selection scores were predictive of performance in the major end of term work-based assessments. This study reviewed many factors that affect selection processes and outcomes. Several of these—such as the role of procedural justice, identification of desired attributes of trainees, and the validity, reliability, fairness, and acceptability of selection instruments and protocols— implicitly framed RACS GS selection. However, to maximise the effectiveness of selection instruments and protocols, these and other relevant influences could be explicitly defined for the local context. The findings regarding the predictive capacity of the selection instruments— particularly for the referee reports and the interviews—differ from some other studies, but highlight that the implementation and content of selection instruments are key aspects affecting their performance. Combining scores from multiple instruments reduces the influence of any individual selection instrument. Overall, with the possible exception of the CV, the study has shown that the RACS GS selection tools are performing moderately well. However, the RACS GS selection instruments and processes could be modified to maximise their effectiveness and new, emerging selection activities could be considered.

Search Google Scholar

Repository staff login