AIMS AND METHOD
The aim of this study was to investigate the pattern of performance of candidates in the MRCPsych examinations in the period 1997–2002 inclusive. During this period 6306 candidates sat the MRCPsych part I and 4536 sat the MRCPsych part II.
The overall pass rates for MRCPsych part I was 44.7% and 48.9% for MRCPsych part II. The majority of those who passed either part of the examinations did so on their first attempt; females performed better than males and graduates of UK and Republic of Ireland medical schools performed better than foreign graduates in all component parts of the examinations.
The pattern of results suggests that clinical tutors and course organisers need to pay special care to the training needs of trainees who are foreign medical graduates.
The Membership examinations (parts I and II) of the Royal College of Psychiatrists (MRCPsych) are important in determining the career progression of trainee psychiatrists. There is evidence that success in these examinations, as with other postgraduate medical examinations, is associated with age, gender, original medical school of training and ethnicity (Ayres et al, 1996; Krueger, 1998; Tyrer et al, 2002). For both parts of the MRCPsych examination those who passed were significantly younger and had qualified for a significantly shorter period of time than those who failed. Furthermore, the pass rates were higher for women than men; this was accounted for by the relatively higher failure rate of male candidates who qualified from medical schools outside the UK and Ireland (Tyrer et al, 2002).
The previous published study of the factors associated with success (Tyrer et al, 2002) investigated only one cohort each of both parts of the examinations. Also, the study used overall success as the outcome measure. The aim of the current study was to investigate the pattern and determinants of success in the MRCPsych examinations based on a larger cohort utilising the cumulative results of the MRCPsych examinations for a 6-year period (1997–2002 inclusive). Performance in the written and clinical examinations was investigated.
MRCPsych part I
In the study period, 6306 candidates sat the MRCPsych part I examination and 2816 of these passed (44.7%). The majority of those who passed did so on their first attempt (n=1970, 70%). The pass rate for females was 51.9% and 39.8% for males. The overall pass rate varied according to original medical school of training. The pass rate of candidates graduating from UK and Irish medical schools was 59.8% and 48.6% respectively. This compared with 21% for graduates from Indian borders (Afghanistan, Pakistan and Nepal), 37.3% from the Indian subcontinent (India and Sri Lanka) and 36.7% for the others. This difference was statistically significant (χ2=468.25, d.f.=4, P<0.001). The UK and Irish graduates were more likely to pass on their first attempt (Table 1).
The written examination was introduced as a screening test in 1997. The performance in the written and clinical examinations varied according to original medical school of training. UK graduates performed significantly better than the others in the multiple choice question (MCQ) paper (χ2=113.9, P<0.001) and the UK and Irish graduates both performed significantly better than the others in the clinical examination (χ2=271.1, P<0.001). The performance of graduates from the Indian borders was worse in both aspects of the examinations (Table 2).
MRCPsych part II
In the study period, 4536 candidates sat the MRCPsych part II examinations, and 2220 of these passed (48.9%). The majority of those who passed did so on their first attempt (n=1372; 69%). The pass rate for females was 54% and for males 45.1%. The overall pass rate varied according to original medical school of training. The pass rate for candidates graduating from UK and Irish medical schools was 68.0% and 55.5% respectively. This compared with 24.8% for candidates graduating from Indian borders, 35.5% for candidates from the Indian subcontinent and 33.2% from other medical schools. This difference was statistically significant (χ2=527.5, d.f.=4, P<0.001). UK graduates were more likely to pass on their first attempt (χ2=592.6, P<0.001) (Table 3).
Data were available for the performance of candidates on the different component parts of the examinations for autumn 1998, spring 1999, spring and autumn 2000, and spring 2001. The UK and Irish graduates were consistently better across all the component parts of the examination (Table 4). The difference was most marked in the written papers, particularly the critical review paper and the MCQ paper.
The performance of UK and Irish graduates in both parts of the MRCPsych examinations was better than that of foreign medical graduates. This difference was evident in the written as well as in the clinical examinations. This pattern of results in which home graduates perform better at medical examinations compared with foreign graduates has been reported in the UK (Wakeford et al, 1992), USA (Meskauskas et al, 1977; Benson et al, 1981; Quick & Robinowitz, 1981; Mick & Mou, 1991) and Canada (Lowy & Dongier, 1979). Indeed, there is evidence that US citizens who are foreign medical graduates do comparably worse than US medical graduates on standardised medical examinations (Weinberg & Bell, 1978; Norcini et al, 2006). The finding that country of original medical qualification predicts performance suggests that the type and quality of undergraduate medical education may continue to have an effect on postgraduate education and performance. This is probably not the whole story; it may be that the culture of medical education and cultural aspects of communication are contributing to the outcome of examinations.
In an investigation of the performance of candidates from Asia in the oral examinations for the Membership of the Royal College of General Practitioners (MRCGP), Roberts et al (2000) showed that ‘the language of the oral examination is not a transparent medium through which information passes but a set of discourse that actively constructs a particular way of looking at the world’. They showed that there are at least three types of discourse: personal experience discourse; professional discourse; and institutional discourse. Success in the oral examination depended not simply on technical knowledge but on the facility of the candidate to manipulate different kinds of discourse that are not explicitly delineated. For example, examiners may ask questions seemingly in one area of discourse, yet they very often expect an institutional discourse response regardless of the apparent nature of the question. In other words, language and culture influence the outcome for candidates. Given the degree to which psychiatry as a subject is intricately dependent upon language and culture, it would be surprising if the same kind of issues were not relevant to the MRCPsych clinical examinations.
The performance of foreign medical graduates appears to improve over time, suggesting that there is a practice effect operating. It may be that the culture of examinations in the UK is being acquired at real attempts at the examination. Local tutors may have to ensure that foreign medical graduates have opportunities to practise at realistic ‘mock’ examinations before sitting the real thing.
The difference in performance holds true for all aspects of the College examinations and is even more salient in the written examinations. The written examinations include MCQ papers, essay papers and the critical review paper. The MCQ format requires a high degree of linguistic proficiency. It is possible that language comprehension is influencing the outcome in the MCQ. There is evidence that MCQs include imprecise terms for which consensus does not exist even among examiners (Holsgrove & Elzubeir, 1998). These imprecise terms might pose more difficulty for candidates who do not have English as a first language. In addition, the content of the MCQ paper may be an issue. For example, behavioural science is a major component of undergraduate medical education in the Western world, including the UK. However, this is not necessarily the case in other parts of the world. UK graduates already have grounding in this subject, whereas foreign graduates may find themselves introduced to this subject area for the first time. Organisers of local MRCPsych courses might need to be aware of this, and ensure that the behavioural science component of the courses is well taught.
In the previous study by Tyrer et al (2002), age was associated with the outcome of sitting the MRCPsych examination. Older age as an important factor in determining poor performance is a recurring finding in studies of performance in postgraduate medical examinations (Blacket, 1990; Mick & Mou, 1991). However, in an Israeli study investigating the results of medical licensing examinations, the highest success rate was in the 31- to 35-year age-group, particularly those with an average of 12 years of clinical experience (Schenker, 1993). In our study we did not examine the influence of age on performance. Furthermore, Tyrer et al found (2002) gender to be associated with success in the MRCPsych examination. This was confirmed in the current study.
There is evidence that the quality of the training programme influences the pass rates in postgraduate qualifying examinations. For example, it has been shown that the number of trainees in a postgraduate training programme is associated with the outcome in examinations. In a Canadian study, small training programmes, that is those with three or fewer trainees, had significantly lower pass rates in written examinations. These training programmes also had a higher proportion of foreign medical graduates (McKendry & Dale, 1995). In the USA, foreign medical graduates and US citizens trained abroad performed worse in the certifying examination in internal medicine, and this poor performance was associated with attending different kinds of training programme than US citizens (Norcini et al, 1986). Although, there is concern that rural and remote training programmes in family medicine in Canada may disadvantage trainees compared with those attending urban training centres, investigation does not support this (McKendry et al, 2000). It may be helpful in future studies to investigate whether foreign graduates are more likely to be working in small or remote training programmes and whether this is associated with performance in the examinations.
It is important that the College examinations are fair to all candidates and that the outcome for candidates is a true reflection of knowledge and skills. Equally, it is important that foreign medical graduates have similar opportunities to UK graduates in order to gain adequate knowledge and clinical experience so as to be able to adequately prepare for the College examinations. Further research into the College examinations will help to clarify the mechanisms by which the factors that are established as influencing outcome operate.
- © 2007 Royal College of Psychiatrists