Hostname: page-component-7c8c6479df-nwzlb Total loading time: 0 Render date: 2024-03-29T15:55:38.917Z Has data issue: false hasContentIssue false

Foundation programme assessment tools in psychiatry

Published online by Cambridge University Press:  02 January 2018

Asim Naeem
Affiliation:
Division of Mental Health, St George's, University of London, Cranmer Terrace, London SW17 0RE, email: a.naeem@sgul.ac.uk
Andrew Kent
Affiliation:
Division of Mental Health, St George's, University of London
Ajay Vijayakrishnan
Affiliation:
Charing Cross Higher Specialist Training Scheme, London
Rights & Permissions [Opens in a new window]

Extract

In line with Modernising Medical Careers (Department of Health, 2003), the foundation year programme aims to bridge the gap between undergraduate and specialist training. Psychiatry posts have been incorporated into the second year of this programme, with satisfactory progress of doctors being monitored via a range of workplace-based assessment tools. Learning that occurs in the context of the daily workplace is more likely to be relevant and reinforced, leading to better practice (Davis et al, 1995).

Type
Education & Training
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2007

In line with Modernising Medical Careers (Department of Health, 2003), the foundation year programme aims to bridge the gap between undergraduate and specialist training. Psychiatry posts have been incorporated into the second year of this programme, with satisfactory progress of doctors being monitored via a range of workplace-based assessment tools. Learning that occurs in the context of the daily workplace is more likely to be relevant and reinforced, leading to better practice (Reference Davis, Thompson and OxmanDavis et al, 1995).

This paper provides an overview for consultants, specialist registrars (SpRs) and staff grade/associate specialists, all of whom may be approached to assess foundation year 2 trainees using these competency-based assessments. Examples of psychiatric settings in which the range of workplace-based assessment tools can be used and a critical review of their usefulness are considered.

The assessment tools

There are four tools that assessors may be asked to complete by foundation year 2 psychiatric trainees:

  1. mini-Clinical Evaluation Exercise (mini-CEX)

  2. case-based discussion (CbD)

  3. mini-Peer Assessment Tool (mini-PAT)

  4. direct observation of procedural skills (DOPS)

Although the trainee holds responsibility for deciding the time of the assessments, where they occur, and selecting their assessors (Reference Davies, Archer and HeardDavies et al, 2005), they have to complete a minimum number of each during foundation year 2. General instructions for using these tools are available at the Modernising Medical Careers website (http://www.mmc.nhs.uk/pages/assessment)

Box 1. Examples of foundation year 2 mini-Clinical Evaluation Exercise (mini-CEX) in psychiatry

  1. Risk assessment in schizophrenia/case of self-harm

  2. Explaining the treatment options of depression/schizophrenia to a user/carer

  3. Assessing capacity to consent

  4. Cognitive assessment in an older person

  5. Assessing for features of alcohol dependence syndrome and its complications

  6. Eliciting extrapyramidal side-effects of antipsychotic medication and explaining treatment options

  7. Assessing for the physical complications of substance misuse

mini-Clinical Evaluation Exercise (mini-CEX)

This is a 15-min ‘snapshot’ assessment of an observed clinical encounter, adapted from the American mini-CEX, which was originally designed as a replacement for the traditional long case format. It is adaptable to a range of clinical scenarios, producing roughly comparable scores over examiners and settings (Reference Norcini, Blank and ArnoldNorcini et al, 1997), and has been shown to be a feasible and reliable evaluation tool (Reference Durning, Catian and MarkertDurning et al, 2002). Each mini-CEX should be followed by a 5-min instant feedback by the assessor. Although the difficulty of each mini-CEX is dependent on the patient's complexity, assessors tend to take this into account by overcompensating for patient difficulty (Reference Norcini, Blank and DuffyNorcini et al, 2003).

Strengths

Being similar to the MRCPsych objective structured clinical examination (OSCE) format, certain areas lend themselves well to being assessed with this tool (Box 1). Scenarios should be incorporated into daily clinical practice settings such as ward reviews or out-patient clinics. There is scope for other professionals (for example, liaison psychiatric nurses) to act as assessors, while the opportunity for instant feedback can help trainees refine their interview techniques and diagnostic skills by encouraging reflective practice.

Physical examination skills with a relevance to psychiatry can also be assessed. This should encourage assessors to keep their skills up to date, a concern highlighted by Garden (Reference Garden2005).

Weaknesses

Only small aspects of the psychiatric consultation process can be assessed, and there may be a greater variance with psychiatric patients (in terms of rapport, willingness to discuss issues or volunteering information). Concerns have been raised that assessors tend to form limited general impressions of trainees based on their assessment of only one or two objectives (for example clinical skills, professionalism). This may be exaggerated in psychiatry, where verbal communication skills play a key role.

Box 2. Example of case-based discussion questions in psychiatry

Assessment of alcohol misuse for detoxification

  1. How did you assess motivation in the patient?

  2. How did you decide on in-patient v. out-patient detoxification?

  3. What are the goals of treatment, and how does it fit in with the overall management of this patient?

  4. How did you assess for withdrawal symptoms?

  5. How did you calculate the dose of chlordiazepoxide required?

  6. What physical complications have you foreseen, and have you guarded against them? Have you requested any physical investigations, and what are you looking for?

  7. What psychiatric comorbidities have you looked for?

Case-based discussion (CbD)

By focusing on a trainee's case note records, this allows the assessment of the trainee's clinical decision-making, reasoning and application of medical knowledge with actual patients (Reference Brown and DoshiBrown & Doshi, 2006). It is based upon the concept of ‘chart stimulated recall’, used in the USA and Canada. As each CbD lasts only 15 min, only one aspect of the case (rather than the entire case) should be explored (see Box 2).

Strengths

‘Holistic’ aspects of the treatment of common mental illnesses can be discussed, in relation to what the trainee actually did. It can also be useful to explore the trainee's ‘hierarchy’ of drug management (for example using antidepressants or antipsychotics), and whether it is consistent with the current evidence base. Issues of capacity and consent may be better looked at within a psychiatric context, as can differential diagnoses and underlying aetiological factors.

Case-based discussion can enhance the skills of doctors who may enter other specialties (for example general practitioners, gastroenterologists and endocrinologists who come across comorbid alcohol misuse or complications of obesity), and can encourage good record-keeping.

Weaknesses

The duration is not long enough to explore complex patient's problems, and there can be a danger of the exercise turning into a mini-long-case viva, particularly with assessors who are more familiar with that format. Psychiatric notes are more comprehensive than those of other specialties, so it is often necessary to adapt the questioning for certain cases.

mini-Peer Assessment Tool (mini-PAT)

This is a form of multi-source feedback, and has been adapted from the Sheffield Peer Review Assessment Tool (Reference Archer, Norcini and DaviesArcher et al, 2005). It is based on the concept of 360° assessment, in which a trainee seeks feedback about their performance at work from a variety of colleagues, highlighting areas of strength and those in need of improvement (Reference KingKing, 2002). It is a useful way of assessing generic skills (such as communication, team working, teaching and reliability), which indirectly measure performance (Reference Hays, Davies and BeardHays et al, 2002). In UK pilots, it has been found to be practical and acceptable to senior house officers in hospital settings (Reference White House, Walzman and WallWhitehouse et al, 2002). A variation of this tool (the Team Assessment of Behaviour) is used in some regions.

Strengths

The principles of 360° appraisal are supportive to interprofessional team development (Reference McLellan, Bateman and BaileyMcLellan et al, 2005). It can help foundation year 2 trainees to develop a holistic approach to patient care by playing a part in multiprofessional ward reviews and care programme approach (CPA) meetings.

By assessing aspects of the doctor-patient relationship, this tool assesses the qualities which overlap with attributes of a good psychiatrist (Reference Bhugra and HolsgroveBhugra & Holsgrove, 2005). Ward reviews or CPA meetings can allow trainees to display their communication skills with patients and carers (for example explaining diagnoses or treatments) in front of other health professionals, some of whom could be selected to complete the mini-PAT forms. Out-patient psychiatric clinic letters also allow a trainee's written communication skills to be assessed, as copies are usually sent to other members of the multidisciplinary team. The use of several assessors in the mini-PAT process leaves the tool less open to bias.

Weaknesses

In the context of the shift system of psychiatric on-call work, and the short duration of each post, how many peer colleagues are in a position to accurately comment on a trainee's performance? Evans et al (Reference Evans, Elwyn and Edwards2004) have commented on this concern in other specialties. It can also encourage unduly positive feedback in the ‘space for comments’ section, owing to concerns that trainees may recognise anonymous comments highlighting poor performance. The most valid source of ratings for ‘humanistic’ dimensions are patients (Reference ChurchChurch, 1997), particularly in psychiatry, but they are surprisingly excluded from the assessment process.

Direct observation of procedural skills (DOPS)

This assesses trainees’ practical skills, in a range of pre-determined tasks with a patient. Each DOPS should last no longer than 15 min, followed by 5 min of feedback.

Tasks such as venepuncture (for clozapine blood monitoring or for plasma lithium levels), performing an electrocardiogram (for example prior to considering antipsychotic treatment) or giving electroconvulsive therapy can be readily assessed, placing emphasis on the trainee's communication skills (for example obtaining valid consent and explaining the need for the test) in performing the task. Aside from these examples, this tool currently has limited applicability in foundation year 2 psychiatry posts.

Discussion

There is no nationally agreed undergraduate curriculum in psychiatry to act as a baseline template for foundation year 2 competency assessments, with medical schools having considerable autonomy in how they teach and examine psychiatry. The ‘ten essential shared capabilities’ (Reference HopeHope, 2004) are an attempt to reach a consensus on what competencies should be core for the entire mental health workforce. This can act as a parallel framework for applying workplace-based assessment tools for foundation year 2 psychiatric trainees.

It is essential that trainees get a fair assessment of their competence, by pitching the tools at an appropriate level, in the context of more generic career development; this requires adequate training of all potential assessors. It would be reasonable to expect a foundation year 2 psychiatric trainee to perform a safe and effective suicide risk assessment, or be able to describe their routine approach to the assessment of pyrexia or confusional state within a psychiatric context. However, it may not be appropriate to expect a detailed knowledge or experience of cognitive-behavioural therapy.

Although foundation year 2 trainees may have more medically oriented approaches to the presentation of common mental health problems (for example assessment and management of substance misuse), the assessment tools can provide an opportunity to reinforce the importance of personal, family, social and cultural factors.

Feedback from these tools should encourage trainees to create self-directed learning plans. However, there are concerns as to whether this actually happens in reality (Reference Norman, Shannon and MarrinNorman et al, 2004). Criticisms also exist regarding the vagueness of the scoring systems and the absence of independent assessors, as most assessors are known to the trainee (Reference RoseRose, 2006).

Exposure to foundation year 2 psychiatry posts offers an opportunity to boost recruitment into our specialty, but evidence suggests that undergraduate experience may be a more positive determining factor (Reference Goldacre, Turner and FazelGoldacre et al, 2005). Therefore, these assessment tools should be used as an opportunity to develop more advanced psychiatric competencies in trainees who may enter other specialties.

Workplace-based assessment tools for specialist training

Although the foundation year 2 assessment tools will also be used to assess doctors in specialist training, some of the formats have been adapted by the Royal College of Psychiatrists (for example the mini-CEX has been revised to become the mini-Assessed Clinical Encounter or mini-ACE). Additional tools are also currently being piloted, including the Assessment of Clinical Expertise (ACE), case and journal club presentations, and a patient satisfaction questionnaire (details are available from the Royal College of Psychiatrists’ website: http://www.rcpsych.ac.uk/training/specialtytrainingassess.aspx). Assessors are likely to have to use different sets of workplace-based assessment tools for trainees in their foundation year 2 or specialist training years 1-5.

Conclusions

Foundation year 2 assessment tools provide structured, standardised methods of monitoring doctors’ progress, against a broad range of core competencies. Brown & Bhugra (Reference Brown and Bhugra2005) have highlighted the need for such tools to assess clinical and non-clinical competencies. By grounding these assessments within a relevant workplace context, it is hoped that trainees will retain the skills they learn. As the character Arthur said in Patrick White's The Solid Mandala: ‘I forget what I was taught. I only remember what I’ve learnt.’

Declaration of interest

None.

Acknowledgement

We are grateful to Dr Joan Rutherford (Consultant Psychiatrist, South West London and St George's Mental Health NHS Trust) for reading and commenting on the final draft of this paper.

References

Archer, J. C., Norcini, J. & Davies, H. A. (2005) Use of SPRAT for peer review of paediatricians in training. BMJ, 330, 12511253.Google Scholar
Bhugra, D. & Holsgrove, G. (2005) Patient-centred psychiatry. Training and assessment: the way forward? Psychiatric Bulletin, 29, 4952.CrossRefGoogle Scholar
Brown, N. & Bhugra, D. (2005) Modernising medical careers, the Foundation Programme and psychiatry. Psychiatric Bulletin, 29, 204206.Google Scholar
Brown, N. & Doshi, M. (2006) Assessing professional and clinical competence: the way forward. Advances in PsychiatricTreatment, 12, 8189.Google Scholar
Church, A. (1997) Do you see what I see? An exploration of congruence in ratings from multiple perspectives. Journal of Applied Social Psychology, 27, 9831020.Google Scholar
Davies, H., Archer, J., Heard, S., et al (2005) Assessment tools for foundation programmes – a practical guide. BMJ Careers, 330, 195196.Google Scholar
Davis, D. A., Thompson, M. A., Oxman, A. D., et al (1995) Changing physician performance: a systematic review of the effect of continuing medical education strategies. Journal of the American Medical Association, 274, 700705.Google Scholar
Department of Health (2003) Modernising Medical Careers. Department of Health.Google Scholar
Durning, S. J., Catian, L. J., Markert, R. J., et al (2002) Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Academic Medicine, 77, 900904.CrossRefGoogle ScholarPubMed
Evans, R., Elwyn, G. & Edwards, A. (2004) Review of peer assessments of physicians. BMJ, 328, 12401242.Google Scholar
Garden, G. (2005) Physical examination in psychiatric practice. Advances in PsychiatricTreatment, 11, 142149.Google Scholar
Goldacre, M. J., Turner, G., Fazel, S., et al (2005) Career choices for psychiatry: national surveys of graduates of1974–2000 from UK medical schools. British Journal of Psychiatry, 186, 158164.Google Scholar
Hays, R. B., Davies, H. A., Beard, J. D., et al (2002) Selecting performance assessment methods for experienced physicians. Medical Education, 36, 910917.Google Scholar
Hope, R. (2004)The Ten Essential Shared Capabilities – A Framework for the Whole of the Mental Health Workforce. Department of Health.CrossRefGoogle Scholar
King, J. (2002) 360 degree appraisal. BMJ Career Focus, 324, s195.Google Scholar
McLellan, H., Bateman, H. & Bailey, P. (2005) The place of 360 degree appraisal within a team approach to professional development. Journal of Interprofessional Care, 19, 137148.Google Scholar
Norcini, J. J., Blank, L. L., Arnold, G. K., et al (1997) Examiner differences in the mini-CEX. Advances in Health Sciences Education: Theoryand Practice, 1, 2733.Google Scholar
Norcini, J. J., Blank, L. L., Duffy, D., et al (2003) The Mini-CEX: a method for assessing clinical skills. Annals of Internal Medicine, 138, 476481.Google Scholar
Norman, G. R., Shannon, S. I. & Marrin, M. L. (2004) The need for needs assessment in continuous medical education. BMJ, 328, 9991001.Google Scholar
Rose, N. (2006) Hazards ahead? Invited commentary on: Assessing professional and clinical competence. Advances in Psychiatric Treatment, 12, 8991.Google Scholar
White, P. (1969)The Solid Mandala. Penguin.Google Scholar
White House, A., Walzman, M. & Wall, D. (2002) Pilot study of 360 degree appraisal of personal skills to inform record of in-training assessments for senior house officers. Hospital Medicine, 63, 172175.Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.