Hostname: page-component-7c8c6479df-24hb2 Total loading time: 0 Render date: 2024-03-28T17:36:47.029Z Has data issue: false hasContentIssue false

Performance concerns in psychiatrists referred to the National Clinical Assessment Service

Published online by Cambridge University Press:  02 January 2018

Denis O'Leary*
Affiliation:
University of Oxford
Pauline McAvoy
Affiliation:
National Clinical Assessment Service
John Wilson
Affiliation:
University of Oxford
*
Rights & Permissions [Opens in a new window]

Abstract

Aims and method

Relative to workforce numbers, the National Clinical Assessment Service (NCAS) has reported high rates of referrals for performance concerns in psychiatrists. Our aim was to see how these concerns would be distributed across good medical practice (GMP) domains. Such specification would help identify appropriate training and development interventions. Concerns were obtained from consecutive case reports (anonymised) of psychiatrists referred to NCAS between October 2004 and April 2006.

Results

The proportion of psychiatrists with concerns in any domain was highest (just over 50%) in the domains of good clinical care and working with colleagues. Two-thirds had concerns across multiple domains.

Clinical implications

Training and development needs of psychiatrists referred to NCAS are more complex and more prevalent in the GMP domain of good clinical care than previously reported.

Type
Original Papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2010

The National Clinical Assessment Service (NCAS), inaugurated in 2001, advises National Health Service (NHS) trusts on the management of performance concerns in doctors and dentists. 1,2 Across the medical specialties, referral rates relative to workforce numbers are consistently high for psychiatrists. Reference Berrow, Faw and Jobanputra3-6 These findings are consistent with studies reporting high proportions of psychiatrists referred to professional regulatory bodies. Reference Donaldson7,Reference Morrison and Morrison8 Improving the health of psychiatrists might help, Reference Firth-Cozens9 yet there remains a need to understand the potential training and development implications of these findings. However, the reporting of concerns under such broad categories as behaviour, clinical capability and health, derived from national disciplinary procedures, 2 does not allow for useful learning needs analyses at the level of the individuals, their employers or the College. Reference Boydell10-Reference Buckley and Caple12 Addressing these performance concerns through existing postgraduate training and continuing professional development (CPD) programmes is difficult therefore. Reference Palmer and Wilson13 This challenge could be overcome by adopting the performance domains detailed in Good Medical Practice (GMP), 14 because they are more specific and form the basis of postgraduate training curricula and NHS career grade appraisal systems. 15,16

This paper aims to describe the performance concerns in psychiatrists referred to NCAS using the domains in GMP. To our knowledge it is the first report to present concerns specifically in this way.

Method

Cohort selection and case management

The NCAS database was used to recruit a consecutive series of referred psychiatrists for the period 1 October 2004 to 31 March 2006. On referral, NCAS advice reception staff record preliminary details for the case in the database (including a case number, specialty and nature of performance concerns) and forward them to the NCAS case advisor. The advisor obtains more details on the concerns from the referrer, records the details using a standardised template – the case report – and forwards the advice to the referrer using another template, the advice summary.

Practitioners' baseline categories and characteristics

Practitioners' categories and characteristics at baseline were obtained from the NCAS database and the anonymised case files, and included the following information:

  1. sociodemographic: age, gender and ethnicity;

  2. professional: place of qualification, grade, specialty;

  3. employer: contract type, trust type, country;

  4. referral source: employer or self.

Identification and typing of concerns

Source and duration of concerns

Information on the source and duration of the reported concerns was obtained from the case reports. Sources were categorised as staff, patient and ‘other’. The last included data from clinical governance and/or performance management systems from within (e.g. suicide review) and external (e.g. coroner's inquest) to the referring trust.

Typing across NCAS domains

At the time this study was undertaken NCAS typed performance concerns into the categories of behaviour, clinical capability and health (the number of reporting categories has subsequently been increased to seven). 6 These were obtained from the database for each case. For ease of comparison with previous NCAS reports published at the time this study was reporting, the results are presented as the percentage of practitioners with concerns in any domain and across each domain. 4

Typing across GMP domains

The concerns recorded by the NCAS advisor in each case report were listed and typed across the seven GMP performance domains: good clinical care; relationships with patients; working with colleagues; maintaining good medical practice; teaching and training; appraising and assessing; probity; and health. 14 Concerns pertaining to inappropriate behaviour with patients and/or colleagues were typed in the probity domain and also in the other relevant domain: for example, a sexual remark to a colleague was typed in the domains of probity and working with colleagues. Other probity concerns (e.g. falsifying expenses) were recorded in the probity domain only.

Factors associated with performance concerns

To identify factors possibly linked to performance concerns, a chi-squared analysis was undertaken of baseline characteristics (independent variables) across NCAS and GMP performance domains (dependent variables) using SPSS version 10 for Windows. Statistical significance was set at a probability of 5% (P < 0.05). Fisher's exact test was used when the expected number of cases within any of the two-by-two table cells was less than 5. As the analysis was exploratory, neither Bonferroni's modification nor a multiple regression analysis was undertaken.

Construction of a GMP profile of performance concerns

To allow for greater specification of the performance concerns, a GMP profile of performance concerns is presented. First, a performance framework was constructed that specified performance aspects within each GMP domain: for example, the domain of good clinical care is underpinned by six sub-domains (e.g. care provision, supporting self-care). Within the sub-domain of care provision there are 12 underpinning components. For three of these components (prescribing, record-keeping and availability) the required performance is underpinned by two elements. The naming of the levels within each domain is not specified by the GMP guidelines, but is done here for communication purposes. Second, the profile of performance concerns was constructed by calculating for each level within the framework the percentage of practitioners with concerns in each performance aspect at that level. Aspects were included in the profile only if there were concerns in 10% or more of referred practitioners at that level.

Data release and coding

Data collection and release were approved by NCAS under its Data Collection and Release policy, 17 which aims to balance the confidentiality needs of referred practitioners with the needs of NCAS to account for its casework activity and to inform the development of NHS training and development systems. Within the framework of the National Research Ethics Service the analysis is coded as service evaluation rather than research. 18

Results

Data completeness

A consecutive series of 123 cases was recruited. Complete data were available for all the employer and referral source characteristics and for gender and grade. Data completeness was 77% and 82% on the sample's age bands and ethnicity respectively; for place of qualification and specialty it was 88% and 94% respectively.

Baseline characteristics

Most practitioners (72%) were male, with an even distribution between White and other ethnic groups (Table 1). The most frequent age band was 50-54 years. Slightly more had obtained their primary medical qualification outside the UK (53% v. 47%) and three-quarters were in career-grade posts. All grades were represented, with consultants being the largest subgroup (57%). Although all specialties were represented, most (71%) were from the adults of working age specialty. Nearly all (98%) worked in the NHS (across 67 NHS trusts). Almost a fifth had fixed term (locum) contracts and two-thirds were employed by mental healthcare trusts.

Table 1 Baseline characteristics

n (%)
Gender
   Male 88 (72)
   Female 35 (28)
Place of qualification
   UK & Ireland 51 (47)
   Other 57 (53)
Job contract
   Substantive 100 (81)
   Locum 21 (17)
   Other 2 (2)
Age, yearsa
   25–29 9 (10)
   30–34 7 (7)
   35–39 8 (8)
   40–44 14 (15)
   45–49 13 (14)
   50–54 20 (21)
   54–59 16 (17)
   60+ 8 (8)
Grade
   Career 95 (77)
   Consultant 71 (58)
   Other 24 (20)
   Training 28 (23)
Trust type
   Mental health 84 (68)
   Primary care 16 (13)
   Combined care 21 (17)
   Other 2 (2)
Specialtya
   Child/adolescent 4 (3)
   Older adults 19 (16)
   Adults 85 (73)
   Other 8 (7)
Country of referral
   England 117 (95)
   Other 6 (5)
Ethnicitya
   White 50 (50)
   Other ethnicity 51 (50)

Source and duration of concerns

Details of source and duration of concerns were available for 118 and 116 cases respectively. There was a single source of concern in 86% of cases as follows: staff 46%, ‘other’ 31%, patients 9%. Of the 14% cases with multiple sources of concerns, all involved staff. The modal duration was 1–4 weeks. In total, 38 (33%) had concerns of up to a month's duration and 89 (77%) had concerns of up to 1 year in duration.

Typing of concerns

NCAS domains

Two practitioners working in the independent sector were excluded on confidentiality grounds from the typing analysis. Based on the initial employer report to NCAS, four-fifths (80%) of practitioners had behavioural concerns; half (48%) and a fifth (22%) had concerns in the clinical and health domains respectively. Sixty-seven practitioners (55%) had concerns in a single domain.

GMP domains

Among the 121 NHS-employed practitioners, the NCAS advisors had recorded 395 performance concerns in total. The percentages of practitioners with concerns in each domain are shown in Figure 1. More than half had concerns in the domains of good clinical care (56%) and colleague relationships (55%); about a third had concerns in the probity and patient relationship domains (40% and 34% respectively). The majority of practitioners (66%) had concerns across several domains as follows: 38% across two domains, 16% in three, 10% in four and 2% in five. The mode (and median) was two domains.

Fig 1 Distribution of performance concerns across the seven Good Medical Practice domains.

Relationships between GMP domains

The nature of the relationships between the four most frequently presenting domains (good clinical practice, colleague relationships, probity and patient relationships) was explored. As concerns in the probity domain included some that were secondary to poor conduct with patients and/or colleagues, this exploration focused initially on the relationships between the three domains of good clinical practice, colleague relationships and patient relationships. Of the 121 referred practitioners, 103 (85%) had concerns in at least one of these three domains. Within this trio of domains, the most common combination of concerns was from the good clinical practice and colleague relationships domains (24% of practitioners). Of the remaining 18 practitioners, 8 had no concerns in a GMP domain, 8 had probity concerns only and 2 had probity concerns secondary to health concerns.

Factors associated with performance concerns

None of the baseline characteristics were associated with increased numbers of concerns in the NCAS domains or the GMP domains of good clinical practice and patient relationships. Significantly more concerns in colleague relationships were identified for those over 50 years old (χ2 = 4.52, d.f. = 1, P = 0.04), those who had graduated from a UK medical school (χ2 = 4.82, d.f. = 1, P = 0.03) and those with career-grade employment (χ2 = 6.21, d.f. = 1, P = 0.01). Concerns about teaching and training were linked to career-grade employment status (χ2 = 4.01, d.f. = 1, P = 0.05). In the probity domain, male gender (χ2 = 8.94, d.f. = 1, P = 0.003), training grade (χ2 = 5.13, d.f. = 1, P = 0.03) and substantive employment (χ2 = 4.19, d.f. = 1, P = 0.04) were associated with increased numbers of concerns. The small numbers of concerns in the remaining domains precluded useful statistical analysis.

Profile of GMP performance concerns

Within the domain of good clinical care, most of the concerns were in care provision (Table 2). They included concerns about diagnosis, management and prescribing. In the patient relationships domain, underperformance in the sub-domains of the doctor-patient partnership and maintaining trust in the profession met the profile threshold. Loss of trust in the profession was due to improper emotional relationships with patients in each case. In the colleague relationships domain, concerns about team working (including communication skills) and respect for colleagues (including bullying and harassment) were present in about a third of practitioners. Within the probity domain, most of the concerns pertained to conduct with staff and/or patients that undermined trust.

Table 2 Good Medical Practice performance profile for psychiatrists

Domain Sub-domain Component Element
Good clinical care (56%) Care provision (36%) Assessment (11%)
Diagnosis and management (22%)
Work within competency limits (11%)
Prescribing (19%)
Record-keeping (13%) Clear, legible content and contemporaneous (11%)
Availability (22%) Available on duty (14%)
Patient relationships (34%) Doctor–patient partnership (12%)
Maintaining trust (12%) Improper emotional relationships (12%)
Colleague relationships (55%) Teamwork (35.5%) Respect contributions/skills (16%)
Respect for colleagues (27%) Bullying and harassment (23%) Communication skills (17%)
Probity (40%) Honest and trustworthy (36%) Conduct justifying trust (31%)

Discussion

Although original in describing workplace performance concerns across GMP domains nationally in psychiatrists, our study has several limitations. Methodologically it is retrospective and restricted to those working in the NHS (independent providers were not included in NCAS referrals at the time). Furthermore, in the absence of recent NHS data, it is difficult to know whether the sample is representative of psychiatrists in the NHS workforce. Existing NCAS data for all specialties show that male gender, consultant grade, ethnicity, older age and primary medical qualification outside the UK are associated with increased risk of referral. 5,6 Conceptually, any referred performance concerns may not have been upheld by subsequent investigation, and the paper does not address any contributory factors. Concerns may be due primarily to work context rather than individual competency, for example. 6,19 Finally, GMP guidance is not a performance or competency framework. 14 However, its use can be justified on the grounds that it has been the basis for the psychiatry postgraduate curriculum and NHS appraisal. 15,16 Before discussing the findings in relation to GMP, we will consider the findings in relation to referral source and previous NCAS data on concerns in psychiatrists.

Source of referral

Such data have not been published previously by NCAS. Concerns about psychiatrists' performance were raised most commonly by other staff members, reflecting possibly the multidisciplinary nature of the specialty. If replicated, the results highlight some important implications for the identification of performance concerns: the importance of having a variety of mechanisms in place; the role of informal (e.g. staff verbal report) in addition to formal (e.g. clinical governance) processes; and the importance of patient feedback. A related issue is whether other specialties (such as surgery) would have a similar distribution of referral source. This is as yet unknown; however, the present findings can be compared with previous NCAS data on psychiatrists. Reference Berrow, Faw and Jobanputra3-6

Comparison with previous NCAS data

The distribution of concerns across the domains of clinical capability, health and behaviour is similar to previous NCAS reports on performance concerns in psychiatrists. Reference Berrow, Faw and Jobanputra3-5 More recently, NCAS has reported across an increased number of categories (seven) and reported that clinical difficulties were present in 50%, behavioural concerns in 70% (misconduct and other behaviours combined) and concerns about health, including alcohol or drug misuse, in 31%. Reported in this way, concerns about behaviour remain high. At employer level it would appear that cases are referred for advice earlier than previously. This may reflect cohort effects as the duration of concerns in those referred to NCAS has been decreasing annually. 5,6 This earlier involvement in concerns' handling by NCAS is one of its key objectives. 1 Nevertheless, the consistency of concerns across these data-sets, spanning up to 8 years since NCAS inception (2001–9), suggests the need for a more systematic response to addressing these concerns. Any response can be guided by more clarification on the nature of these concerns, specifically their distribution across GMP domains.

Performance concerns across GMP domains

Using the GMP framework and after further exploration and discussion of concerns by the NCAS advisor with the referrer, the referred cases are seen to be more complex. Concerns about good clinical care are the most common, along with concerns about working with colleagues. About two-thirds have concerns in at least two GMP domains, compared with less than half (46%) using the NCAS domains; 85% of practitioners had concerns in at least one of the good clinical care, colleague and patient relationship domains; and the most common pattern was a combination of concerns in the domains of good clinical care and working with colleagues. Most of the probity concerns were due to improper relationships with patients and or staff. Managers appear to refer the more complex cases.

This paper throws further light on presenting concerns through exploration of characteristics that may be associated with increased numbers of concerns and construction of the GMP performance profile. Although exploratory and preliminary, the results tentatively suggest that male, career-grade psychiatrists over 50 years old may be experiencing problems with colleague relationships, whereas their trainee counterparts may be at increased risk of probity concerns.

The GMP performance profile is helpful in two respects. It identifies those performance aspects across several levels that underpin concerns at domain level, thus providing more detailed training needs analysis. Additionally, the profile indicates that there are training needs in areas that are more specific to psychiatry (e.g. diagnosis and management) and those that are generic to all specialties (e.g. respect for colleagues). Intriguingly, it is the more generic type of concern that appears most often in the profile, thus supporting efforts to base training across the range of good medical practice, as in the curricular innovations of the Modernising Medical Careers programme. Reference Palmer and Wilson13 Do the findings have any further implications for addressing performance concerns in psychiatrists?

Further implications

Psychiatrists, relative to workforce numbers, have a high rate of referral to the NCAS and a high prevalence of behavioural concerns relative to most other specialties. 5,6 Modelling of referral data to NCAS suggests that the referral pattern might be partly due to effects of ethnicity and place of qualification within the psychiatry workforce. Concerns that are secondary to health problems might respond to initiatives aimed at health improvement. Reference Firth-Cozens9 From a training and development perspective it is unclear from our findings whether poor performance is due to inadequate transfer of skills from training, attrition of skills once in the workplace, or other factors affecting performance such as health and workload. In terms of a systematic response by the College, the options include reviewing the CPD programme and developing relevant specialty-specific and generic interventions. Generic interventions should focus on team-working skills and interpersonal behaviours that affect trust in relationships, both with staff and with patients. Specialty-specific interventions should focus on diagnosis and management, prescribing and record-keeping. Helping career-grade psychiatrists adapt to cultural shifts in team working may be relevant in addition.

For trainees, our findings on the source of concerns supports the College's approach of using staff and patient feedback in workplace-based assessments. The results also show that a more valid workplace-based assessment requires the integration of both patient and staff feedback with results obtained from clinical governance and performance management processes. Attributing increased weight within the curriculum to inclusion of competencies in the domains of working with colleagues and patient relationships is supported by these data. There may also be a role for employers through incorporation of clear guidelines on performance management issues, including expected behavioural norms towards patients and staff, in trainee workplace induction programmes.

Footnotes

Declaration of interest

None.

References

1 Department of Health. Assuring the Quality of Medical Practice: Implementing Supporting Doctors, Protecting Patients. Department of Health, 2001.Google Scholar
2 Department of Health. Maintaining High Professional Standards in the Modern NHS. Department of Health, 2004.Google Scholar
3 Berrow, D, Faw, L, Jobanputra, R. Evaluation, Research and Development, Analysis of the First 50 NCAS Assessment Cases. National Clinical Assessment Service, 2005.Google Scholar
4 National Clinical Assessment Service. Analysis of the First Four Years' Referral Data. NCAS, 2005.Google Scholar
5 National Clinical Assessment Service. Analysis of the First Four Years' Referral Data. National Patient Safety Agency, 2006.Google Scholar
6 National Clinical Assessment Service. NCAS Casework. The First Eight Years. National Patient Safety Agency, 2009.Google Scholar
7 Donaldson, LJ. Doctors with problems in an NHS workforce. BMJ 1994; 308: 1277–82.Google Scholar
8 Morrison, J, Morrison, T. Psychiatrists disciplined by a State Medical Board. Am J Psychiatry 2001; 158: 474–8.CrossRefGoogle ScholarPubMed
9 Firth-Cozens, J. Improving the health of psychiatrists. Adv Psychiatr Treat 2007; 13: 161–8.Google Scholar
10 Boydell, TH. Identification of Training Needs. British Association of Commercial and Industrial Education, 1983.Google Scholar
11 Buckley, R, Caple, J. The Theory and Practice of Training, 6th edn. Kogan Page, 2004.Google Scholar
12 Palmer, R. The identification of learning needs. In Human Resource Development, 2nd edn (ed Wilson, JP): 7788. Kogan Page, 2005.Google Scholar
13 Department of Health. Modernising Medical Careers. The Next Steps. Department of Health, 2004.Google Scholar
14 General Medical Council. Good Medical Practice. GMC, 2006.Google Scholar
15 Postgraduate Medical Education and Training Board. Standards for Curriculum Development: Background Paper. PMETB, 2004.Google Scholar
16 Department of Health. Safety First. Department of Health, 2006.Google Scholar
17 National Clinical Assessment Service. Data Collection and Release Policy. NCAS, 2004.Google Scholar
18 National Research Ethics Service. Definition of Research. National Patient Safety Agency, 2008.Google Scholar
19 Miller, GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 9: 63–7.Google Scholar
Figure 0

Table 1 Baseline characteristics

Figure 1

Fig 1 Distribution of performance concerns across the seven Good Medical Practice domains.

Figure 2

Table 2 Good Medical Practice performance profile for psychiatrists

Submit a response

eLetters

No eLetters have been published for this article.