Hostname: page-component-7c8c6479df-ph5wq Total loading time: 0 Render date: 2024-03-28T08:06:21.447Z Has data issue: false hasContentIssue false

Inviting a statistician to join an evidence-based journal club

Published online by Cambridge University Press:  02 January 2018

G. Swift
Affiliation:
Department of Psychiatry, Rawnsley Building, Manchester Royal Infirmary, Oxford Road, Manchester M13 9WL
F. Crotty
Affiliation:
Department of Adult Psychiatry, Mater Misericordiae Hospital, Dublin
M. Moran
Affiliation:
Department of Adult Psychiatry, Mater Misericordiae Hospital, Dublin
C. M. McDonough
Affiliation:
Department of Adult Psychiatry, Mater Misericordiae Hospital, Dublin
A. Moran
Affiliation:
Department of Adult Psychiatry, Mater Misericordiae Hospital, Dublin
P. R. Casey
Affiliation:
Department of Adult Psychiatry, Mater Misericordiae Hospital, Dublin
Rights & Permissions [Opens in a new window]

Abstract

AIMS AND METHOD

In response to frequent cancellations, we recently changed the format of our journal club. We invited a statistician to attend and each meeting focused specifically on critical appraisal and statistical analysis. Active learning principles were used where every participant critically appraised an aspect of the paper presented at each meeting. The aim of this paper was to evaluate the impact of the new format on attendance and satisfaction levels. Attendance and satisfaction levels were measured in the 3 months before and the 3 months after the new format was introduced.

RESULTS

Average attendance increased from 10 to 15, with two psychologists and two trainees who did not work in the hospital starting to attend regularly. Satisfaction levels also increased significantly with respect to academic interest, enjoyment, usefulness and overall satisfaction.

CONCLUSION

Combining statistical expertise with principles of active learning has led to increased satisfaction levels among participants in our journal club.

Type
Education & training
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © 2001. The Royal College of Psychiatrists

We were concerned about our journal club. Traditionally, based as it was in the department of psychiatry in a large teaching hospital, the weekly meetings had been successful in terms of longevity and attendance. In contrast, over the previous 6 months we had experienced last minute cancellations, presentations that were poorly prepared and low levels of attendance. All this despite the fact that we fulfilled many of the criteria recommended by Sidorov (Reference Sidorov1995): a small training programme, available food, mandatory attendance and an active research programme. At a focus group held to identify the causes of the recent decline in enthusiasm, trainees attributed the problem to the introduction of the new clinical appraisal paper in the Membership exam. The traditional journal club format, where an article was presented and then discussed, was seen as no longer relevant to trainees' needs. When an attempt was made to introduce a more evidence-based format as described by Gilbody (Reference Gilbody1996) pre-Membership trainees did not feel confident in presenting. Moreover, more senior trainees who had taken the Membership exam before the introduction of the new paper also lacked confidence in chairing the meetings. Taking into account these concerns about the journal club, the focus group generated several suggestions to develop the journal club. We have now implemented these suggestions and the aim of this paper is to evaluate the impact of the new format on attendance and satisfaction levels.

Method

New format

Trainees expressed a wish for the journal club to be made more relevant to their needs, particularly with regard to statistics and the critical appraisal paper of the Membership examination, and the following changes were therefore made:

  1. (a) A statistician who has a background in psychology was asked to attend the meetings on a weekly basis.

  2. (b) A list of papers that illustrates all the statistical terms and methods covered in the Membership curriculum was identified.

  3. (c) The nominated presenter reads the paper in advance and decides on a list of questions similar to those asked in the critical appraisal paper.

  4. (d) At the meeting, those attending are divided into small groups, given the paper to read and allocated several questions. Each group has 20 minutes to decide on answers to the questions, and they then present their conclusions to the large group.

  5. (e) The presenter chairs the discussion that ensues and the statistician is used as a resource to help with areas of difficulty.

  6. (f) At the end of the meeting, the presenter gives a brief résumé of the statistical questions that were mentioned in the paper and distributes a short (less than a page) definition of any terms considered relevant.

Assessment of impact of new format

We studied attendance and satisfaction in the last 3 months of the traditional journal club and the first 3 months of the new format using the following measures:

  1. (a) the number of meetings cancelled

  2. (b) the average number of people attending the meeting

  3. (c) a 4-point Likert scale (1=no interest/satisfaction to 4=highly interesting/very satisfied) to examine clinical interest, academic interest, enjoyment and overall satisfaction.

T-tests were used to look for significant differences between the mean scores in each of the categories before and after the new format was introduced. Finally, participants were asked for any additional comments.

Results

Prior to the intervention, the core group attending the journal club consisted of 10 doctors: three consultants, three post-Membership trainees involved in teaching and research and four pre-Membership trainees. Two members of the psychology department had been invited to the old format journal club but after one meeting had decided that it was not relevant to their work. In a 3-month period, only four of 14 meetings were actually held, the rest being cancelled owing to trainees not being prepared or failing to attend. Table 1 shows the mean satisfaction levels.

Table 1. Mean satisfaction levels before and after introduction of a new format for journal clubs

95% Cl
Before or after intervention Mean d.f. Significance (2 tailed) Lower Upper n
Clinical interest Before 2.6 21 0.142 -1.22 -0.19 9
After 3.1 15 14
Academic interest Before 2.4 22 0.011 -1.9 -0.3 10
After 3.5 12 14
Enjoyable Before 2.4 22 0.037 -1.6 -6.11 10
After 3.3 11 14
Usefulness Before 2.4 22 0.005 -2.2 -0.5 10
After 3.8 11 14
Overall satisfaction Before 2.2 22 0.008 -2.7 -0.4 10
After 3.5 12 14

Three months after the intervention, the number of people attending the meetings had increased to 15. One new lecturer started working in the department. The two psychologists now attend on a regular basis. Two pre-Membership trainees working in another hospital heard about our new format and felt that the additional statistical input and training in critical appraisal techniques was sufficiently helpful for their exam preparation to travel to the department. All 14 of the scheduled meetings were held. As can be seen in Table 1, satisfaction levels improved in all areas, with statistically significant increases in academic interest, enjoyment, usefulness and overall satisfaction. Comments offered by participants included acknowledgement of a greater sense of ownership of the journal club by trainees. Several people noted that where the meeting would have been cancelled in the past if no consultants had been able to attend, now there was enough confidence to carry on regardless. Trainees also said that they found it easier to ask questions of someone who was outside the medical hierarchy, particularly when these questions might reveal their lack of knowledge in statistics. Finally, there were several comments about the benefits of regular scheduled contact with a statistician, when informal advice could be sought regarding on-going research projects.

Discussion

This study examines the impact on our journal club of a new format that placed increased emphasis on critical appraisal and used adult learning principles. We found a significant increase in satisfaction levels and improved attendance rates.

Some limitations to this study are obvious. The length of follow-up is short, merely 3 months. It is likely that any change that is perceived as emanating from the group will be associated with a sense of dynamism and enthusiasm, which is then reflected in higher morale. It is interesting that in the literature to date, although many different formats for journal clubs have been studied, every intervention has been reported as having positive effects on attendance and satisfaction.

A second limitation is that we have no objective measurements of improvements in statistical knowledge or ability to critically appraise a piece of research. Previous studies in this field have found inconsistent evidence of objective improvement in critical appraisal skills even where trainees have reported satisfaction (Reference SeeligSeelig et al, 1991; Reference Sandifer, Lo and CromptonSandifer et al, 1996; Reference Norman and ShannonNorman & Shannon, 1998; Reference Fu, Hodges and RegehrFu et al, 1999). Most of these studies were based in the US or Canada where, as far as we are aware, trainees are not formally assessed in critical appraisal as part of their qualification as a specialist. It is possible that trainees in the UK and Ireland may be more realistic in their evaluation of their own abilities given that they will be examined in critical appraisal skills. Therefore, their expression of satisfaction, and particularly the attendance at the journal club of trainees who are not obliged to attend, may be given more weight. The advantages of our new format are several. First, identifying specific learning objectives (namely statistical knowledge and critical appraisal ability) that are seen as relevant to participants' needs has created an improved sense of ownership. Both Sidorov (Reference Sidorov1995) and Linzer et al (Reference Linzer, Delong and Hupart1987) suggested that a balance needed to be struck between allowing trainees to feel a sense of self-determination and involving faculty in teaching. In a similar vein, Rowlands and Geddes (Reference Rowlands and Geddes1993) pointed out the importance of active involvement in a journal club to the morale of junior doctors. We feel our format encourages trainees to take responsibility for the journal club while ensuring that expertise is readily available.

Second, dividing participants into small groups and requiring them to devise answers and to debate these answers with other participants leads to active learner involvement. Like Elnicki et al (Reference Elnicki, Halperin and Shockor1999), we were concerned about the risk of embarrassing individuals who might feel self-conscious about their lack of knowledge in an area. In general we find that this is not an issue once there are at least three participants in each group. Moreover, the involvement of all those present reduces the workload for the presenter: in general, participants have found that the preparation time is less than for the traditional format.

Finally, the presence of statisticians at each meeting has created an environment where consultants, psychologists and trainees feel freer to question each other in a scientific manner. Knowing that a statistician is there to answer and elaborate on difficult points encourages all those present to focus on the statistical strengths and weaknesses of a study.

We are not aware of other journal clubs that have sought regular statistical support. Other groups seeking outside expertise have mainly looked to librarians for assistance in conducting adequate literature searches (Reference ScherrerScherrer, 1999). Clearly the determining factor here is the learning objectives set for the meetings, but we feel that other groups might, like ourselves, identify statistical analysis as a particularly challenging area.

Conclusion

Reviews of journal clubs (Reference Jouriles, Cordell and MartinJouriles et al, 1996; Alguire, 1998) suggest that there is no one ideal format: rather, different groups need to tailor a journal club according to their needs. In our department, trainees and researchers expressed an interest in meetings that focused primarily on critical appraisal. Combining expert statistical advice with active learning has produced a format associated with improved participant satisfaction. To maintain enthusiasm we hope to develop this format further, allowing participants to move from pure critical appraisal to a broader evidence-based medicine approach, formulating a clinical question and searching for the most relevant literature.

Acknowledgements

Thanks to SmithKlineBeecham and to Pfizer for their sponsorship of the journal club.

References

Alguire, P. C. (1988) A review of journal clubs in postgraduate medical education. Journal of General Internal Medicine, 13, 347353.Google Scholar
Elnicki, D. M., Halperin, A. K., Shockor, W. T., et al (1999) Multidisciplinary evidence-based medicine journal clubs: curriculum design and participants' reactions. The American Journal of the Medical Sciences, 317, 243246.Google Scholar
Fu, C. H. Y., Hodges, B., Regehr, G., et al (1999) Is a journal club effective for teaching critical appraisal skills? A controlled trial with residents in psychiatry. Academic Psychiatry, 23, 205209.Google Scholar
Gilbody, S. (1996) Evidence-based medicine. An improved format for journal clubs. Psychiatric Bulletin, 20, 673675.Google Scholar
Jouriles, N. J., Cordell, W. H., Martin, D. R., et al (1996) Emergency medicine journal clubs. Academic Emergency Medicine, 3, 872878.Google Scholar
Linzer, M., Delong, E. R. & Hupart, K. H. (1987) A comparison of two formats for teaching critical reading skills in a medical journal club. Journal of Medical Education, 62, 690692.Google Scholar
Norman, G. R. & Shannon, S. I. (1998) Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal. Canadian Medical Association Journal, 158, 511.Google Scholar
Rowlands, P. & Geddes, J. (1993) Trainees' forum. Journal clubs. Psychiatric Bulletin, 17, 363364.CrossRefGoogle Scholar
Sandifer, Q. D., Lo, S. V. & Crompton, G. (1996) Evaluation of a journal club as a forum to practise critical appraisal skills. Journal of the Royal College of Physicians of London, 30, 520522.Google Scholar
Scherrer, C. S. (1999) The evolving role of the librarian in evidence-based medicine. Bulletin of the Medical Library Association, 87, 322328.Google Scholar
Seelig, C. B. (1991) Affecting residents' literature reading, attitudes, behaviours and knowledge through a journal club intervention. Journal of General Internal Medicine, 6, 330334.Google Scholar
Sidorov, J. (1995) How are internal medicine residency journal clubs organised, and what makes them successful? Archives of Internal Medicine, 155, 11931197.Google Scholar
Figure 0

Table 1. Mean satisfaction levels before and after introduction of a new format for journal clubs

Submit a response

eLetters

No eLetters have been published for this article.