Contents:
- Editorial
- WPBAs – one year on
- Assessments online
- National Trainees’ Clinical Experience
Survey
- Speciality selection in the future
- New exam eligibility criteria
- Review of the Trainees’ Day at the Annual
Meeting
- Guide to Postgraduate Training in
Psychiatry
- PTC newsletter editorial team
Editorial
Welcome to the third e-newsletter of the
Psychiatric Trainees’ Committee (PTC) of the Royal College of
Psychiatrists.
Further information about
training issues can be found on our website: www.rcpsych.ac.uk/training/traineessection.aspx
Please feel free to email us at ptc@rcpsych.ac.uk about any training
issues – we would also appreciate any feedback about this
newsletter so that we can improve subsequent editions.
Back to top
WPBAs – one year on
Dr Andy Brittlebank, Associate Dean
for Postgraduate Education
The
introduction of the new Psychiatry Assessment System in August 2007
means that over the course of six years of training a Specialty
Registrar in Psychiatry can expect to receive around 90 hours
observation and structured feedback from senior colleagues. This
contrasts with the training that my contemporaries and I received.
For most of us the only time that senior colleagues saw us
interacting with a patient was during the ten minutes’ observed
interview in the old Part 2 Examination.
What have we
learned over the last two years while we have been piloting and
implementing Workplace-based Assessments (WPBAs) in psychiatry?
I think the answer to this falls
under three headings: -
- The tension between formative and summative assessment
- The potential WPBAs have for being reliable and valid forms of
assessment
- The gaps in our assessment system
Formative versus
summative assessment
Doctors have
been socialized into a pass/fail examination culture. This is in
contrast with our approach to the clinical assessment of patients
where we are used to a painstaking process of evidence collection
and interpretation from which we draw broad, often narrative-based,
formulations that lead to holistic management plans. I suggest that
we need to import this methodology to our approach to the
assessment of professionalism. Rather than thinking of an item of
WPBA as representing a pass or a fail, we should rather reflect on
what information this assessment conveys and how this can be used
to further improve the doctor’s performance.
We have seen
from the portfolios that trainees submit for the Annual Review of
Competence Progression (ARCP), and that consultants submit to their
annual appraisal, that many doctors are explicitly and
conscientiously reflecting on evidence about their performance.
These doctors are engaged in a continuous process of quality
improvement. I know, however, that in many situations and for all
sorts of reasons, people have approached assessment as a chore to
be endured or as a bureaucratic exercise to have its boxes ticked.
This is sad, because we will lose opportunities to enhance the
professionalism of our discipline.
The reliability and
validity of assessments
We have learned
from our piloting of WPBAs in psychiatry that they have robust
psychometric properties, especially where a trainee has been
assessed by a number of different consultant level assessors. So
for example, if a trainee has had Case-based Discussion (CbD)
assessments performed by four different consultant assessors, in
different clinical situations, this will produce a highly reliable
and valid picture of the doctor’s performance. I am most grateful
to the trainees and their assessors who took part in the pilot
project and I want to assure you that the findings from the pilot
studies will be used to inform the development of the assessment
system.
The gaps
In designing
the Assessment System our initial priority was to put a system in
place that primarily addressed patient safety especially around the
supervision of core (ST1-3) trainees. I readily acknowledge that we
have paid less attention to the assessment needs of higher (ST4-6)
trainees. It is time to address this shortcoming.
The first step
we are taking is to pilot a new WPBA tool, the Direct Observation
of non-Clinical Skills (DOCS). The DOCS has been designed to
provide feedback on a trainee’s performance of higher-level skills,
such as chairing meetings, providing supervision, giving evidence
at formal hearings and giving psychiatric advice to external
agencies. We are working with ten sites around the country and are
currently training assessors and trainees in the use of the tool
with the aim of starting to collect data after the August
changeover.
The future
One thing that
I think we have all learned in the last couple of years is that
nothing stands still. I believe that just as we have a professional
duty to continually seek to improve our performance, we also have a
duty to improve the performance of the systems around us. So we
should seek to develop the quality of our Assessment System. I am
grateful to the Psychiatric Trainees’ Committee and to individual
trainees for the comments and critical feedback they have given
and, I hope, will continue to give to help us in the enterprise of
continual improvement.
I would leave
you to consider an aphorism attributed to the composer Sir Walford
Davies; “we are not out to beat each other, but to pace each other
on the road to perfection”. Perfection is of course, a destination
at which we will never arrive.
Back to top
Assessments online
Dr Amit Malik, Training Policy
Advisor
Based on this year’s
experience of HcAT and extensive feedback from many trainees and
trainers, the College has decided to bring the online assessment
system in-house in order to make the process more trainee-focussed
and enable us to be more responsive to trainees’ and trainers’
concerns. The new system will be referred to as
Assessments online to avoid using any
more four letter acronyms.
The system is expected to
go live in August for the next training year. We are currently
piloting the new system with assessors and trainees so that we can
resolve any difficulties before we launch the system. The new
system will have the following salient advantages:
- The
usernames and passwords of all registered trainees and assessors
will be preserved. We are currently negotiating with HcAT for the
smooth transfer of this information to our system.
- The
assessments that trainees have completed over the last year will
also be preserved and we are in the process of obtaining these from
HcAT.
- We are
redesigning the website to make it easier and more intuitive to
navigate.
- The
assessor registration process and completing assessments will be
made much more user-friendly.
- College
Tutors, Training Programme Directors, Directors of Medical
Education and Heads of School will be given access to view their
trainees’ progress in the context of WPBA.
-
Trainees will be able to access and print off live collated reports
at any point during the training year.
- A
responsive system of accessing help with any technical or
procedural difficulties will be made available to all
users.
The new system will be
available at http://training.rcpsych.ac.uk
Trainees and assessors
will be able to access help with the system by e-mailing:
support@training.rcpsych.ac.uk
In order to allow the
most efficient and prompt response to the majority to queries,
e-mailing should be the first step for those with technical
difficulties. If it is not possible to address the query via
mail a phone number is available: 0844 800 6516.
For queries relating to
the WPBA themselves and training issues please contact:
cdrummond@rcpsych.ac.uk
We are grateful to all of
you for your patience and extensive feedback with regards to HcAT
over the last year. We also acknowledge that some of you have found
the process frustrating and cumbersome as a result of the
difficulties with HcAT. We do, however, believe that if implemented
properly, an online system is the most efficient way of collating
WPBA. We hope that in the future this will be linked to an
e-portfolio where these assessments will be recorded alongside a
trainee’s other experiences and evidences of competence
acquisition. We are working very hard at the College to develop an
efficient and user-friendly system and as always would be very keen
to receive your feedback so that we can continually make
improvements to it.
We will send out an email
when the system goes live with detailed information about how to
use it. Please continue to use the existing HcAT system for any
WPBA for now. If you have already registered for HcAT you will not
be required to re-register for Assessments
online.
We thank all trainees for their ongoing
cooperation and support.
Back to top
National Trainees’ Clinical Experience
Survey
The national training
survey will close on August 1st. However, we are still
short of the numbers that will be required to give the results good
validity.
Many thanks to all who
have completed the survey to date. To those of you who haven't
completed the survey could you please take the 10 minutes required
to complete the 24 questions. The survey can be found at:
www.nationaltrainingexperiencesurvey.co.uk
Back to top
Speciality selection in the future
The Medical Training
Application Service (MTAS) system attracted widespread criticism
from both trainees and trainers. It resulted in extremely
difficult situations for many trainees and their
families. As plans are made for the future of speciality
selection it is crucial that trainees are actively involved in the
discussions to help ensure that a similar fiasco is not
repeated. The PTC has produced a report considering the
options for future selection processes and outlining proposals that
are in the best interests of trainees.
It can be found by clicking
here (opens as a word file in a new window).
We recently presented this report to the College and a trainee
representative will now be included in the working group that has
been set up for speciality selection.
Back to top
New exam eligibility
criteria
The first diet of the new
Royal College of Psychiatry exams is now complete. Trainees had
raised concerns with the examination eligibility criteria and the
criteria have now been revised to take these concerns into account.
The new criteria will be effective from 1st January 2009
and the subspecialty areas which will be classed as adult
psychiatry are clearly specified. A recommended timeframe is given
for the written papers, which is similar to the previous criteria,
although all papers can be taken after 12 months post-foundation
training. The papers can now be taken in any order, although it is
not recommended to do so.
The CASC can be taken after 30 months of full
time equivalent training, of which a minimum of 24 months should
have been in psychiatry. Trainees need to have a minimum of 12
months of adult psychiatry experience and a minimum of 12 months in
two or three other psychiatric specialties. A pass in all three
written papers is required prior to sitting the CASC and a pass in
Paper 3 can be banked for 635 days from the day that the Paper 3
results are released. The other requirement for trainees in a
programme approved by PMETB is that they must have evidence of
successful completion of their Annual Review of Competence and
Progression documentation. Further guidance is awaited for flexible
trainees.
A copy of the full guidance is available by
clicking
here.
The written paper at a glance guides can be
viewed by clicking
here.
The CASC at a glance guides can be viewed by
clicking here.
Please note, the above files will all open in
a new window.
Back to top
Report of Trainees’ Day at the College AGM
2nd July 2008
Meinou Simmons, PTC
member
The Trainees’ Day 2008
presented an interesting and varied programme of debates and
informative lectures. The day was split into four sessions
described below:
Global challenges
facing psychiatric training: an international perspective
It was refreshing and
interesting to hear of trainees’ experiences from foreign
countries, and compare them to our UK experiences. The original
programme was altered slightly, but this did not detract from the
quality of the perspectives offered. We heard from Olivier
Andlauer, a delegate from France, where a ‘laissez faire’ approach
to postgraduate training (minus formal exams or assessments),
manages to produce the highest patient satisfaction in Europe! We
heard from the Czech Republic where psychiatric practice is still
predominantly based in Victorian asylums and trainees are paid
little more than a UK teenager’s pocket money. We were also
fortunate to have Abigail Donovan, from the US American Psychiatric
Association Board of Trustees present. The reports of difficulties
with US training with almost permanent on calls (with 80 hour weeks
being considered a luxury), and constant weekly battles with
untrained Insurance company staff really made UK trainees thank
their lucky stars for UK training and think twice about moaning
about our lot!
Debate: ‘This house
believes that there is no place for any Mental Health Act in modern
society’
Well attended by trainees
and consultants alike, this was an interesting and well argued
debate. Dr Anthony Zigmond and Dr Julian Beezhold presented an
argument supporting the motion based on humanitarian grounds. It
was Dr Zigmond’s view that an altered capacity act would be
sufficient and more humane, as used in several countries, and would
alter the balance of power in favour of patients. Opposing the
motion were Professor Tom Fahy and Dr Arun Chopra. They articulated
the reasons why a civilised society needs to retain its Mental
Health Act in order to maintain safety and retain professional
responsibility for patients who are at risk. They also explained
why in their view a Mental Capacity Act alone would not be
sufficient to maintain order. The vote was cast against the
motion.
Workplace based
assessments: a year on
The important aspects to this timely
and important topic were well articulated by those most informed
within the College. Dr Andrew Brittlebank, Dr Amit Malik and Dr
Clare Oakley helped to present the main advantages and difficulties
of this new compulsory part of trainees’ portfolios. They explained
that problems with the unwieldy HCAT-online system have led the
College to transfer across to their own in-house system from
August. Dr Brittlebank showed statistics of the pilot programmes
developed within the College. Of interest was the fact that the
degree of statistical validity of assessments improves
significantly with number of assessors: so the take home
message was for trainees to get as many different assessors as
possible. Consultants had mixed experiences of using these
tools, and although some felt they had been a positive move, some
felt the tools would not necessarily be useful in helping to
identify a failing trainee.
Debate: ‘This house
believes that the quality of psychiatric postgraduate education
will decline unless the College quality assurance visits are
reinstated’
In this debate we had the humorous
and unusual situation of the main supporter of the motion,
Professor Rob Howard, not actually supporting the motion as it
stood. Instead the new Dean, enthusiastically presented the case
for a hybrid model of College visits with PMETB input and
independent quality control. He wanted to revive all the good parts
of the College visits i.e. ‘looking in all the dark corners of all
training units’, however recognising that it is not possible to go
back to such visits without PMETB input. Dr Greg Lydall, also
supporting, expressed frustration of the ‘watered down’ less
rigorous regulation offered so far by PMETB. Opposing the motion
were Mrs Patricia le Rolland Director of Quality for PMETB, and Dr
Ross Overshott, new consultant and playwright. Mrs le Rolland
explained the time for in-house regulation has passed, and she
explained the importance of an independent regulator to work
alongside the College in carrying out its function. Dr Overshott
also eloquently spoke of the old cronyism of the old College
visits, where trainees were not always fully able to speak
independently due to fear of tarnished reputations. The conclusion
seemed to be to create a more harmonised model of regulation with
the independence offered by PMETB but with College input to push
for psychiatric expertise and increased rigour.
Back to top
Guide to Postgraduate Training in
Psychiatry
To ensure that trainees
and trainers have easy access to the information they need, the PTC
have taken a lead in developing a guide to the new training
system. This guide has now been published by the College
as Occasional Paper 65 and was distributed to all members with the
July journals. It provides an overview of postgraduate
training in psychiatry and sets out standards for this
training. If you have not received a copy it can be found
at:
www.rcpsych.ac.uk/files/pdfversion/OP65.pdf (opens
in a new window).
Back to top
PTC newsletter editorial team
Christian Hosker, Newsletter editor
Elena Baker-Glenn, Website editor
Ollie White, Chair
Clare Oakley, Vice Chair
Paul Henderson, Honorary Secretary
Back to top
Back to trainees' home page
Page last updated on 17th
July 2008