Trainees
Assessors
Technical
Back to top
Why have workplace-based assessments?
The new competency based curriculum
approved by PMETB focuses achievement in medical training on
attainment of specific competencies. It allows the performance of
the doctor in the workplace to be a more robust measure of outcome.
A variety of workplace-based assessments (WPBA) have been developed
to allow trainees' to demonstrate the required competencies.
Back to top
What are they?
Several WPBA tools have been developed for use in
psychiatry:
- Assessment of Clinical Expertise (ACE)
The assessor observes a whole new
patient encounter in order to be able to assess the trainee's
ability to take a full history and mental state examination and
arrive at a diagnosis and management plan.
- Mini-Assessed Clinical Encounter
(mini-ACE)
The assessor observes part of a
patient interaction, for example history taking or negotiating a
treatment plan, and rates the trainee's performance.
- Case-based Discussion (CbD)
The trainee selects two sets of
notes of patients they have recently seen and the assessor picks
one to discuss. The discussion will allow demonstration of clinical
decision-making and the application of clinical knowledge.
This tool can be used when trainees
give clinical presentations and involves assessment of domains such
as presentation skills and interpretation of evidence.
- Journal Club Presentation (JCP)
This can be used when trainees present a journal article and
covers domains such as analysis and critique and answering
questions.
- Directly Observed Procedural Skills
(DOPS)
This has more limited use in
psychiatry compared to other areas of medicine but can used in
situations such as administering ECT.
- Mini-Peer Assessment Tool (mini-PAT)
It allows co-workers to assess the
trainee's attitudes and behaviours and ability to work well with
colleagues.
- Assessment of Teaching (AoT)
This is a new tool that is being
developed after feedback from the pilot programme. It allows an
assessment to be made of teaching skills and may relate to a
lecture, tutorial or small group teaching session that a trainee
leads.
Back to top
Trainees
How many WPBA do
I need to do each year?
From January 2009, participation in
WPBA will no longer per se be a requirement for everyone entering
the MRCPsych Examination. Candidates who are in PMETB approved
training posts will have to supply evidence that they have
successfully completed the relevant ARCP’s, which will depend, in
part, on successful completion of WPBAs. The learning needs of
individual trainees will determine which competencies they need to
be assessed on and the number of assessments that need to be
performed. The College Assessment Programme’s eligibility structure
should be regarded as a minimum number. The number of WPBA’s
stipulated in the Assessment Programme for CT1 and ST2 and ST3 is
as follows:
|
WPBA
|
Minimum number required per
year
|
|
ACE
|
2 in CT 1, 3 in CT 2 and 3 in CT
3
|
|
mini-ACE
|
4
|
|
CbD
|
4
|
|
mini-PAT
|
2
|
|
WPBA
|
Recommended number per
year
|
|
CP
|
1
|
|
JCP
|
1
|
|
DOPS
|
As the opportunity arises
|
|
AoT
|
A the opportunity arises
|
The guidelines in the above tables
should be regarded as the minimum number of WPBAs. Many trainees
will need to do more, for example, to provide additional evidence
of achievement in a particular area of professional practice.
There will be five rounds of
mini-PAT a year from which you can select two rounds to be entered
for.
Back to top
What about ST 4 - 6?
For trainees at the level of ST4
and above a new tool, Direct Observation of non-Clinical
Skills (DOCS), is currently being piloted to assess higher
level performance and competencies such as leadership skills. The
existing WPBA tools should be used to guide educational development
and as evidence for your ARCP. You should complete two rounds of
mini-PAT a year and undertake one WPBA a month.
Back to top
How will they be used to monitor my
progress?
The new The Guide to Postgraduate
Specialty Training in the UK (Gold Guide) outlines the Annual
Review of Competence Progression (ARCP). This will be an annual
review of a trainee's progress and will include consideration of
WPBA, examinations obtained and other evidence from a trainee's
portfolio. The Educational Supervisor's Report and the College
Tutor's Report will be submitted for consideration by the ARCP. It
is the ARCP that validates the achievement of competence after
considering the presented portfolio information. It is important to
note that as the ARCP considers a range of evidence it is possible
for trainees to do very well in their WPBAs but still at their ARCP
be considered to require further development either with extra time
or within the time scheduled.
Back to top
How do I arrange to do them?
It is the trainee's responsibility
to arrange the appropriate WPBA. You will need to plan what WPBA
you need to complete in each placement and then discuss them with
your educational supervisor at the start of the post. You will then
need to identify an appropriate assessor to complete the WPBA with
you. There is further practical guidance for trainees and assessors
in the guidance for each tool which can be found on the WPBA page
of the College website.
Back to top
Who is my educational supervisor?
The structures for educational
supervisors will differ locally. It will usually be the consultant
you are working for. In some places you may have one educational
supervisor for the whole year, despite working for different
consultants. In other places your educational supervisor will
change every time your consultant changes. Please check what your
local arrangements are. You must enter the details of your
educational supervisor online so that they receive your assessment
feedback and can discuss it with you. You must update your online
account when your educational supervisor changes.
Back to top
Who can assess me?
ST1-3 trainees can be assessed by:
- ST4-6 trainees
- SpRs
- Associate Specialists
- Consultants
- Band 7 nurses or psychologists or the equivalent
ST4-6 trainees can be assessed by:
- Consultants
- Band 8 nurses or psychologists or the equivalent
Your educational supervisor should
complete some of your assessments but other assessors should also
be used to ensure triangulation of information.
In the future some WPBA will be
conducted by a College approved assessor in order for them to
contribute towards the final mark of the CASC.
For the mini-PAT you should
nominate between 8-12 colleagues with whom you currently work.
These should include:
- At least 2 senior medical staff (ST 1-3: 1 consultant and 1 SpR
or ST 4-6 would suffice, ST 4-6: 2 consultants required)
- At least 2 non-medical clinical staff (e.g. nurses, social
workers, occupational therapists and psychologists)
- Not more than 2 trainees at your level (ST 1-3 level or ST 4-6
level)
- You may include administrative staff
- You may not include patients
Back to top
When will I get feedback?
You will receive mini-PAT feedback
a week after the assessors' deadline. For the other assessments,
you should receive immediate feedback from your assessor and will
receive collated feedback for all assessments in June in time for
educational supervision meetings prior to the Annual Review of
Competence Progression (ARCP).
Back to top
Who has access to my assessments and
feedback?
Your current educational supervisor
will have access to your WPBA for the last year. Your College tutor
will have access to all the WPBA you have ever completed to allow
them an overview of your training and progress. Depending on the
structure of your training system locally others may have access to
your assessments, such as your Training Programme Director or your
Director for Medical Education.
Back to top
What if I do badly?
WPBA are intended to provide
valuable feedback on progress during training. As such it is likely
that your feedback will highlight both good points and areas for
development. If a trainee does not meet the desired standard in one
area then this will allow them to plan ways to address this
training need with their educational supervisor. Further WPBA can
then be carried out to demonstrate that they have achieved the
required competency. However on the rare occasion that serious
concerns are raised where patient safety is an issue then your
educational supervisor or assessor is obliged to raise these
through the appropriate channels as they would with any other
formal or informal interaction concerning patient safety.
Back to top
Why do I have to nominate 8-12 assessors for the
mini-PAT?
Research suggests that feedback
from as few as 4 assessors can be enough to represent an accurate
picture of performance for the majority of competent doctors. We
therefore recommend you nominate between 8 and 12 assessors to
ensure that even a lower response rate achieves this level.
Additional assessors simply further assure reliable and valid
feedback. You will still receive feedback if less than 4 assessors
respond but this will be accompanied with a comment to state this
feedback is of low reliability and validity. We impose deadlines
for trainees to nominate assessors to ensure they have enough time
to complete and submit the assessment and once assessors have been
nominated for an assessment they receive regular reminders via
email. Incorrect assessor email addresses are highlighted to the
trainee immediately in order for them to correct any mistakes.
Back to top
I have missed the deadline for mini-PAT, what can I
do?
We impose deadlines to ensure that
assessors have enough time to complete the assessment and cannot
extend these as it will result in delays in other trainees
receiving their feedback.
Back to top
Can you tell me which of my assessors have submitted my
mini-PAT form?
We cannot provide information on
which assessors have or have not completed and submitted a
multi-source feedback assessment, only the total number of
assessors who have responded.
Back to top
One of my
assessors for mini-PAT has put comments I disagree with. Can they
be removed and can I find out who they were?
Comments cannot be removed from
your record. The identity of any assessor will not be
revealed to a participant. If you wish to discuss the comments on
your feedback, please do so with your educational supervisor.
Back to top
What about portfolios?
Electronic list of competencies (formerly, 'Portfolio of
Progress and Attainment')
Each phase of training (ST1; 2/3;
4/5; and 6) has a designated List of Competencies: Log of
Progress and Attainment. You can use the boxes in the List of
Competencies to describe the evidence you wish to use to
demonstrate your achievement of that competence. The evidence may
consist of one of the WPBA tools but may also include other
evidence such as reference to supervision notes or to reflective
notes. You can print the electronic list of competencies and fill
it in by hand, but it is better to complete it electronically and
print it out annually. Then you can add it to your paper
portfolio prior to your ARCP. This means that for stages of
training that span two years (e.g. ST2-3) you should have a half
completed list of competencies printed out at the end of ST2 and a
completed one at the end of ST3.
Paper portfolio
This replaces the previous College
logbook. It allows you to collate other evidence in addition to
your WPBA. A portfolio framework
can be found here.
Back to top
Can I print a copy of my assessments for my
portfolio?
You can print off any assessments
completed online by your assessors. To do this, log into your
account, click on assessments and select the assessment you wish to
print. When you click to view that completed assessment there will
be an option to print the completed form.
Assessors
What is the
role of the educational supervisor?
As with many aspects of the
trainee's learning experience, the success of WPBA depends very
much on the support of the educational supervisor. Your role is to
advise and encourage your trainee to develop their competencies and
to demonstrate this through their participating in regular
assessments, but it is neither desirable nor feasible that you
conduct the majority of them for your trainee. Indeed, by being
assessed by different assessors, the psychometric properties of the
assessment tools are improved. The Educational Supervisor's Guide
to WPBA contains all the information that you will need to help
your trainee through the assessment process. The Guide to
Postgraduate Specialty Training in the UK (Gold Guide) sets out the
duties of educational supervisors to conduct the educational and
workplace based (NHS) appraisal of trainees and to collate the
structured report that summarises the assessment evidence for the
trainee's ARCP.
Back to top
How do I register as an assessor?
If you were registered with HcAT
then you will not need to re-register and the username and password
are the same.
It is possible to register at:
https://training.rcpsych.ac.uk/register However,
it is no longer mandatory to register to complete an
assessment.
Back to top
If I state concerns about a trainee's fitness to
practice will my identity be revealed and will I be contacted
regarding this?
When producing feedback, comments
made by assessors are reproduced verbatim and anonymised so they
are not attributable to any one assessor. As part of our quality
assurance processes all anonymised health and probity concerns are
reported to the Royal College of Psychiatrists, which will be
passed to the relevant Deanery and cascaded as necessary. If we are
asked by the Royal College of Psychiatrists your comments will be
de-anonymised in order that you can be potentially be approached by
the Deanery or employing Trust for further details. Your identity
will never be revealed by us to the participant about whom you made
the comments.
Back to top
I am not clinically trained, can I still be an
assessor?
For the mini-PAT we recognize the
valid input from administrative staff who play an important role
and who are able to provide valuable insight from a different
perspective. Any questions that you cannot answer can be skipped by
clicking on the unable to comment (U/C) button.
Back to top
Technical
Who should I contact I have any
problems/queries?
If you have any technical problems
or questions regarding the assessments and the assessment
processes, please contact support@training.rcpsych.ac.uk
When e-mailing for help could you
please ensure that you include as much information as possible
(i.e. current training year, full name, trainee or assessor, type
of assessment, etc).
There is also a telephone number
which is available between 9am and 5pm on weekdays: 0844 800
6516.
Back to top
Can I use a
personal email address instead of a work email
address?
Any email address is permitted by
the system but you are encouraged not to use free email accounts
such as hotmail as they tend to frequently direct emails from us
into spam folders, particularly automated emails generated to
remind you of assessments that need to be completed.
Back to top
When I try to log in I am told my username or password
are incorrect.
Please ensure that you do not
include any spaces before or after your username or password if you
copy and paste these. Usernames are not case sensitive but
passwords are case sensitive. Please also take care when entering
passwords as these are a mix of alphanumeric characters.
Back to top
What if I've entered an incorrect email address for an
assessor?
If you have entered an incorrect
email address and it bounces back you will be informed of this via
email with instructions on how to correct it. If an e-mail address
continues to bounce back and you are sure that the address is
correct, please check spelling and punctuation carefully.
Back to top
Back to Trainees' Home Page
Page last updated on 19th September
2008