CASC candidate questionnaire
Candidates are invited to take part in a questionnaire upon completion of their stations as a part of ongoing evaluation of exam standards, whilst giving them the opportunity to assess their perceptions and raise concerns.
Key findings are presented below:
- The majority of candidates strongly agreed or agreed that content of scenarios were realistic representations of those encountered in clinical practice
- The majority of candidates strongly agreed or agreed that the performance of role players was representative of patients encountered in clinical practice
- The majority of candidates strongly agreed or agreed that the CASC is testing clinical skills and knowledge essential for any competent Psychiatrist
- The majority of candidates strongly agreed or agreed that the CASC examination overall provides a fair assessment of the skills required in Psychiatry
Specific areas of concerns identified by candidates are addressed below:
- Insufficient time in stations
- Improve concept of time during stations
Longer stations have been piloted and considered extensively. They have been shown to have no beneficial impact on scoring or pass rates.
Whilst candidates may be keen to have one-hour stations, not only is there no evidence that this would improve the quality of the exam, but also would require the exam to be conducted over a period of time exceeding a day as well as two examiners per stations, neither of which we have the resources for.
The current station design requires candidates to be focused in their approach and promotes an ordered and organised style of managing the station.
Following candidate feedback, the reading time for instructions was reduced from 2 minutes to 90 seconds, as it was felt this was too long and served to increase anxiety in between stations. We are however considering some stations with longer reading times in the future.
- Can the number of stations be increased/decreased?
- CASC should include a wider range of station types
- Can the stations be video recorded?
Whilst some candidates will prefer fewer stations, some will prefer more of them, covering a wider range of assessment areas. The current 16 station design is providing optimum exam reliability.
We are currently reviewing the use of more management scenario stations and the use of active examiners within stations.
All examiners are trained and assessed as a part of the recruitment process, and invested in the standard setting for each station they examine.
The design of the CASC is such that each examiner will be a candidates first and another candidates last. Primacy and recency effects, fatigue and familiarity are balanced out throughout the day.
Should candidates have any concerns about their examiner, this should be reported immediately so that it can be investigated.
- Exam should use real patients instead of role players
- Role players across stations aren’t consistent
- Role players are too loud/too quiet
The use of role players rather than real patients increases the reliability of the exam. The use of patients would mean the stations were not standardised, and the candidate experience would therefore be too variable.
The role players are currently given guidance on emotional tone so they all portray the same standard, as agreed upon in the standard setting exercise. Again, each role player will be a candidate’s first and another candidate’s last, allowing for any effects of this to be balanced out throughout the day.
Role players are also given guidance to react to candidate’s questions, and thus provide information when prompted to do so.
If candidates feel their role players could not be understood, or were too loud, they can reasonably ask them to repeat themselves or speak up. Candidates are not penalised for such actions.
- Consider reducing the break times
- Consider reducing the cost
- Consider more sittings
- Inconvenience of Sheffield venue
The day is planned with tight examiner breaks, which cannot be reduced further. Candidate breaks allow for more sessions across the day.
Unfortunately, we're not in a position to reduce the cost as it is currently cost-neutral.
We are currently considering increasing the number of sittings for the written papers.
The feedback provided is in line with the Academy of Medical Colleges guidance. It is designed to help identify areas of weaknesses throughout the exam as a whole, and is not station specific.
We are unable to provide feedback on specific stations as this would compromise the integrity of the exam, and would be unfair to candidates not in receipt of such insight. CASC Examiners are not able to provide further qualitative feedback due to time constraints on the day.
Candidates are of course able to feed back to the exams team and the College at any opportunity, including the CASC candidate questionnaires now being administered annually.
- Exam is too stressful to be useful
- Exam is too artificial, and dependent on acting skills
- Exam is too easy for UK Graduates and too difficult for Overseas graduates
- Exam is too dependent on UK culture and communication
The exam is not designed to be unduly stressful, but to assess skills. Good candidates will portray skills effectively, as in any examination.
Candidates are expected to display skills in a simulated environment, which may feel artificial, but provides the most controlled and standardised environment in which to assess candidates fairly and reliably. Role players are instructed to allow for the development of rapport, and to allow the consultation to be as realistic as can be possible in an exam environment.
Unfortunately, one of the reasons for the perceived difficulty for overseas graduates is due to training, and needs to be addressed by training providers. The College is assisting this process through several initiatives.
Candidates are assessed on their communication and clinical skills specific to each station. We are currently looking at ways to improve the assessment of communication, whilst acknowledging that it is an essential element of psychiatric clinical practice. The MRCPsych is of course a UK exam.
All candidates need to face same communication challenges, and thus be assessed against the same standards.