Chapter 1 General guidance
David Limb
Introduction#
The structured oral (viva) examinations are the second component of Section 2 of the Intercollegiate examinations, usually occurring over a two-day period after the clinical section, but for any individual candidate the four vivas will occur on the same day. It is perhaps worth puting the vivas into context:
between them the vivas contribute 48 of the 96 marking episodes inSection 2. The clinicals (intermediate and short cases) together make up the other 48 episodes, but in general it is more common for a poor mark in the vivas to be compensated for by a good mark in the clinicals than vice versa. Employing the training principles of a heptathlete, effort may be better spent on the weaker disciplines than becoming better at one’s strengths.
This chapter will review the overall marking structure for the exam and outline the contribution of the structured clinical orals to the overall result. The process will then be explained in detail so that you, the candidate, can understand why the exam has evolved into its current form (incidentally, one of the most reliable high-stakes professional examinations in the world). By understanding this process you will be in the best position to prepare yourself for assessment against the examination standards. These standards are not set to ensure examination income for colleges, or to impose a limit on the supply of qualified professionals. The standards are set to reassure the regulator (GMC), employers and, most importantly, patients that those being awarded a certificate of completion of training today are of the same high standard as those awarded it last year and the year before. The FRCS (Tr & Orthis one component of that assessment and if everyone presenting for the examination shows themselves to meet that standard, then every candidate will pass!
Overall structure of Section 2
To reach Section 2 of the Intercollegiate examination candidates must first pass Section 1. Section 1 is a computer-based test using ‘single best answer’, which over the past few years has evolved to focus principally on higher-order thinking. The large majority of factual, knowledge-based questions have been removed from the question bank. Therefore, to arrive at Section 2 you have already shown that you have a knowledge base and can apply that knowledge to solve problems posed in clinically relevant scenarios.
Section 2 moves us higher up the ladder of higher-order thinking: it enables professional behaviours to be observed while the application of knowledge to real clinical problems in a time-pr essured environment gives insight into how candidates might behave in independent clinical practice. Decisions have to be made on information elicited by the candidate and these have to be in the patients best interests.
There are two components to Section 2: the clinical examinations (usually taking place in a hospital facility on a Sunday) and the structured oral examinations on the following two days, in an examination hall, often the ballroom of a hotel. The clinical examination will involve two 15-minute intermediate cases, one upper limb/cervical spine and one lower limb/thoracolumbar spine, and two 15-minute short case examinations with the same upper/lo wer limb split and each with three cases for 5 minutes each.
This chapter focuses on the structured oral examinations and each candidate will undertake four such vivas, each 30 minutes in length. Together these broadly cover the curriculum and are themed thus:
Trauma (including spine)
Basic science
Adult and pathology (including spine)
Children and hands (including upper limb)
Note there are qualifications against some viva titles – this is to ensure that wide syllabus coverage is possible, and this is facilitated by each candidate being preceded at the viva table by a topic sheet indicating the specific questions they have been asked to that point. Thus, if supracondylar fracture is a topic in the trauma viva it will not reappear in the children’s viva. If spine has been omift ed from trauma and children’s vivas it is very likely to be asked in the adult and pathology viva.
Each viva is now quite rigidly structured – a 30-minute viva with two examiners will consist of
15 minutes with each examiner. Each examiner will ask on three topics for 5 minutes each (with a bell sounding to indicate each 5-minute interval). Each viva therefore involves six topics and each of these is marked independently by the two examiners, giving a total of 12 marking episodes for each of the four vivas. The practic alities of siting the viva will be described later.
Marking scheme#
InSection 2 of the FRCS (Tr & Orth) an examiner has only five choices of mark to award for each marking episode. A mark of 6 is a pass mark; 7 is a good pass and 8 a very good pass; 5 is a fail and 4 a bad fail. As noted above, the vivas carry a total of 48 marking episodes and that is matched by 48 marking episodes in the clinical section: 24 in the intermediate cases and 24 in the short cases. Altogether, therefore, there are
96 marking episodes and a score of 6 in every episode reaches the pass mark for the exam, which is therefore 576.
In the past a mark of 4 in any part of the clinical examination meant an automatic fail, no matter what marks were achieved in the vivas. This skewed examiner behaviour and now no such ‘killer’ mark exists. It is possible to compensate for a 4 in one marking episode by achieving an 8 in another episode, or indeed by two 7s in two separate marking episodes. The disaster of course would be topass 95 episodes with marks of 6 and therefore fail the exam because of a single score of 5. There is no discussion around the marks at the end: no vouching for candidates by examiners who know them is possible. Examiners award a mark at the end of each marking episode independently of their co-examiner and enter it in their tablet computer. The sum of 96 episodes determines the total mark and if this is 576 or above the candidate has passed. If it is 575 or below the candidate has failed.
The mark awarded is not simply a grading based on the examiners’ whim. There is a marking scheme which ascribes descriptors to levels of quality in response and this determines the mark that should be awarded. Although it is still up to the examiner to assess your performance and allocate the appropriate mark, the quality of response needed to achieve a 6, 7 or 8 is agreed at the examiner standard-seting meeting , which will be described later. Examiners are not allowed to confer before awarding their marks
(except to clarify matters of fact, such as might occur if the co-examiner mishears something but is not allowed to interrupt), and they should mark according to the standards agreed at the standard seting discussion. Therefore, marks do not vary significantly – although it is acceptable for the examiners to give different marks, only a difference of one markis accepted and examiners are not allowed to change their mark after allocating it. A discrepancy of two marks triggers an investigation, but fortunately this is rare.
Practicalities#
For examiners the day is split into three or four sessions with three to six vivas in each session. Candidates are examined in groups, which may therefore have vivas either side of a coffee break. Each group of candidates receives a briefing from the Chairman of the Board immediately before their block of vivas begins.
The examiners use the same batch of standardized questions for each session. Resist the temptation to find ways of discovering what others in your group have been asked – this could give you an unfair advantage and is unprofessional. The GMC would take a dim view of any attempt to gain such an advantage in the examination process and a GMC referral is not helpful in gaining access to a consultant post.
You will be led into the examination hall and accompanied to your table by a member of intercollegiate staff who will identify your table and indicate your candidate number to the examiners. The examiners will stand, greet you and check your candidate number. They will not know whether you are a trainee or out of training. They will not know if this is your first attempt or if you are a returning candidate. Your heart will be racing and your mouth dry, but the examiners will be aware of this. It is their job to find out how well you can perform, not to humiliate you, so expect a polite introduction, a check of your candidate number, an orientation to which viva you are about to sit and an outline of how the next
30 minutes will be spent (‘three questions of 5 minutes from myself with a bell between, followed by three more from my colleague’). The actual questioning doesn’t start until the first bell sounds, ensuring that all candidates receive the same time, particularly when the hall is long, and some candidates have further to walk to their examiners than others. Commonly the topic is introduced by asking you to look at an image or diagram on a tablet computer screen and describe what you see, which leads into the questioning.
There will follow 30 minutes that seem to rush by, punctuated by bells at 5-minute intervals. When a bell sounds the examiner moves on to the next question – you will not be interrupted if you are part-way through a sentence in response to the previous question, but the examiner will simply stop the line of inquiry related to the previous question and introduce the next question. After six questions, three with each examiner, the final bell will be met with a polite but swift termination of the viva and you will be invited to leave the hall with the other candidates. Outside the Intercollegiate staff will organize you in preparation for your next viva, or allow you to leave if you have come to the end.
Examiner behaviour#
Examiners are human beings and will naturally be different. However, they are trained to get the best from you and to minimize the chances that your performance in one component of the exam will affect it in another. There is a significant amount to consider in an examiner’s training course, and examiners then attend an exam as ‘examiners in training’, so what follows is a very brief outline of how that training should impact on you.
Apart from being polite and courteous, examiners can steer you through a viva question and give you opportunities to elicit responses that show that you have reached a certain level in the marking scheme.
In doing so you should find that most of the examiners’ responses are emotionally flat, encouraging you to impart more or steering you away from areas that do not gain marks. They should not give you the impression that you are performing very well (‘Excellent! Well done!’) or very badly, as this may influence your performance in subsequent questions and vivas. They should not harass you and co-examiners are trained to intervene appropriately if unacceptable examiner behaviour is witnessed. Of course, personalities will come through and you may hear beforehand of examiners who are reputed to be fierce
– it may interest you to know that the marking behaviour of examiners is very strictly observed and analysed and bears no relation to candidates’ perceptions.
Each viva will involve two examiners, each asking three questions. The examiner who is not asking questions is still actively participating and will be marking you. This examiner may also take some notes –do not be concerned if you see this happening. Of course, notes may be made for feedback purposes or to indicate why a low mark has been given. They can also be made simply to document areas discussed, identify any clarification the co-examiner might want from the examiner before marking or even to indicate why an ‘8’ was awarded. Notes can also be for more mundane reasons, such as completing a topic sheet (which is passed ahead of the candidate so that examiners know what the candidate has been asked about previously – including a note of the short and intermediate cases). When optical marking sheets were used a candidate even apparently complained that he saw the examiner award him two 4s and a 5 before the viva was over, when in fact the examiner had been filling in his unique three-digit examiner number on the mark sheet.
The general pattern of a viva will be that the examiner asks you a series of questions. Eventually you will be asked a question that you cannot answer, or you can only partly answer. The examiner may rephrase the question or ask it in a different way. You may or may not be able to answer it, but the examiner then moves on to a related path of questions. This pattern is the same for all vivas, whether the candidate ends up with a 4 or an 8. If the examiner is having to rephrase the basic ‘competence questions ’
that gain you a 6, then you may not pass. If you have quickly responded to the competence questions early on you may soon be in to the questions determining whether you should get a 7 or an 8 and in many cases candidates at this level are asked more questions that they cannot answer. The basic message is do not try to second-guess what mark you have achieved by the way you have been asked questions and answered them. Just treat every bellas a new start and try not to be influenced by whatever experience you perceived in the previous question.
Who are the examiners?#
Examiners are not selected for their sadistic tendencies or cold hearts. They are consultants who have been in practice for at least 5 years and in that time have demonstrated an interest in, and continuing involvement in, training and education. They have put themselves forwards with the support of their medical director and usually have ongoing roles in regional and national training commift ees, teaching roles and the supervision of trainees. They must also demonstrate that they have remained active in research and that they can make the time to fulfil the role (which includes unpaid weekends away from home).
Applications are considered by the Intercollegiate Board and successful applicants are invited to attend an examiners training course. Successful completion of this allows them to attend an examination as an ‘examiner in training’, where they will observe and learn, and discuss marking (without influencing it) until eventually they can examine with an experienced examiner. Only after completion of the training exam does the examiner’s term begin, but that is not where the oversight ends.
In every diet of the examination there will be a small team of ‘examiner assessors’. They report back to the Intercollegiate Board on all aspects of the examination, from facilities and case mix to catering arrangements and environment. These assessors are also trained (usually after finishing the maximum 10-year terma san examiner) to assess and feedback on examiner performance. As a candidate you may have an assessor siting out of your eyeline, slightly behind you, during vivas or clinicals. The assessors are actually assessing the examiners. Usually each examiner is assessed four times during one examination –twice during clinicals and twice during vivas.
The assessors ensure the standard of examining remains high, but this is supplemented by detailed analysis of the marking behaviour of examiners afterwards, again being fed back to the examiners after the event. Each examiner gets to see how they marked candidates compared to their peers. Hawkish or
Dovish tendencies can be observed and reflected upon. Rest assured that stories of the examiner who routinely fails all candidates simply could not be true – such an examiner would be a wide outlier and could not continue thus.
How are marks allocated?#
The key to this quest ionis the examiners’ standard-seting meeting , which takes place the day before the clinical examinations. Examiners attend a day earlier than candidates and are organized into groups according to which vivas they are examining. Each group then receives the questions that are to be asked in the vivas, with a different block of questions for each session of the two viva days. The questions themselves are taken from the Section 2 question bank and the Section 2 question writing commift ee has a lead examiner for each section, who chooses the questions to be used in each viva ensuring a spread of questions covering the curriculum widely. Thus, at standard-seting the trauma examiners, for example, will receive tablet computers preloaded with all of the trauma questions to be used. The questions have been wrift en with a structure that begins with an opening statement or question that orientates the candidate to the topic, moves on through questions that stimulate discussion that should show whether the candidate is competent in the topic, before opening up into advanced questions that enable high marks to be reached. An accompanying data sheet from the bank will include information on where in the question competence is identified, either from the question writers or from previous diets of the exam. As a group the examiners agree what level has to be achieved to reach a ‘6’, what higher-order responses will take the candidate to a ‘7’ or ‘8’ and what unsatisfactory or dangerous responses might earn the candidate a ‘4’. Examiners can annotate the data sheet with the group decision and this can inform future diets. This also ensures that the standard of the ‘day one consultant’ can be identified and agreed and should be consistently applied.
This process means that candidates examined by different pairs of examiners have the same chance of achieving a pass mark, and candidates being asked different questions in a later session still have the same standard to achieve to obtain a pass. It is accepted that marking a discussion will inevitably introduce some variation, but the standard seting process minimizes this and, when it is applied to the 48 different sets of marks a candidate will be awarded across the vivas, ensures the same standard is required topass the examination for all candidates.
What do the marks mean?#
A closed marking system is used from 4 to 8 and this equates to the following.
4 – Bad fail.
5 – Fail.
6 – Pass.
7 – Good pass.
8 – Exceptional pass.
Examiners assess nine trainee characteristics during the standardized oral examination.
1. Personal qualities.
2. Communication skills.
3. Professionalism.
4. Surgical experience.
5. Organizational and logical, step-wise sequencing of thought processes, ability to focus on the answers quickly.
6. Clinical reasoning and decision making.
7. Ability to handle stress.
8. Ability to deal with grey areas in practice and complex issues.
9. Ability to justify an answer with evidence from the literature.
This has been simplified into three domains.
Overall professional capability/patien t care
Personal qualities professionalism and ethics, surgical experience, ability to deal with grey areas.
Knowledge and judgement
Knowledgeability to justify clinical reasoning.
Quality of response
Communication skills organisation and logical thought process. Assess questions answers and prompting (QAP).
Detailed marking descriptors indicate the behaviours typical of each mark: this helps examiners identify the mark boundaries during the standard seting meeting before the vivas take place. These can be interpreted as follows:
4 – Unsafe and potentially dangerous. A very poor answer. Gross basic mistakes and poor knowledge.
Should not be siting the exam. The examiners have severe reservations about the candidate’s performance and are essentially flagging this up. Too ignorant of the fundamentals of orthopaedic practice topass. Candidate is scoring a 4 in the first instance. Did not get beyond the default questions, f ails in all/most competencies. Poor basic knowledge/judgement/understanding to a level of concern.
5 – Some hesitancy and indecisiveness. The answer is really not good enough with too many deficiencies. Too many basic errors and not geting to the nub of the issue. Wandering off at tangents and not staying focused on the question Misinterpreting the question. Repeats the same ATLS and/or radiograph talk with each oral viva question. Difficulty in prioritizing , large gaps in knowledge, poor deductive skills, patchy performance, struggled to apply knowledge and judgement. Confused or disorganized answer. Poor higher-order thinking.
6 – Satisfactory performance. Covered the basics well, safe and would be a sound consultant. No concerns. Performance OK, but certainly not anything special or outstanding. Good knowledge and judgement of common problems. Important points mentioned, no major errors and required only occasional minor prompting.
7 – Good performance. Would make a good consultant. Articulate and to the pointAble to identify some literature to support their answers, knows various guidelines and publications. Coped well with difficult topics/problems. Goes beyond the competency questions. Logical answers. Strong interpretation/judg ement but wasn’t able to quote specific literature effectively. Good supporting reasons for answers. No prompting needed for answers but prompting required to identify the literature.
8 – Potential g old medal or prize-winning performance. Smooth, articulate and polished. Able to succinctly discuss controversial orthopaedic issues in a sensible way. Excellent command of the literature. Switched on and makes the examiners feel very reassured. Looks and talks the part.
Stretches the examiners, no prompting necessary. Confident, clear, logical and focused answers.
While it is impossible to reference this list while computing an answer, knowledge of the principles of
‘what makes a good answer’ can certainly help your preparation.
Answering questions#
From the above it should be apparent that advice to ‘steer the examiners to ask about something you know about’ is a tactic that is doomed to failure. You will obtain marks as you pass through mark boundaries agreed at Standard Seting by the examiners. You will therefore be steered to these mark boundaries by the examiner’s questions. If you attempt to move the examiner into a different line of questioning , you will be moving them into an area where no marks are available. Of course, the examiners will resist and steer you back to the line of discussion they had started, but in the process you will have wasted time.
The wise candidate will answer the question posed by the examiners. It is entirely appropriate to develop the answer by starting to talk about options, or justifying your answer by referring to literature or whatever – follow the examiner’s cues. If the examiner is listening intently then continue. If the examiner seems to be wanting to interrupt then allow this, as they are probably saving you from wasting your time or you have said something ambiguous that they need you to clarify before they can move on.
If one considers some of the underpinning educational theory, it may help understand the marking structure and how you can best approach answering questions (and even preparing for the exam).
Bloom’s taxonomy describes levels of complexity in using learned material:
1. Knowledge/recall.
2. Comprehension or understanding.
3. Application.
4. Analysis.
5. Synthesis.
6. Evaluation.
Level 1, factual recall, has almost been removed from the Intercollegiate exam. You may still be asked a question that demands a factual answer at some point in a viva, and there may have been occasional level
1 questions (particularly basic sciences) in the SBA paper. In general terms, however, the exam will be checking that you understand the facts and that you can apply your factual knowledge to help you analyse a problem and synthesize a solution, then suggest how to evaluate the outcome. The vivas will be structured where possible to take you along this pathway.
Thus, a viva might start by describing a clinical scenario. From a set of described symptoms and signs, or by looking at a radiograph, your first question might be ‘what do you think is going on here?’ Even interpreting a radiograph, for instance classifying loosening of a hip prosthesis or identifying an AP3 pelvic fracture, shows that not only do you have knowledge and understand it, but that you can apply it.
The examiners will then move you on to adding further clinical detail, for example, which require you to analyse the impact of this new information and predict its impact on the scenario. It is easy to see how the viva becomes an excellent method for testing higher-order thinking, whereas the constraints of thew rift en secft on mean that although it can test the curriculum very broadly, it is largely restricted to level 2 and 3 knowledge.
What the examiners are looking for, therefore, is not simply that you ‘know stuff’, but that you can use it. You can work with limited or incomplete information to make sensible choices. You can make safe decisions on how to initiate management and you will initiate the next steps to fill in the missing data that allow you to come to a conclusion that is effective Finally, why do you do it like that? What are the alternatives You should be prepared to justify your choices not just by saying ‘because that’s what my trainer does’ but by showing that you have thought about the alternatives and have come to a reasoned choice. To score 7s and 8s an argument based on good-quality evidence quoted from the literature and justified by your own training and experience gets you there.
Preparation#
It is easy to fall into the trap of believing that you need to spend months working in the library and working through textbooks in order topass the examination. There is an awful lot to learn in a six-year training programme so that you can be safe to manage a general trauma take and screen referrals to a general orthopaedic service, managing the majority and identifying those that need more specialist care.
However, demonstrating that you have level 1 knowledge far wider even than the examiners does not help if you can’t apply the more mainstream elements of that knowledge base to solve clinical problems.
Sure, the books will help a lot, especially with basic sciences and rare conditions that you may not have met, but for the most part your day job is the best preparation you can get. However, transferring this to the exam environment can feel hard.
Vivas are about discussing clinical scenarios – solving a problem based on information, building on that solution and using the options to identify the best way to a good outcome. It’s therefore more about geting used to talking through that process rather than looking somewhere to find all the answers. There are a few ways to approach this.
Probably the best way is what the best candidates have unconsciously done through their training –discussing cases with peers and with trainers. Take every opportunity to ask questions about cases. Add in
‘what if?’ questions whenever you get an answer. Talk through cases from presentation through management to outcome. Talk about the alternatives and why some people do one thing and others another. This is how you will decide on your own practice when you become a consultant and the examiners want to see how you make such decisions.
Talking over cases with trainers and peers can be morphed into ‘viva practice ’ as the exam draws near. Instead of a wide-ranging discussion of all the possibilities, try to hone it down to specific circumstances and try to become concise. Focus on the sort of case in which decision making can be critical, such as rare cases that might turnup in clinic, and it is important that you recognize them. There are well-publicized lists of the sort of cases that have been asked, so nothing should be a surprise. It is no surprise that the most commonly asked questions in the children’s viva, for example, relate to DDH, SUFE, clubfoot, septic arthritis of the hip and cerebral palsy. Courses are available that specifically offer viva practice, and many find these useful, if only to get them into the frame of mind to work and to give them some idea of what to expect. Remember, however, that examiners are not allowed to take part in
‘crammer courses’ to prepare individuals for the exam. Examiners can, however, help their own trainees to prepare as they will never be called on to examine their own trainees in the real exam. Most such crammer courses involve enthusiastic trainers and trainees who have relatively recently passed the exam.
Most will not have had any examiner training, so the practice may not be a good mirror of the genuine event.
Finally, I would recommend that you try doing some preparation for vivas in the same way that gymnasts can train for complex routines even between training sessions – just think it through in your head. In gymnasts the engram – that cerebrally encoded complex pattern of muscle movement and contraction required to perform a particular skill – can be reinforced by imagining it in real time. ‘Thinking through’ the routine can actually improve physical execution. The same could be said for viva practice –imagine you are asked a question; how will you answer it? Think through in real time what words you would use in your answer to avoid ambiguity. Think what the examiner might say in response and how you will react. You may find that the form of words that comes to your mind in the first instance is clumsy
– could you say it better? For the more commonly asked viva topics prepare in advance the phrases you will use to indicate your personal preference for treatment and the evidence that backs this up. In this way a lot of useful viva preparation can actually be done while ‘relaxing’ or siting on the journey to and from work each day.
On the day#
Don’t panic. Don’t stay up late trying to pack in last-minute revision and missout on sleep as a consequence. Think about your appearance – it is not a beauty contest, but the exam is one of the few occasions in your training where professionalism is formally assessed. How will you present yourself to patient sin the future? You will not be marked down for your choice of shirt or blouse, but if you are scruffy and unkempt for such an important event the examiners will probably assume that you will present yourself in no better a light in the outpatient clinic, where you are supposed to be gaining the patient trust and confidence.
Undoubtedly you will be nervous. The examiners expect this and will try to put you at ease.
Remember that anxiety improves performance up to a point, so nerves can be helpful. Go in expecting a robust discussion on a number of topics with a series of questions culminating in you running out of answers. That is the pattern of all vivas and each viva will end with you in the realms of questions you aren’t sure you are answering correctly. Each bellis therefore a new opportunity to score points and just forget about what has gone before.
Listen to the examiners and answer the questions they pose – do not volunteer an answer to a related question because you know the subject better. If you really are unsure about what has been asked, request clarification. However, do not deliberately try to slow things down as you will only restrict the opportunity for yourself to progress through the marking structure. If your mouth is dry take a sip of water – it is always provided. Expect to finish your sentence when the bell rings then be moved on to the next question. Do not try to pack in more detail on the last question as you will be receiving no marks after the bell and will eat into the opportunities to score on the next question.
Be human! The examiners want to see how you will work under some time pressure and when faced with real clinical problems. They want a discussion with a colleague, to be able to assess how you will behave when you start as a consultant, potentially in a very general post and with a trauma take that includes the full range of emergencies that can present anywhere at any time.
Summary#
The viva section of the Intercollegiate examination is rather like a clinical examination without patient sIt is used to see if you have the knowledge base needed to work as a day one consultant in the generality of orthopaedics and trauma. More importantly, however, it tests whether you can use that knowledge base to solve clinical problems, identify solutions and test that your proposed solutions have worked. It also gives some opportunity to test professional behaviours; after all, it is a discussion between colleagues.
Analysis suggests that the FRCS (Tr & Orthis one of the most reliable high-stakes professional examinations in the world. Go into it with the ambition that you will be back five or so years lateras an examiner!