The New Fellowship Exam
So here it is. The unofficial review of the New 2015 FACEM Exam.
Guest blogger Dr Edward Rêjé gives his view of the examination in the form of an open letter. We accept no liability for any offence caused…
Re: The Fellowship Exam Re-booted
Dear Dr Cross,
So the first of the “new’ exams is done and dusted. What was it like? Well, let me put it like this; in between the MCQ’s and SAQ’s a seagull crapped on my shoulders and that was the best thing that happened to me all day.
Before I give my opinion of the exam I should say that I am about 95% sure I failed thus my opinions may well be just sour grapes. However, I have spoken to several other people who sat the exam and their opinions are very similar to mine.
A couple of weeks before the exam we got an email saying that there would no longer be critical errors.
A few days later another email went out saying we should ignore that – critical errors were back!
After being repeatedly warned that we needed to be very careful with the timing of our answers, we were told in the last week that we could not take timers into the exam. (but we didn’t need to worry because there would be an “analog clock” on the wall). However, on the day people used timers and no one said anything. Minor points perhaps, but annoying none-the-less.
Prior to the exam the college made available very few practice SAQ’s. Whilst there were many “home grown” questions in circulation, I thought most of them were a bit Mickey Mouse. I sat 3 trial exams, including the AFEM exam in Queensland and passed them all with time to spare. Furthermore, whilst I may not have always known the answer, I always understood what they were asking which was not always the case with the old style SAQ’s and VAQ’s. Thus, whilst I was not really sure what to expect, I was quietly optimistic about the new exam format…
The MCQ’s seemed straight forward, clinically relevant and fair. My only criticism would be that you had to stay in the room for all 3 hours so many of us spent 2 hours staring at the clock on the wall (it was analog – at least the college got that right). One complaint I heard was that you couldn’t have a water bottle at the table in case you spilt it on the computer. Thus people were continually walking to the front of the room to get a drink. I didn’t notice but clearly others found it distracting.
It has also been claimed that you could read the answers off the computer belonging to the person sitting in front of you. Perhaps that is what the college means by “full, open disclosure.”
And that brings me to the SAQ’s. My DEMT spent a lot of time running teaching sessions and trial exams (sorry to waste your time, dude). Given that he is also an examiner, I figured he would have lots of useful insights. In hindsight, I think he was as much in the dark as we were. He did say that before marking the questions, the examiners would answer them to see what you could realistically expect in 6 minutes. Frankly, that’s bollocks. If you want to mark those questions fairly, you need to answer the whole paper in 3 hours. To just do 3 questions and think that reflects the real exam is just nonsense. Secondly, it didn’t seem like each question was designed to be answered in 6 minutes. One question went for 4 if not 5 pages whilst others were only 1 and a half pages. It may be 30 questions in 3 hours but it wasn’t 6 minutes per question.
Before talking further about the exam paper, it is worth describing the AFEM course in Brisbane. These guys, with far less money, time and resources than the college, came up with several hundred excellent practice questions and a 30 question exam with publication quality images. They also gave us 5 separate exams books. The college came up with one gigantic book. You only had to write your name on the front cover. When asked if this was a mistake the rather snarky reply was “Why? Are you afraid the staple will fall out?” The appropriate come back to that is “No, but you’re going to take the book apart so that it can be marked and my exam number won’t be on any of the pages. How much thought did you give this?”
An even bigger problem was the quality of the ECG’s and x-rays. Frankly I’d be embarrassed to use such poor quality images in an exam for medical students let alone in a Fellowship exam. There was one x-ray that was only about 4 inches by 4 inches (and I’m really old with possible cataracts). They asked for 3 abnormalities. I could barely tell that it was a wrist let alone spot the abnormalities. I kept looking around to see if they were providing magnifying glasses but no such luck. I don’t know how we can expect the other specialties to treat Emergency Medicine with respect when the AFEM guys take more pride in their trial exam than the college does with the real thing.
Based on the ‘hot’ debrief in the pub afterwards, the most common complaints about the SAQ’s seemed to be:
1) not enough time
2) the questions were poorly written
3) the questions did not reflect the curriculum
It was a very hard exam to finish in 3 hours – many didn’t and God help you if English is not your first language. I felt like I had fallen behind time quite early on and got quite flustered. As a result, what may have been a simple question in isolation became far more challenging in a 3 hour slog. I am curious to know if any of the examiners has actually tried to do that exam in 3 hours. I suspect not (but I would be impressed if they have).
I don’t understand why the college is so hung up about doing the exam quickly. If you worked at that pace on the floor you would kill some-one. It is also ironic that they expect so much in so little time and then have the nerve to complain about poor hand writing. For me, 28 questions in 3 hours would be far fairer.
As I have said above, the majority of the practice questions were well written – you knew what they were asking. This was not the case with the real exam. For far too many questions I found myself (hysterically) thinking “WTH? WTH?” This is made worse by the fact that you simply don’t have time to re-read a question or spend time trying to work out what they are asking. On several occasions I gave essentially the same answer to successive questions. Maybe that was just me. I’d be curious to hear what others felt.
Finally, what sort of thing did they ask? Well, let’s start with what they didn’t ask. There were no chest x-rays (or if there was, I was so panic stricken that I missed them). Maybe CXR’s are on the OSCE. On the other hand since even the college doesn’t know what is on the OSCEs, I kind of doubt it (but that’s alright boys, you’ve got 66 days to sort it out). There was also bugger all trauma and airway questions. On the other hand there were 2 child abuse and about 4 admin questions. (The college apparently loves “full, open disclosure. I wonder if that will apply to a review of the exam?) That ratio of questions certainly does not reflect life on the floor where I work.
I had a hunch that the first “new exam” would be have some teething problems. Sadly, it was more like “meth-mouth”
Resigned to may fate.
Dr Edward Reje
I am still a believer in the “new exam”.
However, I also believe that the college wasn’t ready but went ahead with the exam anyway.
The candidates deserve better.