As an evaluation instructor, alumni often send me examples of atrocious examinations they encounter in their subsequent courses. (Most of our alumni go on to a second undergraduate degree or graduate work.) Graduates of my exam-making course generally do very well on other people's exams, because once one knows how to write good examination questions oneself, it is easy to spot the errors in poorly written questions, and so be able to work out the answers. (See http://www.edu.uleth.ca/courses/ed3604/take/mc/how.html#Tricks for examples of what I mean.) On the other hand, my graduates also become hypercritical of sloppy exam-writing, and are often offended by the poor evaluation technique of otherwise excellent instructors. And thus my collection of really bad examinations continues to grow as alumni mail me the poor examples they encounter in other programs.
This week a former student sent me a wonderfully awful test from which I have drawn the following examples. (I won't, for obvious reasons, identify the campus that that student is now on, but suffice to say, this is from an experienced instructor at a legitimate North American university of some little repute, and not in any way an exceptional or unusual case.)
Mummification is first mentioned in 2nd Dynasty texts.
A) True
B) False
C) Maybe
Okay, ignore for the moment the embarrassment of a university instructor using true and false questions, how can one have a "maybe" category in a true/false items?! The whole point of true/false is that they address absolutes. The maybe category is invalid because a case can always be made for 'maybe' -- in this case, that there may well be other texts that have yet to be discovered. Since some questions on this test are true/false and others true/false/maybes, I would suspect "maybe" as the correct alternative anyway, since the instructor probably used it in those cases where there is some existing debate in the field (say an ambiguous reference in some earlier text that may or may not refer to mumification) but it doesn't really matter &emdash; given a "maybe" in a true false question, I can always justify it as the correct answer. It will always win any formal grade appeal.
Old Kingdom Kings did no trade with Asiatics.
a) Not True
b) Not False
The classic double negative question! I have been looking for one of these for years! All the test construction textbooks warn against the use of a double negative (negative in both stem and alternatives) but I have never actually seen a real example of one of these before. Evaluation nstructors have always had to make up our own examples, and students always say, "Oh, nobody would really do that, would they?" and now at last I have a real example.
"Despite his reputation as a tomb robber, Belzoni was nevertheless a fine archaeologist".
A) True
B) False
The archetypal "opinion" question, the ultimate taboo in true/false item writing. Again, I have been looking for an example like this for quite a while. True/false questions can only be used for testing absolutes, not opinions, since one can always make the case for the other side (however tenuously) and we do not score people on their opinions in a democracy. In a formal grade appeal, the student will always win.
The rest of the test is of similarly disappointing quality. Every campus has some kind of Teaching Development Center (or at least a Teaching Development Committee, if the campus is too small to afford dedicated staff) that sponsors 'how to' workshops on instruction and assessment techniques, but of course those that need the workshops are never the ones who attend.
The sad thing is that the former student who sent me this test had been absolutely raving to me about what a wonderful professor this was and what a great course and how much the student was enjoying the class, prior to the test. Afterwards, the student was so disappointed with being robbed of the opportunity to demonstrate the deep learning achieved in that class, that their enthusiasm was considerably eroded. The student still considers that professor an all time favorite, but then this is a student motivated by a thirst for knowledge rather than grades, and so perhaps more willing then most to forgive such tragic flaws.
No comments:
Post a Comment