As an evaluation instructor, alumni often send me examples of atrocious examinations they encounter in their subsequent courses. (Most of our alumni go on to a second undergraduate degree or graduate work.) Graduates of my exam-making course generally do very well on other people's exams, because once one knows how to write good examination questions oneself, it is easy to spot the errors in poorly written questions, and so be able to work out the answers. (See http://www.edu.uleth.ca/courses/ed3604/take/mc/how.html#Tricks for examples of what I mean.) On the other hand, my graduates also become hypercritical of sloppy exam-writing, and are often offended by the poor evaluation technique of otherwise excellent instructors. And thus my collection of really bad examinations continues to grow as alumni mail me the poor examples they encounter in other programs.
This week a former student sent me a wonderfully awful test from which I have drawn the following examples. (I won't, for obvious reasons, identify the campus that that student is now on, but suffice to say, this is from an experienced instructor at a legitimate North American university of some little repute, and not in any way an exceptional or unusual case.)
Mummification is first mentioned in 2nd Dynasty texts.
A) True
B) False
C) Maybe
Okay, ignore for the moment the embarrassment of a university instructor using true and false questions, how can one have a "maybe" category in a true/false items?! The whole point of true/false is that they address absolutes. The maybe category is invalid because a case can always be made for 'maybe' -- in this case, that there may well be other texts that have yet to be discovered. Since some questions on this test are true/false and others true/false/maybes, I would suspect "maybe" as the correct alternative anyway, since the instructor probably used it in those cases where there is some existing debate in the field (say an ambiguous reference in some earlier text that may or may not refer to mumification) but it doesn't really matter -- given a "maybe" in a true false question, I can always justify it as the correct answer. It will always win any formal grade appeal.
Old Kingdom Kings did no trade with Asiatics.
a) Not True
b) Not False
The classic double negative question! I have been looking for one of these for years! All the test construction textbooks warn against the use of a double negative (negative in both stem and alternatives) but I have never actually seen a real example of one of these before. Evaluation nstructors have always had to make up our own examples, and students always say, "Oh, nobody would really do that, would they?" and now at last I have a real example.
"Despite his reputation as a tomb robber, Belzoni was nevertheless a fine archaeologist".
A) True
B) False
The archetypal "opinion" question, the ultimate taboo in true/false item writing. Again, I have been looking for an example like this for quite a while. True/false questions can only be used for testing absolutes, not opinions, since one can always make the case for the other side (however tenuously) and we do not score people on their opinions in a democracy. In a formal grade appeal, the student will always win.
The rest of the test is of similarly disappointing quality. Every campus has some kind of Teaching Development Center (or at least a Teaching Development Committee, if the campus is too small to afford dedicated staff) that sponsors 'how to' workshops on instruction and assessment techniques, but of course those that need the workshops are never the ones who attend.
The sad thing is that the former student who sent me this test had been absolutely raving to me about what a wonderful professor this was and what a great course and how much the student was enjoying the class, prior to the test. Afterwards, the student was so disappointed with being robbed of the opportunity to demonstrate the deep learning achieved in that class, that their enthusiasm was considerably eroded. The student still considers that professor an all time favorite, but then this is a student motivated by a thirst for knowledge rather than grades, and so perhaps more willing then most to forgive such tragic flaws.
(Not, I hasten to add, that any of us are flawless. My exams may be good, but I could no doubt learn a thing or two about lecturing and motivating students from this instructor, judging by my graduate's enthusiasm for an otherwise arcane subject. But wouldn't it be great if professors could learn from each other, and build on each other's strengths, rather than merely perpetuate the poor teaching and assessment techniques they themselves endured as undergraduates?)
=================================================================
A news bite from John Herbert (current Editor of Under the Ozone Hole):
Here's a little Columbia disaster news blip that came and went under the radar....
I just happened to be watching NASA Administrator Sean O'Keefe testifying in front of Congress last week. An American Congressman (whose name I didn't catch but it was no one you've heard of) began questioning O'Keefe by laying out a string of events.
He said (and I paraphrase) that the contract for the glue (or "urea") that held the heat protection tiles in place was originally awarded to a Canadian company in "Fort Sasketchwhichwan, Alberta, Canada."
No company in the United States could meet the specs that NASA required for this urea -- in fact, only one manufacturer in the world could, and it was located in Fort Saskatchewan. Up until five years, this was the only source of shuttle glue tile.
What changed five years ago? This company was bought out by a larger Canadian company that does business in Cuba.
American government agencies cannot deal with companies that do business in Cuba, so NASA could no longer buy tile glue from the only source in the world that could meet its specs.
He then asked O'Keefe to comment and O'Keefe offered none, saying that he was unfamiliar with these events (O'Keefe has only been with NASA a year.)
So it may turn out that the American policy of sanctions against foreign companies operating in Cuba might have played a large role in the death of seven American astronauts.
Bet you won't see this on CNN.
================================end quote============================
