Friday, November 25, 2016

Techniques for Taking Essay Tests

Before beginning to answer the questions:

  • Check that you have all the question pages; check the back of the pages to see if there are questions there.

  • Read all test or examination instructions and questions.

  • Note which question or sections are compulsory, optional, or allow for choice. In the instructions, underline or circle all key words or phrases such as ‘and’, ‘or’, ‘answer three of the following. . . .’

  • Make certain you know the number of questions you are required to answer.

  • Note the numerical value of each question and the total time allowed for the test or examination. Then convert the numerical value to the appropriate number of minutes per question. Remember to deduct five minutes for preparation and at least ten minutes for reading, correcting and revising. The proof-reading and revision will be made easier if questions are answered on every other line.

  • Within each question underline or circle all key words or phrases such as ‘discuss’, ‘list’, ‘explain’, ‘compare’, ‘three causes of’. . . .

  • Choose the question you wish to do first. The question you answer first should be the one with which you feel most confident. Spend only the appropriate amount of time, or less, on this question.

While answering the questions:

  • For each question, jot down on a separate piece of paper (rough work), the key ideas or facts pertaining to that question.

  • Decide which ideas or facts to include in your answer, based on what is required in the question. For example, if the question asks for three facts or concepts, choose from your list of key ideas or facts the three most important facts upon which you can expand. Do not waste valuable time providing information not specifically asked for by this particular question.

  • Take into account the marks allotted for each question or sub-question, and decide how much emphasis should be placed on each idea or fact. For example, if a question asks for three concepts or facts, and the total number of marks is twelve, assume that each fact or concept should be developed with four points each in order to add up to 12 – or three points each if marks are also assigned for organization and style.

  • Decide on the format of the answer. Will you discuss each concept in a paragraph of its own, or will you compare concepts within a paragraph?

  • Use headings or underlining to make it easy for the marker to quickly identify your key concepts.

  • Leave some space between the end of each answer and the start of the next one. This space will allow you to add further information if you recall something important, and you have time at the end of the examination.

  • Go immediately to the next question of your choice. Start each answer on a new page. Draw an arrow to the next page to make sure marker finds all your answers.
After answering all your questions:

  • Make certain you have answered the required number of questions.

  • Read all your answers carefully and correct or revise where necessary.

Take-Home Tests

  • Always type your answers if given the option; typed papers are worth 10% more than the identical paper handwritten because typing is easier to read and therefore typed essays appear to ‘flow’ better.

Friday, November 18, 2016

Do Testwise Strategies Really Work (Testwiseness Part Three)

Hell, yeah.

Here is a personal anecdote from back when I worked as a full-time Test Development Specialist.

I had been working for a few years as a Test Specialist when the senior manager suddenly realized that there was only one person on staff who knew how the software for grading our exams worked. As it occurred to him that the whole operation could grind to a halt were something to happen to that individual, he transferred me into the computer programming section as the backup programmer. (This made sense to him because I was the only person back in those days—30 years ago!—who owned his own personal computer, so I looked like a computer guy to him.) Since I had no programming training, and since all our computer programs were written in FORTRAN, my boss signed me up for a university-level correspondence course on FORTRAN programming. By the time the course materials had arrived in the mail two weeks later (this before online courses!), however, my boss had come to his senses, hired a team of FORTRAN programmers, and switched me back to writing tests. I therefore had no further need to learn FORTRAN, and I pushed the unopened box of course materials under my bed and forgot about it.

Six months later I received a notice in the mail saying the examination for the FORTRAN course would be at such and such location in a week's time. And I thought, "Well, the course has already been paid for, why not take the test?" Not, you understand, that I had ever opened the box or looked at any of the course materials. I just wondered if I could pass the test on my knowledge of test design.

The examination, if I recall, was 70% multiple-choice and 30% written response. I had no hope of answering the written response part, since I had absolutely no idea how to write a FORTRAN program. I did repeat back the question (because some markers wrongly give at least one mark for a student filling in the space with something) and attempted a line or two of code based on examples from the multiple-choice part of the test, but since I had no idea what it was I was copying, I highly doubt it made any sense at all.

The multiple-choice questions, however, were a different matter. As a test specialist, I was able to examine each question for the tiny flaws that gave the answer away. Examining the alternatives for the odd one out, or for one answer that was significantly longer than the others, and so on, (see previous post for these techniques) I gave it my best college try, even though I often had no idea what the question was asking, let alone what the correct answer might be.

Three weeks later I received a letter saying I had obtained 70% on the test; which considering I had left the written response essentially blank, means I must have aced the multiple-choice. Not bad for a topic I literally knew nothing about!

I subsequently wrote the institution in question pointing out this weakness in their tests, and sometime later they hired me to give a one-day workshop on the do's and don'ts of multiple-choice item writing. There is, therefore, no use asking me how to register for that course; these techniques won't work to pass that test again!

This was probably an extreme case; and, to be fair, such tests are not intended for people who make their living designing multiple-choice tests. Most people would not have found the flaws quite so obvious.

In sharp contrast, I sat down with a copy of Alberta's Grade 12 Mathematics Diploma Examination to see what I could get using these test-wise tricks. In this case, I only managed 23%, slightly less than one would expect by pure chance. This test had been written by test experts and it was impossible for me to get through it without actually knowing the course content (calculus) which I did not.

Most classroom teachers' tests are somewhere in between these two extremes. A student will not likely be able to pass a teacher-constructed exam solely on knowing testwise strategies, but such strategies are likely to improve the testwise student's grade a by significant percentage if the teacher does not know and implement correct test construction technique.

In that context, it is worth noting that teachers should be cautious about using test banks that come with textbooks. Some of the larger textbook publishers do employ professional test developers as editors for their test banks, but many do not. Test banks are sometimes written by the author of the textbook (who likely will not have any expertise in test-construction) but are more often written by graduate students from that discipline desperate for cash (and who definitely do not have test-construction skills) and dashed off over a weekend before the publisher's deadline. In other words, don't assume the test bank is worth anything. At a minimum, any instructor using a test bank written by anyone else should edit the questions before putting them on their test to ensure the questions are less flawed and more relevant to their own classroom. (On the other hand, starting from a flawed test bank is sometimes easier than writing one's own items starting from a blank page. Using a test bank as initial rough draft can be okay, provided one guards against it being all rote memorization. You will probably need to write your own higher level thinking questions, because publishers can't pay grad students enough to come up with those!)

Friday, November 11, 2016

Why Teach Students to be Testwise (Testwiseness Part Two)

Some instructors may be horrified at the suggestion that we teach students test-taking strategies, let alone testwise strategies. Allow me to suggest why teaching students some testwise strategies is a good thing.
  1. It levels the playing field

    Some students already know slogans like "when in doubt, choose c" even if they don't know the rationale behind it. If some students are testwise and others are not, it is not a level playing field. The assessment becomes inaccurate because the test is now measuring testwiseness rather than the skills and knowledge intended, since students who are testwise can significantly increase their scores (on poorly constructed tests). Instructors need to ensure their assessment is measuring learning outcomes not cultural capital. Teaching everyone the same basic testwise skills levels the playing field by ensuring everyone has the same knowledge.

  2. It reduces test anxiety

    Students often do poorly in testing situations because of test anxiety. Students with test anxiety often feel they have no control over the many variables that make the outcome of the test uncertain: which subtopics will questions be drawn from, how hard each question will be, how long it will take them to remember and answer or solve for a question, the reading level of the question, the ambiguity of the question stem or of the answers offered, and so on. Having some strategies available for topics about which they know or recall little significantly reduces overall anxiety because it gives the student a sense of control. "Well, at least I know what to do if I have no clue what the answer is!" As part of a lesson on how to study for and take multiple-choice tests (two previous posts on this blog), giving students some testwise strategies goes a long way towards lowering (though never eliminating) test anxiety.

  3. It forces teachers to write better multiple-choice tests

    Testwise strategies only work on poorly constructed tests, so if students know testwise strategies, teachers are forced to develop the test-construction skills, and to take the time necessary, to write the better tests necessary to defeat testwiseness.

Students must always be cautioned, however, that the only way to do well on tests is to learn the course material, and that they should not rely on testwise strategies. It is natural for some students (especially in middle school) to embrace testwise strategies as a substitute for studying, so it is important for instructors to note that these strategies will not work on one's own tests, because one's own tests are properly constructed.

Sadly, I have also encountered some instructors who take teaching test-taking strategies to the point of emphasizing these strategies over actual curriculum (i.e., teaching to the test). Such teachers are thinking like a middle school adolescent and should be relieved of their teaching certification.

Friday, November 4, 2016

Testwiseness (Part One)

The best way to do well on a test is to know the answers. However, when faced with a question you cannot answer, these tricks will sometimes work on poorly written multiple-choice tests. Only use them if you do not know the answer and have to guess.

  1. When you have to complete a sentence, see if one of the answers fits better grammatically than the others.

    23. A dog is an
            A) animal
            B) machine
            C) mineral
            D) vegetable

    In this question, the “an” gives you a clue to the right answer, because you know the correct answer has to start with a vowel. “An animal” works, but it would have to be “a machine” or “a mineral” or “a vegetable” to be a correct sentence. Since the question says “a dog is an” rather than “a dog is a”, the answer has to be “animal” — because “animal” is the only one that fits with the “an” in the question.

  2. See if one of the answers is stated in more "textbook language" than the others.
  3. Instructors sometimes write questions by quoting a statement right out of the textbook, using the first half as the "question" and the second half as the correct "answer". Then they have to invent 3 wrong answers in their own words. Sometimes it is possible to identify which answers are in the textbook's style, and which in the instructor's.

  4. See if one of the answers repeats key words from the question.

    43. An example of a nuclear reaction is
            A) hydration
            B) combustion
            C) sublimination
            D) nuclear fission

    Even if you do not know what the question means, the answer is probably (D) because it repeats the word “nuclear”, which is a key word in the question.

  5. See if one of the answers is much longer than the others.

    The person making the test wants the right answer to be inarguably correct. To make it inarguably correct, they sometimes add more details to the correct answer, making it much longer than the others.

    45. The freezing point of water is
            A) 100 degrees centigrade
            B) 0 degrees Fahrenheit
            C) 0 degrees Kelvin
            D) 0 degrees centigrade for pure water at sea level

    (D) is the correct answer.

  6. See if one of the answers sticks out as the “odd one out”.

    Since there is only one correct answer to a question, that answer has to be different from the other answers. Consequently, if you see two or three answers that all mean the same thing, they must be the wrong answers.

            A) Boy
            B) Girl
            C) Son
            D) Lad

    Since “Boy”, “Son”, and “Lad” all mean roughly the same thing, chances are the answer is “Girl”, even if we do not know what the question is.

    This trick is dangerous, however. Sometimes the “odd one out” can fool you. Maybe the answer was “Son” because it is the only answer about family relationships. But this trick might help if you are guessing blind anyway.

  7. If you have to guess, and spot a typing error in one of the answers, choose one of the other answers.

    Sometimes the instructor making a multiple-choice test will proofread it by reading the question, looking at the right answer, and going on to the next question. They may forget to proof the wrong answers. So if there is an error, it is more likely to be a wrong answer. (This trick used to work better before spell-checking software became common, but not all instructors avail themselves of spell checking, and you can still check for homonyms.)

  8. If you have to guess, eliminate answers with unconditional words like "always" or "never".

    In the real world, it is usually possible to find one or two exceptions to every rule, so answers with unconditional statements in them like "never" or "always" are usually the wrong answer (except in math or science where absolutes are possible). Similarly, for the same reason, weasel words like "approximately", "often", "usually", sometimes indicate a correct answer.

  9. In questions where "all of the above" is used, see if you can eliminate one of the answers; or if you can identify two of the answers as correct.

    If two answers are correct, "all of the above" must be correct. If any one of the other answers is clearly wrong, "all of the above" must be wrong. By eliminating "all of the above" along with the wrong answer, you are now down to choosing out of two answers, and so have a 50/50 chance of getting the question right just by guessing.

  10. In questions where "none of the above" is used, choose "none of the above".

    On tests where "none of the above" is only used occasionally, it is often the right answer. Even when it is not correct, it is often possible to argue for it by demonstrating some inadequacy in each of the other answers, thus making "none of the above" correct by default.

  11. If you have absolutely no idea what the answer is, choose ‘(C)’.

    Instructors often try to “hide” the correct answer in the middle of the wrong answers. So they unconsciously choose (C) for the correct answer more often than any other letter. [A few instructors habitually choose B instead, If they are using a computer program or test bank, however, this may not work, since computers are usually programmed to randomize the placement of the correct answer.