Best Practices for Creating Multiple-Choice Questions

When teaching online, instructors often turn to multiple-choice quizzes and exams to assess student learning. This option provides many benefits, including instantaneous feedback to students and efficient grading for faculty. There are some key strengths and limitations (Zimmaro, 2010) to keep in mind when writing multiple-choice questions:

Strengths

  • Can measure a broad sample of achievement
  • Item analysis can reveal the difficulty of each item, along with how well the assessment discriminated between the strong and weaker students in the class
  • Can cover a lot of material very efficiently (about one item per minute of testing time)

Limitations

  • Constructing strong items is time-consuming
  • Real-world problem solving utilizes a different process (proposing a solution versus selecting a solution from a set of alternatives)
  • Can test factual information, but fails to test higher levels of cognitive thinking

How to Get Started

Creating effective multiple-choice questions involves viewing the course holistically. It is important to consider all major components of the course, including learning objectives, instruction, assessment, and evaluation (Zimmaro, 2010). If the instruction has conveyed the importance of well-designed objectives, and the instructor thoughtfully evaluates the quality of student performance, then assessment will be a successful, positive experience.

Before writing items, create a test blueprint (Zimmaro, 2010) to ensure that you have covered the appropriate topics and objectives through different levels of Bloom’s Taxonomy. An example is below:

 Topic A: American RealismTopic B: SatireTopic C: American ModernismTopic D: Contemporary LiteratureTOTAL
Remember12115 (12.5%)
Understand21227 (17.5%)
Apply443415 (37.5%)
Analyze323210 (25%)
Evaluate 1 12 (5%)
Create  1 1 (2.5%)
 10 (25%)10 (25%)10 (25%)10 (25%)40*

*note: this example is based on a 40-item exam

Hints for Preparing to Write Items

  • Create test items while you prepare class lessons
  • Make note of questions that students ask frequently during class
  • Make note of common misconceptions students make during class or in homework
  • Invite students to submit items at the end of class or at other times (Zimmaro, 2010)
  • When a learning objective focuses on higher-order learning, it is much easier to write corresponding assessment questions that reflect higher-order thinking skills
  • Item writing is iterative; expect to revise your questions several times (Theide & Goodman, 2010)

Best Practices

  • When selection-type items are to be used (multiple-choice, true-false, matching, check all that apply) an effective method is to start each item as a multiple-choice item and switch to another item type if needed
  • Present practical or real-world situations to the students
  • Present the student with a diagram of equipment and ask for application, analysis or evaluation
  • Present actual quotations taken from the news or other published sources and ask for the interpretation or evaluation of these quotations
  • Use pictorial materials that require students to apply principles and concepts
  • Use charts, tables or figures that require interpretation
  • Avoid giving grammatical clues to the correct answer (“a vs. an,” singular vs. plural, etc.)
  • Avoid using “All of these,” “None of these,” or “Both A & B” as answer choices (Zimmaro, 2010)
  • Use plausible distractors–write options so they are homogeneous in content (Bothell, 2001)
  • Prepare students for the test by specifying objectives or providing study questions
  • Evaluate items; look at how well distractors worked, and examine the performance of high- and low-performing students

Examples

Gronlund (1998) provides useful examples of exam questions based on Bloom’s Taxonomy (examples have been modified to reflect the Revised Bloom’s Taxonomy):

Remember (retrieve relevant knowledge)

  1. Reliability is the same as:
    1. consistency.
    2. relevancy.
    3. representativeness.
    4. usefulness.

Understand (construct meaning)

  1. The statement that “test reliability is a necessary but not sufficient condition of test validity” means that:
    1. a reliable test will have a certain degree of validity.
    2. a valid test will have a certain degree of reliability.
    3. a reliable test may be completely invalid and a valid test completely unreliable.

Apply (use procedure in a given situation)

  1. Which one of the following learning outcomes is properly stated in terms of student performance?
    1. Develops an appreciation of the importance of testing.
    2. Explains the purpose of test specifications.
    3. Learns how to write good test items.
    4. Realizes the importance of validity.

Analyze (break material into parts; determine how parts relate)

  1. Which one of the following unstated assumptions is this teacher making?
    1. Students go to school to learn.
    2. Teachers use essay tests primarily.
    3. Tests make no contribution to learning.
    4. Tests do not indicate a student’s absolute level of learning.

Evaluate (make judgments)

  1. Judge the sentence in italics according to the criteria given below: “The United States took part in the Gulf War against Iraq BECAUSE of the lack of civil liberties imposed on the Kurds by Saddam Hussein’s regime.”
    1. The assertion and the reason are both correct, and the reason is valid.
    2. The assertion and the reason are both correct, but the reason is invalid.
    3. The assertion is correct but the reason is incorrect.
    4. The assertion is incorrect but the reason is correct.

Create (reorganize into a new pattern)

  1. Which one of the following propositions is most essential to the final conclusion?
    1. Effective self-evaluation does not require the use of tests.
    2. Tests place students in rank order only.
    3. Test scores are influenced by factors other than achievement.
    4. Students do not go to school to take tests.

Resources