A Multiple-Choice Study: The Impact of Transparent Question Design on Student Performance
This university classroom study seeks to better understand how, and to what extent, designing more transparent (or TiLTed) multiple-choice questions would impact student performance. Ninety-two students in an introductory American Government class were randomly assigned “TiLTed” and “unTiLTed” versions of thirty-five test questions. Questions were “TiLTed” and “unTiLTed” in one of three ways—involving either (a) adding or eliminating unnecessarily difficult vocabulary from the stem; (b) adding or eliminating “all-of-the-above” and “none-of-the-above” answer options; or (c) adding or omitting additional cues or context. Statistical analysis showed that TiLTing questions generally increased student scores, with twelve questions showing positive statistical significance at the p ≤ .10 level. The most robust positive effects involved simplifying question vocabulary and avoiding all-of-the-above options (none-of-the-above was not examined in isolation). Adding additional cues and context produced mixed, and in some cases negative, results.
This is an original work
This work has not been previously published
IRB approval verification
LeJeune, J. (2023). A Multiple-Choice Study: The Impact of Transparent Question Design on Student Performance. Perspectives In Learning, 20 (1). Retrieved from https://csuepress.columbusstate.edu/pil/vol20/iss1/8