| One of the continuing challenges of developing professional educational and psychological assessments is to develop test items that fall within a specified range of difficulty. The ability to modulate the relative difficulty of items by following a blueprint for item construction would enable item developers to target gaps within item banks, increase item survival rates, and enhance the development of algorithms for automatic item generation. One current recommendation to modulate the difficulty of multiple-choice items is to focus on developing options that are more similar or less similar to the key. Items where all options are similar to the key (homogeneous) theoretically should be more difficult and discriminate better than items where the options are less similar to the key (heterogeneous). The purpose of this study is to quantify the changes in item parameters associated with a change in option homogeneity. It expands on previous work by attempting to quantify the difference in item difficulty or discrimination that can be correlated with increasing or decreasing the similarity of options. Creating homogeneous options has become an accepted guideline for writing multiple-choice items. Haladyna, Downing, and Rodriguez (2002) conducted a survey of 27 textbooks on educational testing and found widespread support for the creation of homogeneous options. Eighteen of the books reviewed supported the use of homogeneous options, whereas the remaining nine neither supported nor refuted the practice. Haladyna et al. (2002) recommended, as a guideline for best practices in item writing, that options that are similar to the key should improve discrimination of the item. However, they qualified the recommendation because not enough empirical evidence existed to definitively support or refute the rule (Haladyna et al., 2002). Data from at least four research studies support using homogeneous options (Ascalon, Meyers, Davis, & Smits, 2007; Green, 1984; Guttman & Schlesinger, 1966; Smith & Smith, 1988). The original idea of creating homogeneous options comes from Guttman and Schlesinger (1967). They theorized that creating options that are similar to the item key (homogeneous options) in a multiple-choice item would result in items that are more difficult and better discriminating than items in which options are less similar to the item key (heterogeneous options). Their theory was based on a logical analysis of how candidates responded to items and the results of a set of ability tests given to 637 students in Grades 7 through 9 (Guttman & Schlesinger, 1966). Green (1984) also found a correlation between item difficulty and option homogeneity which she referred to as option convergence. A sample of 19 items from undergraduate students from the University of Washington was analyzed and data suggested that greater similarity among options produced items that were more difficult. This result was obtained by creating nine variations of each item with differences in option convergence and language difficulty. Approximately 300 responses were obtained for each variation, and an ANOVA produced a significant result for option convergence (alpha = .01). The effect for language difficulty was not significant. |