An investigation of stratification exposure control procedures in CATs using the generalized partial credit model

The a-stratification procedure of item exposure control was designed to stratify items by item discrimination to ensure that an adaptive test would administer items from the entire range of items, not just the most-informative ones. An improvement to the a-stratification method, the a-stratification with b-blocking procedure added stratification according to item difficulty in order to take into account any correlation that might exist within the item pool between item discrimination and item difficulty. These procedures have been shown to work well using dichotomous items. This dissertation explored both stratification procedures using polytomous item pools to investigate whether or not an optimum number of strata could be implemented when administering polytomous computerized adaptive tests. In addition to the stratification procedures, two other exposure control conditions were studied. The randomesque procedure was used in one condition while a no exposure control condition served as a baseline condition. Items calibrated according to the generalized partial credit model were used to construct two item pools. Since the items covered three areas of science, content balancing procedures were incorporated to ensure that each adaptive test provided the appropriate balance of content. Maximum likelihood estimation was used to estimate ability levels from simulated CATs. The number of strata used with both stratification procedures ranged from two to five, to ensure enough items per stratum. Along with descriptive statistics and correlations, bias and root mean squared error helped portray the accuracy of the simulated tests. Item exposure and item pool usage rates were used to show how much of the item pools were being used across administrations of the tests. Finally, item overlap rates were calculated to show how many of the same items were being used among simulated examinees of similar and different abilities. The results of this study did not reveal an optimum number of strata for the stratification procedures with either item pool. Furthermore, the randomesque procedure outperformed the stratification procedures in terms of item exposure and item overlap rates for both item pools. This surprising result was not affected by the number of strata used within the stratification procedures.