Strategies for controlling testlet exposure rates in computerized adaptive testing systems
| dc.contributor.advisor | Dodd, Barbara Glenzing | en |
| dc.creator | Boyd, Aimee Michelle | en |
| dc.date.accessioned | 2011-06-23T19:55:08Z | en |
| dc.date.available | 2011-06-23T19:55:08Z | en |
| dc.date.issued | 2003-05 | en |
| dc.description | text | en |
| dc.description.abstract | Exposure control procedures in computerized adaptive testing (CAT) systems protect item pools from being compromised, however, this impacts measurement precision. Previous research indicates that exposure control procedures perform differently for dichotomously scored versus polytomously scored CAT systems. For dichotomously scored CATs, conditional selection procedures are often the optimal choice, while randomization procedures perform best for polytomously scored CATs. CAT systems modeled with testlet response theory have not been examined to determine optimal exposure control procedures. This dissertation examined various exposure control procedures in testletbased CAT systems using the three-parameter logistic testlet response theory model and the partial credit model. The exposure control procedures were the randomesque procedure, the modified within .10 logits procedure, two levels of the progressive restricted procedure, and two levels of the Sympson-Hetter procedure. Each of these was compared to a baseline no exposure control procedure, maximum information. The testlets were reading passages with six to ten multiple-choice items. The CAT systems consisted of maximum information testlet selection contingent on an exposure control procedure and content balancing for passage type and the number of items per passage; expected a posteriori ability estimation; and a fixed length stopping rule of seven testlets totaling fifty multiple-choice items. Measurement precision and exposure rates were examined to evaluate the effectiveness of the exposure control procedures for each measurement model. The exposure control procedures yielded similar results for measurement precision within the models. The exposure rates distinguished which exposure control procedures were most effective. The Sympson-Hetter conditions, which are conditional procedures, maintained the pre-specified maximum exposure rate, but performed very poorly in terms of pool utilization. The randomization procedures, randomesque and modified within .10 logits, yielded low maximum exposure rates, but used only about 70% of the testlet pool. Surprisingly, the progressive restricted procedure, which is a combination of both a conditional and randomization procedure, yielded the best results in its ability to maintain and control the maximum exposure rate and it used the entire testlet pool. The progressive restricted conditions were the optimal procedures for both the partial credit CAT systems and the threeparameter logistic testlet response theory CAT systems. | |
| dc.description.department | Educational Psychology | en |
| dc.format.medium | electronic | en |
| dc.identifier.uri | http://hdl.handle.net/2152/11922 | en |
| dc.language.iso | eng | en |
| dc.rights | Copyright is held by the author. Presentation of this material on the Libraries' web site by University Libraries, The University of Texas at Austin was made possible under a limited license grant from the author who has retained all copyrights in the works. | en |
| dc.rights.restriction | Restricted | en |
| dc.subject | Computer adaptive testing | en |
| dc.subject | Educational tests and measurements | en |
| dc.title | Strategies for controlling testlet exposure rates in computerized adaptive testing systems | en |
| thesis.degree.department | Educational Psychology | en |
| thesis.degree.discipline | Educational Psychology | en |
| thesis.degree.grantor | The University of Texas at Austin | en |
| thesis.degree.level | Doctoral | en |
| thesis.degree.name | Doctor of Philosophy | en |
Access full-text files
Original bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- boydam032.pdf
- Size:
- 482.35 KB
- Format:
- Adobe Portable Document Format
- Description:
- Restricted to EID users
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 1.66 KB
- Format:
- Item-specific license agreed upon to submission
- Description: