Browsing by Subject "Standardized testing"

Now showing 1 - 3 of 3

Evaluating the utility of the test of narrative language for use with deaf children via American Sign Language
(2013-05) Cravens, Elizabeth Laura; Bernstein, Mark E.; Mann, Wolfgang (Lecturer)
The expressive language tasks of the Test of Narrative Language (Gillam & Pearson, 2004) were administered to eleven deaf, native signers and fifteen English-speaking hearing children who were between the ages of six and ten years old. These tasks were administered to determine the appropriateness of this measure for use with special populations and bring to light new information about children's narrative development and the differences in the language modalities of these two groups. Also, the application of this information on future testing of deaf populations is examined. The eleven native signers came from a single residential school for the deaf, and all had deaf parents. The fifteen hearing children were recruited from a private school and through associates of the primary investigator. The tasks were administered according to the TNL manual's protocol and script, with the primary investigator speaking English for the hearing children and a native signer using American Sign Language for the deaf children. Their narratives in these tasks were coded according to the standards of the test and examined: factual story comprehension, story retell abilities (and inclusion of target terms), story generation from a picture sequence, and story generation from a single picture scene. This study found that though the hearing group outperformed the deaf group on each task's raw score, the specific subcategories of "Grammar" and "Story" from the picture sequence-based story generation task, and the "Characters" and "Vocabulary and Grammar" coding of the single picture-based generation task showed ASL users as having stronger narrative skills as a whole. Specific target items from the story retell also proved differentially problematic for the ASL group and should be altered in future utilization of the TNL with deaf children. In the future, the need for appropriate and representative testing of deaf children's narrative skills should take a higher priority, and greater understanding of the differences between ASL and English will be desired for both test creators and those testing deaf children.
Longitudinal predictions using alternative binning to reduce regression to the mean
(2017-11-28) Stephens, Sarah Emily Elias; Marder, Michael P., 1960-; Marshall, Jill; Swinney, Harry; Gleeson, Austin; Gordon, Vernita
Educational policies in Texas that regulate the evaluations of students, teachers, and schools, can have profound impacts on the success of those individuals and institutions. The evaluations are largely based on the outcomes from standardized exams, as well as graduation rates and college preparedness. The analysis of standardized exam scores and policy impacts must be accurate, rapid, and reliable if it is to inform new policies. The possibility of using year by year longitudinal series of exams to extract predictions about policy interventions is greatly impacted, in practice, by a statistical phenomenon known as regression to the mean. I present a novel method, inspired by statistical and fluid mechanics, to address this problem, called Alternatively Binned Streamlines. I justify the use of this method through a simple theory. Then I apply it to the Texas State Longitudinal Data System, which contains standardized testing data for primary and secondary school students between 2003 and 2015. I show that regression to the mean can largely be eliminated, making it possible to predict the longitudinal performance of aggregated students, using only two or three years of data, with acceptable accuracy. Through these predictions, I also identify the effects of a state-wide intervention called the Student Success Initiative. Thus, I demonstrate that Alternative Binning provides rapid analysis of policy impacts and predictions of longitudinal student performance with the ability to inform policy.
Reassessing assessment : using Bayesian methods to examine the efficacy of multiple-select multiple-choice items
(2019-05) Golubski, Christopher Michael; Petrosino, Anthony J. (Anthony Joseph), 1961-; Stroup, Walter; Borrego, Maura; Callahan, Rebecca; Beretvas, Susan
Standardized tests play a central role in nearly all modern schools in the United States. Recent research has indicated that many of these standardized tests do not effectively measure students' underlying knowledge of a subject. Additional research must be completed to find innovative and effective items that have not only successfully measure student ability, but also provide educators with information about those abilities so they can effectively address student deficiencies. 371 participants were recruited from introductory statistics courses and given an assessment that contained both traditional single-response multiple choice items and multiple-select multiple-choice items. The participants were also randomly divided into two conditions, one where the directions told them they should select single responses for the single-response items and multiple responses for the multi-select items. The second condition instructed participants that all items could have multiple responses. All items were analyzed using traditional IRT and Bayesian IRT methods. Items analyzed with Bayesian methods have yielded significant statistical differences in item difficulty parameters versus traditional 1-PL Item Response Theory models, especially for items that had high difficulty parameter values. Further validation of the participants results was done with Bayesian regression models, which indicated that there was a moderately weak relationship between student performance on standard in-class examinations versus results on the instrument used in this study, which indicates that participant performance is not strongly related to performance on in-class exams. This indicates that further analysis of multiple-select items is warranted. Additionally, since multiple-select items are multifaceted, they have the potential to provide instructors with the ability to see if a student has full, partial, or no understanding of a concept as opposed to traditional single response multiple-choice items