An examination of the validity of the mathematics exit level Texas Assessment of Knowledge and Skills

Abstract

This study examines the validity of the Spring 2004 Mathematics Exit Level TAKS. In particular, I examined the test through three forms of evidence: content area specialist surveys, statistical analysis of item-level data from 4340 students provided by TEA, and individual interviews conducted with thirty-four 11th grade students. These multiple lines of evidence give a clear understanding of the actual constructs the test measures. Because of the high-stakes nature of this exam, it is important to examine its validity closely. Assessing the validity of a test involves looking at the appropriateness, the meaningfulness and the usefulness of the test through an empirical investigation into the underlying constructs. Each aspect of this study has provided insight into the underlying constructs of the TAKS. TEA states ten broad objectives TAKS is supposed to cover. Each objective is further broken down into detailed sub-objectives. The TEAstated objectives and sub-objectives are used in this study as the intended constructs of the test. In general, the content area specialists’ surveys did not confirm the TEA-stated objectives for the test, however this could be due to nuances in the way TEA defined various sub-objectives. A factor analysis was conducted on the TEA data set in order to see if the items would factor along the TEA-stated objectives, however since the test is designed to be multidimensional, it is not surprising that the items did not factor along objectives. Differential Item Functioning (DIF) was also conducted in order to determine if any items were particularly problematic for various subgroups. Significant DIF was detected in almost one-fourth of the items usually with African American students as the disadvantaged group. The most useful information came from the student interview data conducted on twenty of the items. Through these interviews, the true constructs the items measure were revealed. In some cases the student interviews validated the TEA-stated objectives, however in many cases, the student interviews showed a different construct. It is mostly due to the results of the student interviews that I was not able say the exit level TAKS is a valid measure of the intended constructs.