Toward interdisciplinary science learning: development of an assessment for interdisciplinary understanding of ‘carbon cycling’
MetadataShow full item record
This study aimed at developing and validating an assessment that measures interdisciplinary understanding for the topic carbon cycling. The impetus for this study is the recognition of assessment as “the ‘black hole’ of interdisciplinary education” in K-16 science education (Boix Mansilla, 2005, p. 18). There is no question that the complexity of natural systems and the corresponding scientific problems necessitates interdisciplinary understanding informed by multiple disciplinary backgrounds. This study followed the construct-modeling framework for the interdisciplinary science assessment (ISA) design process (Wilson, 2005). A construct map for interdisciplinary understanding of carbon cycling was developed. Nine different subtopics within carbon cycling were determined based on content experts’ concept maps and analyses of the Next Generation Science Standards. Initial items were reviewed by content experts and piloted with students to establish content validity. Through the item revision process, a final version of the ISA was developed including 11 multiple-choice (MC) items and eight constructed response (CR) items. 454 students (9th grade to college seniors) were recruited and administered the ISA through the Qualtrics online environment. For the CR items scoring rubrics were developed and used to code student responses by a group of evaluators. Agreement between coders was greater than 90%, and analysis of scores indicated excellent inter-rater reliability. Item Response Theory (IRT) models, a two Parameter Logistic Model and a Generalized Partial Credit Model, provided evidence of the construct validity of the assessment items. All items reflected unidimensional construct and local independency in the IRT analyses. All except one item were a good fit to the models. The misfit item was too easy for the range of student performance levels. Two items functioned differentially across gender, indicated a possible bias. The 19 items showed modest internal consistency (Cronbach’s alpha =0.782). The findings suggest that the ISA is a promising and valid tool to assess interdisciplinary understanding in learning carbon cycling but the one misfit item and two DIF items merit further revision to strengthen the psychometric properties of the ISA. It is believed that the shift in the perspective of assessment towards interdisciplinary understanding enables science teachers to design their curriculum and instructional practices in a way that their students can learn how to connect one concept to another across different science disciplines, improving their scientific literacy.