Browsing by Subject "Learning analytics"

Now showing 1 - 4 of 4

Examining scientific thinking processes in open-ended serious games through gameplay data
(2017-06-19) Kang, Jina, Ph. D.; Liu, Min, Ed. D.; Resta, Paul; Riegle-Crumb, Catherine; Horton, Lucas
Research on scientific problem-solving emphasizes the importance of problem solving and scientific inquiry as central components of the twenty-first century skills. Research has shown that open-ended serious games can facilitate students’ development of specific skills and improve learning performance through scientific problem-solving. However, understanding how students learn these complex skills in a game environment is a major challenge, as much research depends on typical paper-and-pencil assessments and self-reported surveys or other traditional observational and quantitative methods. The participants of the study were 237 sixth graders from two middle schools in the Southwestern area of the United States. The students used an open-ended serious game called Alien Rescue as their science curriculum for three weeks. The purpose of this study is, first, to identify students’ navigation behavior patterns in cognitive processes between at-risk and non-at-risk students within Alien Rescue. To accomplish this purpose, this study intends to use gameplay data by incorporating the integrated method of lag sequential analysis and sequential pattern mining together with a statistical analysis. The findings confirmed that the integrated method helped to explore students’ latent navigation behaviors as well as discover the differences of problem-solving processes between non-at-risk and at-risk students. The second purpose of this study is to examine the relationship between students’ learning performance and their scientific inquiry behaviors, which emerged as students engaged with Probe Design Center in this serious game. The results showed that the game metrics developed in Probe Design Center improved the predictions of both in-game and after-game performance. The cluster analyses with game metrics confirmed four unique groups regarding students’ scientific inquiry behaviors in Probe Design Center. This study concluded that the integrated methods of serious games analytics enabled researchers to investigate in-depth cognitive processes and scientific inquiry behaviors within a specific cognitive tool, Probe Design Center, and discover unique behavior groups across different school settings. The researcher identified the challenges of at-risk students in their cognitive processes and highlighted the support needs for these students. Consequently, this study proposed an interactive dashboard using the data-driven evidences to provide teachers just-in-time information to support students’ cognitive processes.
Inclusive learning with assistant chatbot in massive open online courses : examining students’ perceptions, utilizations, and expectations
(2024-05) Han, Songhee; Liu, Min, Ed. D.; Min Kyung Lee; Grace MyHyun Kim; Xiaofen Hamilton
This study examines students’ learning experiences with an assistant chatbot in professional development MOOCs designed for journalists. Utilizing a mixed-methods approach, it focuses on the students’ learning experience’s sub-domains, such as social presence, teaching and cognitive presence, self-regulation, ease of use, and behavioral intention. Employing the Community of Inquiry (CoI) framework and the Technology Acceptance Model (TAM), the study first assesses the impact of demographics like age, gender, region, and native language on these learning experiences. The study revealed that age and gender had no significant influence on learning experiences, while geo-cultural regions showed variations, particularly in social presence and, teaching and cognitive presence. Socioeconomic regions demonstrated more notable differences, especially between lower-middle and high-income areas. However, the native language did not significantly influence learning experiences. Second, structural equation modeling (SEM) validated several hypothesized relationships, highlighting the positive impact of self-regulation on various other learning domains. Interestingly, teaching and cognitive presence did not significantly influence behavioral intention, nor was there a significant relationship between behavioral intention and use time. Age and socioeconomic region factors were identified as full moderators, while gender was a partial moderator from multigroup SEM results. Third, an extensive analysis of student interactions with the chatbot was conducted using various data sources. This analysis revealed eight key topics of chatbot interactions and showed predominantly neutral sentiments in the chatbot text logs. However, survey and interview data indicated a generally positive perception of the chatbot, especially noting its operational effectiveness and ease of use. Sentiments varied across socioeconomic regions, with more positive feedback from lower-income regions, while those from higher-income regions had higher expectations. The study also observed differences in navigational patterns between chatbot users and non-users in the course. Chatbot users exhibited more diverse navigations, indicating deeper engagement with course materials and a higher completion rate. In contrast, non-users followed a more structured progression, mainly relying on the predefined course path. Finally, the study highlighted students’ expectations for the chatbot, emphasizing the need for improvements in response accuracy, diversity, and additional capabilities like multi-language support. The findings emphasize the role of demographic variables in shaping student interactions with chatbots in MOOCs and suggest that modifying chatbot responses for inclusiveness could be key in meeting diverse student needs. The implications include that adhering to Universal Design for Learning principles, empowered by current advancements in AI-based chatbot technology, and informed by the CoI and TAM, could better address the diverse needs in MOOCs, especially in chatbot-enhanced learning environments.
Learning analytics in large college courses : facilitating retention and transfer through targeted retrieval practice
(2016-05) Raley, Nathaniel David; Beretvas, Susan Natasha; Butler, Andrew Cox
Spaced retrieval practice is known to benefit both long-term retention and transfer of learning, two important goals of education. However, most classes are not designed in a way that facilitates frequent quizzing or revisiting previously covered topics; this is particularly true in higher education, where a small number of exams typically account for the bulk of a student’s grade. Recently, a large undergraduate course at the University of Texas has implemented a new class structure that replaces high-stakes tests with daily quizzes administered during class via computer; furthermore, quiz items previously answered incorrectly can appear at random on future quizzes. Together, these innovations are an excellent first step toward bringing spaced retrieval practice into the college classroom. However, I propose that technology can be further leveraged in classes such as these to more optimally choose repeated items. Given graded student quiz data from one semester of this course, I use Multidimensional Item Response Theory (MIRT) and Sparse Factor Analysis (SPARFA) to jointly estimate concepts underlying the items and each students’ mastery of these concepts. After comparing these factor-analytic methods, I also explore free-response and student chat data using basic natural language processing. It is concluded that techniques from learning analytics can help realize the full potential of spaced retrieval practice in the classroom by optimizing the selection of repeated items so as to target remediation. Furthermore, such techniques can be used to introduce variability into retrieval practice, encouraging a deeper understanding of the content which is more likely to transfer to novel problems.
The effects of a learning analytics scaffolding system in problem-based learning activities
(2022-05-26) Pan, Zilong; Liu, Min, Ed. D.; Hughes, Joan; Whittaker , Tiffany; Hamilton, Xiaofen
This explanatory mixed-method study examined the effects of a learning analytics scaffolding system in supporting students and teachers in middle school science problem-based learning activities. A total of 298 middle schools 6th-grade students taught by four science teachers were grouped into three conditions: learning analytics scaffolding group (condition A), non-learning analytics static scaffolding group (condition B), and control group (condition C). This study followed an explanatory mixed-method research design, the qualitative interview data were used for interpreting and explaining the quantitative results. The statistical outcomes showed that students’ problem-solving self-efficacy in condition A is significantly higher than in the other two conditions. No main effects were found on students’ content knowledge acquisition. The student interviews revealed that the real-time support and just-in-time feedback delivered by the learning analytics scaffolding system helped them achieve higher self-efficacy in problem-solving. Moreover, students under each condition were further grouped based on gender or learning mode to explore potential differences. The statistical findings showed that male students achieved higher problem-solving self-efficacy in condition A than female students. In contrast, students in the online mode gained more content knowledge than students in the in-person mode. An important component of this study is the involvement of student-generated usage data. Both student-generated quantitative log and qualitative text data were processed and integrated with qualitative interview outcomes for understanding the quantitative survey findings. For example, survey outcomes revealed that students in the online mode in condition A achieved higher content knowledge acquisition than students in the in-person mode. The quantitative log outcomes revealed that students in the online mode in condition A accessed the scientific concepts with significantly higher frequencies and longer durations than their peers in the in-person mode, which means students in the online mode experienced a larger exposure to scientific knowledge than their in-person mode peers. Considering the findings from qualitative interviews that students in the online modes were less distracted and more likely to follow the scaffoldings, the integration of both log data and qualitative interview outcomes provided a more comprehensive picture to interpret and explain the quantitative survey results. Furthermore, all four participating teachers acknowledged the usefulness of the learning analytics scaffolding system. They indicated in the interviews that this scaffolding system enhanced students’ independence in the problem-solving process. Thus, teachers perceived larger flexibility in managing students in condition A than in the other two conditions. These outcomes indicated that the learning analytics scaffolding system supported students by providing them with more assistance and empowered teachers to facilitate problem-based learning activities in large-sized classrooms. In all, the evaluation of the learning analytics scaffolding yielded positive outcomes for both students and teachers. It supported that enhancing the problem-solving environment by embedding the learning analytics incorporated scaffolding system is a promising direction to better support students and teachers in problem-based learning activities. Last but not least, practical implications for the future implementation of learning analytics scaffolding systems were proposed.