The effectiveness of synchronous massive online courses at The University of Texas at Austin

Is online education an effective and viable alternative to face-to-face education? The purpose of this dissertation was to evaluate the effectiveness of online education at The University of Texas at Austin (UT-Austin). The dissertation focused on Synchronous Massive Online Courses (SMOCs) at The University of Texas at Austin since 2012. This dissertation analyzed the extent to which course effectiveness varies as a function of lecture environment, comparing SMOCs to similar face-to-face (FTF) courses.

In total, 25,726 students across 53 courses at UT-Austin were included in analyses. Researchers compiled all relevant student and course data archived in university databases and merged that with course data compiled from archived course syllabi. Then, Hierarchical Linear Modeling was used to test how (a) final course grades vary as a function of lecture environment (SMOC or FTF), controlling for socioeconomic status, scholastic aptitude, and course exam frequency, (b) subsequent semester grades vary as a function of lecture environment (SMOC or FTF), controlling for socioeconomic status, scholastic aptitude, and course exam frequency, and (c) course completion rates vary as a function of lecture environment (SMOC or FTF), controlling for socioeconomic status, scholastic aptitude, and course exam frequency.

The primary goal of this project was to examine the effectiveness of SMOCs in comparison to FTFs. Course effectiveness was operationally defined with three objective outcomes: final course grades, subsequent semester GPAs, and course completions. Findings show that there were no significant differences between SMOCs and FTFs on any of these objective measures. That is, SMOCs neither outperform nor underperform FTFs in final grades, subsequent semester GPAs, or course completions.

Because previous studies propose that increasing exam frequency may reduce SES-based achievement gaps (e.g., Pennebaker, Gosling, & Ferrell, 2013), and there are some mixed results in the literature about the effectiveness of frequent testing (e.g., Bell, Simone, & Whitfield, 2015), a secondary goal of this dissertation focused on the interaction of SES and exam frequency in the context of course effectiveness outcomes. Exam frequency interacted with lecture environment; such that for FTFs, there was no substantial difference in final course grades by exam frequency; however, for SMOCs, students with more exams had higher final course grades than students with fewer exams. The highest final grades were earned by students in SMOCs that provided the highest exam frequencies (while accounting for control variables). Exam frequency also interacted with socioeconomic status (SES); such that for lower SES students, when exam frequencies are lower the probabilities of course completion are lower than when exam frequencies are higher; and when exam frequencies are higher, the probabilities of course completion are higher than when exam frequencies are lower. For higher SES students, the probabilities of course completion did not vary by exam frequency. Given these findings, increasing exam frequencies in course structures is recommended.

Looking across a wide range of course topics and courses, and large number of students, this dissertation provides evidence that SMOCs are as effective as FTFs on objective course outcomes, both short- and long-term. This includes final course grades, subsequent semester GPAs, and course completion rates as course effectiveness measures. Economically, SMOCs are able to reach thousands of students by relying on fewer faculty without the need for large classrooms. At the same time, it frees faculty to teach more and smaller upper division courses. Although the results of the SMOC and FTF courses were generally similar, the additional payoffs of the SMOCs make them a promising tool for the future of undergraduate education. If the high standard of educational course effectiveness is based in the traditional FTF course, then a comparable SMOC course meets that high standard.