Examining learners’ social presence in a Massive Open Online Course through social network analysis and machine learning

Low engagement has been a longstanding problem in Massive Open Online Courses (MOOCs). However, engagement is crucial in social learning contexts to increase knowledge construction and achieve meaningful learning outcome. To further understand learners’ engagement in MOOC discussion forums, this study focuses on the perspective of social presence, which is defined as learners’ ability to project themselves socially and emotionally in a community of inquiry. Social presence is an important factor that has the potential to affect learners’ learning experience and outcome. This study took place in the context of a professional development MOOC in the field of journalism. The discussion posts, system log data and survey responses were collected and analyzed. The purpose of this study is to understand the learners’ participation patterns in the discussion forums over the six modules of the MOOC, and the relationship between learners’ social presence, their positions in the learner network and their learning outcomes. In terms of data analysis, this study adopted a mixed-method approach to examine the data from both qualitative and quantitative aspects: to qualitatively analyze the posts, a machine learning supported text classification model was developed and applied to automatically analyze the large-scale text data in the forums; social network analysis (SNA) was used to analyze the characteristics of the learner network and determine learners’ centrality (degree, closeness, betweenness and Eigen centrality). Centrality is an important measure because prior studies found it to be an important predictor of learning outcome. Correlation analyses were used to discern the relationship between social presence and learners’ centrality, while regression models were built to investigate how learners’ social presence and posting behaviors (frequency of posting, average length of posts and day of posting) predict learners’ network centrality. Finally, correlation analyses were conducted to understand the association between learners’ network centrality and their certificate status, perceived learning and satisfaction. The purpose of using mixed methods is to see in what ways the qualitative nature of the posts and learners’ posting behaviors impact learners’ positions and influence in the learning community and their learning outcomes. The findings revealed the evolvement of the learner network in relation to the distribution of social presence throughout the MOOC. The results also showed that social presence indicators such as Complimenting others, Expressing agreement, Expressing gratitude and Disagreement/doubts/criticism play important roles in learners’ centrality in the learner network. Beside social presence, frequency of posting has strong effect in predicting learners’ network centrality, while other factors such as the average length of posts and the timing of posting have marginal impact in the prediction. Finally, this study found that learners’ network centrality is correlated with their certificate status as well as their overall satisfaction with the MOOC, but not correlated with their perceived learning in the MOOC. This study is among the first efforts in MOOC research to examine the relationship between social presence, learners’ network centrality and learning outcomes. It provides a critical ground for studying content-related interaction and learning community in MOOC forums. The findings inform MOOC learners in terms of how to strategically present themselves in the discussion forums to increase the possibilities of peer interaction and achieve productive learning outcomes. For examples, findings suggest that learners may obtain more central position in the community by posting more compliments, expressing more gratitude, and communicating agreement and disagreement, doubts etc. While for MOOC instructors, this study will potentially inform them how to effectively mediate the discussions and improve learner engagement as a facilitator, such as paying attention to the changes of learner network, identifying central learners, monitoring learners’ affective states.