As part of my doctoral process, I have to review a lot of literature. A LOT. To motivate me to keep track of the resources as I read them, I am going to try to post summaries of the pertinent articles here. My hope is that I will get in the habit of writing and summarizing so that the literature review portion of my dissertation is easier to write. We’ll see if that works, and if I can keep up with the summaries!
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 170–179). New York, NY, USA: ACM. doi:10.1145/2460296.2460330
Judging MOOCs based only on completion rates is a narrow view that does not include the various ways that learners participate in open courses. However, viewing individuals based on their own learning goals and ideas of success does not provide useful measures. Instead, a meaningful set of patterns of engagement or disengagement would be more balanced and useful.
The researchers analyzed learner trajectories in three MOOCs: “Computer Science 101”, “Algorithms: Design and Analysis”, and “Probabilistic Graphical Models.” They first computed a description for each student based on the way the student engaged throughout the course, then used k-means clustering techniques to find subpopulations. This allowed them to find four prototypical engagement patterns:
- Completing: learners who completed most of the assessments
- Auditing: learners who watched lectures and infrequently completed assessments
- Disengaging: learners who completed assessments in the beginning of the course but then decreases in engagement
- Sampling: learners who watched lectures for only one or two periods (sometimes only one video)
The authors argue that these clusters demonstrate the existence of multiple trajectories through a course that are not represented in the binary completion model. Analyzing course satisfaction measures against these categories shows that, currently, Completing and Auditing learners are equally satisfied with course design, while Disengaging and Sampling learners report lower levels of experience. Future course design could make allowances and provide pathways for Auditing and Sampling learners, which may also prevent Disengaging learners from separating completely.
This is by far the best research in a MOOC that I have seen thus far. I am very impressed by the rigor of their study and the process they followed to identify then verify the categories of engagement. I can see applications to course design, like identifying different course activities that different types of learners should attempt if they are on a specific pathway. I would love to use this framework for examining participation levels in future courses and to validate design decisions.