Etec565A: Understanding Learning Analytics

lamp icon

Identification

When we talk about technology innovation, learning analytics (LA) is one of the tops in the list of promising technologies (Long & Siemens, 2012). In light of the Etec565A experience, I was able to: 

some learning outcomes from etec565a
Setting icon

Conceptual Design

Written comments in teaching and unit evaluations are a valuable source of data because students are free to articulate what they perceive to be important in open-ended questions.  Several text analytics tools like NVivo and survey platforms like Qualtrics and Blue offer features to facilitate the analysis process. However, the developers and researchers of Quantext argue vigorously that these tools are not always be driven by the educators’ and learners’ needs (McDonald et al., 2019).  Also, they are not simple or accessible to educators, and they are seldom utilized without significant incentives and academic development effort (McDonald et al., 2019). Nonetheless, these tools are firmly rooted in business analytics, where there are always concerns about ethics rather than support.

code icon

Implementation

Quantext to Analyze Students’ Qualitative Text Comments : Goal [3] Hover here [Goal [3] To further develop a critical perspective in order to evaluate and draw conclusions about technology developments..

This artifact again demonstrates my ability to evaluate technological solutions. I sifted through all Quantext features and reported that the handy tool facilitated the task of summarizing free-text comments from (SETs) and there is potential to develop this further. I would like to note that when I completed the paper, the tool was a free web-based platform available for educators’ use and evaluation. However, at the time, the web application is no longer available, and the source code available in GitHub has had no activity since 2019. It seems to me that Quantext hasn’t been able to sustain among business analytics tools.

CHECK MARK

Evaluation

While Quantext was a bit buggy (normal as a beta-version) and perhaps less appealing than the market has on offer, I thought of it as a unique case where educators were the developers and designers of their own technologies. Also, it was an exemplary of how educators and students can become active agents involved in the implementation and testing processes. Furthermore, as Quantext was offered as an open-source, I was more comfortable because there was an explanation of what’s under the hood. I truly felt disappointed that such effort hasn’t gone further. And perhaps, this work is a clear demonstration of how locally funded educational innovations do not go beyond the prototype phase and eventually couldn’t sustain within the competitive open market.

My big learning moments in Etec565a

Beyond the artifact scope, the ethical dimensions and legal considerations are significant barriers to LA integration. Even though several organizations and efforts, most notably Jisc in the United Kingdom, has explored and defined the ethical principles of LA integration (Sclater &Bailey, 2015), still each institution need to set up its own policies and legal arrangements and resolve the issue of students’ agency with their data, so it grants an ethical integration. As for the technical side of this dilemma, I am aware that some policies prohibit being open about the coding practices for competitive reasons, but trust is an essential facilitator. Rapaport (2020) puts the situation bluntly as follows: 

[Consider] a documentionless [digital solution] found. Suppose that we discover that it successfully and reliably solves a certain type of problem for us. Even if we cannot understand why or how it does that, there doesn't seem to be any reason not to trust it. So, why should justifications matter? After all, if a computer constantly bests humans at some decision- making task. Why should it matter how it does it? (p.695).

If Rapoport’s (2020) statement doesn’t feel right to you, then educational institutions and academic staff will rightly be hesitant to place absolute trust in analytic solutions (no matter how capable) without having full justification for the analytical results and granting data protection and privacy measures.

Quantext Key Characteristics

  • Simple and intuitive tool.
  • Bottom-up internal development created by educators for students and educators.
  • Source-code was available to evaluate the validity of results.

Datasets

Three datasets from different contexts were used in my investigation. 

Key Findings

The tool facilitated identifying the key strengths and weaknesses of the relevant teaching aspects.

Key Challenges

No documentation to guide how to employ the tool. Few technical issues that needed a fix.

Recommendations

  • Local deployment rather than web-based solution.
  • Additional features, for example, sentiment classifiers.