Learning Analytics: A Traveller’s Guide

Anyone participating in the learning journey that is INF537 would have been intrigued by the title of Colloquium #2 (Welsh, 2015). The content, while very different in delivery from Colloquium #1 (Astbury, 2015), was equally thought provoking. Despite the title, data was not the only aspect covered, and the final comments indicated the incredible potential of learning analytics.

Simon’s opening comments related to his chosen title, as he pointed out that a traveller digs deeper than a tourist. He then commented that the interpretation and mining of data is an aspect of teaching and learning that is still sorting itself out.

For those who share an antipathy to using test scores to predict educational outcomes, Simon’s comments opened a door to improved educational futures. He explained that academic analytics are those used by institutions to aid with student management while learning analytics are interrogated to support learning and teaching for improved outcomes.

Investigating these concepts further indicates that data mining does not occur in a vacuum; it links to power and relationships; the capturing and sharing of data is in itself a development of knowledge capital (Weller, 2011, p. 43). Another aspect of such data is how it is managed and preserved (Weller, 2011, p. 43). Those generating the most data in a digital world are already privileged, and the rapidly expanding body of work is increasing the division between the haves and have-nots.

Simon referred to the example of the ATAR system and its use by schools to target areas that teachers need to improve, compared to its use by the MySchool website, where visitors choose a very different interpretation. This illustrated the importance of context and intent in such data collection and its subsequent use (Welsh, 2015).

There are three aspects of simplistic data use that cause concern:

1. What does it mean for a student to be monitored in this way – is it profiling or determinism, as Hyacinth posted in the accompanying chat?
2. The ethics of such use – who actually owns the data?
3. The fact that teachers are being asked to interpret such data without training in data literacy (Liz Eckert).

It is also important to know how reporting systems are being used and where the data is coming from in order to give appropriate advice based on the conclusions that are being drawn. Much of the data comes from the vendors of Learning Management Systems, who have set up metrics based on ease of use. Algorithms based on the number of clicks or the amount of time spent on any given task are not really a measure of learning and need to be carefully interpreted. There is a big difference between measuring quantities of clicks and measuring the quality of engagement (Welsh, 2015).

The example of using Virtual Learning Environments (VLEs) to capture and mine data was very interesting. VLEs are vendor focussed and often simplistic in terms of the data they gather. Once an institution has invested in providing a VLE it can be stuck with that specific product, as migrating to another platform is expensive and time consuming (a point noted and discussed by several classmates). Weller considers that introducing VLEs has led to the educational institution losing control of data to the manufacturer, and cites the example of Blackboard trying to patent many core e-learning concepts (Weller, Digital Resilience, 2011, pp. 170-171). Andrew questioned consideration of other products as a replacement, notably Moodle, which is open source.

An example Simon explored in some detail was the use of subject forums, such as those used within the Charles Sturt Blackboard internet, and, in the case of my workplace SIMON (School Information Management on the Net). If students have to participate in online forums within their VLEs then a tool to measure this must be able to “read” the type of material being entered. In this way, within an hour of the posted comment a scaffold into deeper learning could be generated, problems within comments across the group can be alerted to the educator, and extra reading could be suggested to those requiring additional explanation, or extension.

This type of monitoring could lead to an easy citation mechanism for resources utilised, which, as Greg commented, would be “referencing heaven”. It is in these potentially positive contributions to learning that most teachers can see the real value of data mining, rather than the click counting and number of visits which are so commonly applied. Resulting real time adaptation of learning programs to personalise student learning experience, development of meta-cognitive skills for learners, fast response to learning design and quick adaptation of technical equipment and systems would all be welcomed by educators (Welsh, 2015).

Weller warns of potential risk from using data to analyse and improve results by stating that it could lead to Google replacing human librarians, and user generated “playlists” of information may make teachers irrelevant (Weller, Digital Resilience, 2011, p. 171). This is a very broad allegation which has been somewhat allayed by Simon’s Colloquium session.

As Rochelle commented: the link between educational data mining, decision support systems and expert systems is inextricable; Deborah’s response that the skill lies in using the power for good sums up the feeling of most educators whose primary focus is the overall well-being of people in their classes.

While Simon’s presentation assuaged some fears, it raised other issues of potential concern for teachers and students. Needless to say, we are living in revolutionary times, and, while a revolution may be bloodless, it is rarely painless (Weller, Digital Resilience, 2011, p. 168). The critical thing for scholars and teachers is that they stay involved, because they need to be in a position to determine what goes, what stays and what comes; passitivity is not an option (Weller, Digital Resilience, 2011, p. 184).

References

Astbury, A. [Host]. (2015, July 21). ABC Splash Online Colloquium 1. Melbourne, Victoria, Australia.

Weller, M. (2011). Digital Resilience. In M. Weller, The Digital Scholar: How Technology is Transforming Scholarly Practice (pp. 168-184). London: Bloomsbury Collections. doi:10.5040/978184966275.ch-014

Weller, M. (2011). The Nature of Scholarship. In M. Weller, The Digital Scholar, How Technology is Transforming Scholarly Practice (pp. 41-51). London: Bllomsbury Collections. doi:10.5040/978184966275.ch-014

Welsh, S. [Host]. (2015, July 28). Learning Analytics: A Traveller’s Guide; Online Colloquium 2. Albury, Victoria, Australia.

Acknowledgements:

Fellow travellers’ comments from the Colloquium chat box are acknowledged in blue.