Case study – Applying Azure Machine Learning in education to student dropout

Having recently written two articles about the theory of applying Machine Learning in Education – “Two ways to use Azure Machine Learning in education” and “Making machine learning in education easier for every day users” – I think it’s time to dive into a specific example of machine learning in education where it is being used to support education outcomes in schools. The story comes from my colleagues on the Machine Learning blog.

Tacoma Public Schools logo

The example is from Washington State, in the US, where Tacoma Public Schools has been using it as part of their ongoing initiative to prevent student dropout for school students. The district has delivered a dramatic turnaround – eight years ago five of the high schools were described as “dropout factories”, and five years ago just 55% of students graduated on time, compared to 81% nationally. But last year that had been boosted to 78%, ensuring that the district is recognised nationally for its educational achievements. The district is now developing the next level of data-driven improvement with the help of Machine Learning.

Starting the data-driven journey

The long journey started several years ago when new leaders joined the school district’s board. The new leaders expressed a desire to be more transparent with data and to use the data to address any shortcomings. They resoundingly embraced the value of data-driven analytics for the benefit of the district and its students.

The board also asked themselves a radical question: What if they could process all their data to predict whether or not a student was likely to disengage and ultimately drop out? This was in the days before Machine Learning in education, and so the project team worked with Microsoft to create a data warehouse with student grades, attendance, health records and other data. And teachers and administrators were given access to data through their tools, like SharePoint and Excel. As the board President, Scott Heinz, described it:

We now have this world-class data system for teachers to use. They want to know what is going on in their classrooms.

Getting predictive

It is only recently that the board have been able to apply Azure Machine Learning to predict future dropout risks – turning historical data into an engine for predicting future success. As Shaun Taylor, the district’s CIO said:

By using predictive analytics, we thought we would be able to intervene earlier and work closely with those at-risk students. Then we would be able to reach our ultimate goal: getting that graduation number close to 100 percent.

When we saw Azure Machine Learning, we started to see how it could be possible for us to realize our vision

The Microsoft team worked with the district to create a proof of concept, using Azure Machine Learning to create a model using five years of demographic, academic and student performance information to predict whether there was a risk of a student dropping out in the next semester. Using the Azure data services in the cloud, and going through a number of iterations of creating a predictive model (what’s key to this is understanding what factors might influence student dropout, and making that historical data available for the analysis) they were able to give a dashboard to board members to see the details of students at-risk of dropping out. Christopher Baidoo-Essien, at the school district summarised their journey:

When we started this POC, we didn’t know if any predictive analytics would be attainable. As we progressed and used more historical data, the model proved to be almost 90 percent accurate.”

Turning conventions upside down

As well as working on developing more real-time, or near-time, data sources for the analysis (using data that’s a month old risks missing key signals) and providing more regular analysis and reports, there is a focus on changing traditional, but incorrect, perceptions about the reasons for students’ struggles. As Christopher says:

Often, students are seen as fitting certain profiles that indicate a potential lack of success, but none of those profiles are supported by analytical data. We wanted to use data to change that perception. And eventually, we want to predict what the key indicators are for kids disengaging.

There’s plenty of work still to be done, but the journey so far has proven that there is significant value in both the data dashboards, as well as the predictive analytics.

You can read the full case study on the Azure Machine Learning blog

Applying machine learning in education in Australia

We will soon start to see examples within Australia of using Azure Machine Learning in education, as similar work is starting here. One of the data science teams behind this work in the US is working with S1 Consulting in Australia to fine tune a model for student dropout from universities, using the Azure Machine Learning service, and there’s another team at Literatu working on predicting opportunities for support and intervention for high school students, based on their performance in school assessments, as well as external assessments, such as NAPLAN and PAT assessments.