Imagine if your school knew exactly how many times you attempted that impossible trignometry problem and adjusted your next assignment accordingly. Imagine if teachers could create project teams based on precise student strengths, weaknesses, and behaviors. Imagine if your educational experience was as personalized as your Netflix home page.


The potential for big data to revolutionize education is so transformative, it’s almost unimaginable. ‘Big data’ within the context of education most commonly refers to data needed to improve student outcomes. Data that, in theory, should change the way teachers teach and students learn based on clearer insights into student learning patterns and performance results. A simplified diagram showing data usage in a traditional K-12 education setting is shown below:


As students go through their educational careers, they create a unique trail of data around their behaviors, progress, and outcomes. Some of this data is currently captured and analyzed. Educational institutions around the country utilize a variety of student information systems (SIS), massive databases, to collect information around demographics, performance, etc. Lack of interoperability of systems across schools and districts can make data extraction and analysis a cumbersome process. Although innovative edtech companies are creating predictive educational software, integration with multiple types of SIS hampers widespread adoption (this is just one issue, others involve school budget restrictions, teacher training needs, etc). One startup trying to solve this problem is Clever, a SF-based venture-funded company that offers software developers a simple API to connect the vast array of SIS with third-party applications. With Clever, mining student data out of SIS becomes an easier task, allowing developers to more easily develop and sell software to schools. This type of data could ultimately predict when a student is at risk of failing and intervene and the right time, flag low-performing districts with personalized improvement recommendations, and more accurately deploy staffing resources. Before higher level analytics can prevail, underlying data needs to be accessible and usable.

Aside from interoperability, a myriad of issues plague the system with respect to the proliferation of data and analytics. Privacy and ethics continue to cause concern for students, parents, and institutions. Educational facilities and their presiding governments need to prioritize data analysis and appropriate necessary support systems and funds to these initiatives. Costs to hire data systems employees are high and compete with funding for more tangible and understandable educational concerns (its unlikely that this sector would even attract the top minds in data science. Heck, if I could do that job, I’d be getting comfortable in an office next to Aaron Levie). Long term data proving benefits of data analytics on student outcomes are insufficient. Storage needs to host educational metadata are immense. The list goes on.

Big data’s potential to materially improve education and other critical industries are enormous. Although these sectors have made strides, implementation is slow and the trickle down effect of data analysis on student outcomes and performance has yet to be felt. Nevertheless, I think the next 10 years will bring an increased focus on the role of data in education. Hopefully, this transition will lead to material improvemetns in nation’s education system.


Infographic Source: