The field of education is in the midst of a data transformation: there are changes in standards, an increasing use of assessments, and a growing demand to measure performance, all of which is driving the need for teachers and school administrators to work with data in new ways. These changes offer an exciting opportunity to use data to inform instructional practice. But reviewing data takes time and capacity, and it’s not easy to translate findings into changes in the classroom. The influx of new standards and assessments has not come with additional funds or guidance to support educators in their efforts to make sense of all the information. There is a growing need to ensure schools and districts have the capacity and systems to effectively work with data. What will it take to equip teachers and administrators with the tools and support to truly use data to inform instructional practice and student learning?
In Marin County, schools and districts are grappling with this question. Since 2010, the Marin Community Foundation has provided the Early School Success grant as part of its Achievement Gap Strategic Initiative, with the goal of building a preschool through third grade (PreK-3) system in Marin County. In collaboration with the Marin County Office of Education, MCF provides ten elementary schools in four Marin County school districts with much-needed resources to implement strategies and interventions designed to support a PreK-3 model. LFA has served as MCF’s evaluation and learning partner since the start of the initiative. In 2016, the sixth year of implementing the Early School Success grant, we worked closely with MCF to examine the topic of data use, and how PreK-3 teams are using data to drive decision-making to best support student achievement. You can access the full Data Matters Report to see details of the study.
The Data Matters Framework outlines a set of critical components that schools must have in place to meaningfully engage in data reflection and learning. These are the building blocks for data use in schools - the capacities and structures that educators need to effectively use data to advance learning and practice.
We offer this framework as a tool that educators and school-based staff can use to strengthen their data use practices. We also offer this as a tool for evaluators who are working with schools and looking for language that can support discussions with clients by illuminating what we mean - specifically and on the ground - when we talk about data use.
What does it mean to use data in the service of student learning?
First, we need to think about the data cycle – the process that data goes through from raw numbers to actionable changes that support student learning. This is part of what educators do in their jobs everyday, but it is helpful to break down the steps involved and talk through how we actually use data to inform decision-making.
Below, we illustrate a theoretical framework we’ve adapted from the literature (Mandinach et al, 2006). We’ve broken it out by what we see as the conceptual phases in the cycle (the concept behind each phase – the outer blue circle) and the practical phases (what educators are actually doing in each of these six phases – the inner red circle).
The Five Phases of an Effective Data Cycle
Effective data cycles (i.e. where data is used in the service of student learning) transform raw data into knowledge that leads to informed action. It is a five-phase approach:
You collect and organize data in order to measure progress toward an identified goal.
Data becomes meaningful information when you consider the numbers within a context – this is when you are analyzing and interpreting the data.
When you tie that information to other information you have, your information becomes knowledge - for example, when you connect what you know about a student’s performance with what you know about the instruction provided to them. This is when you are synthesizing the information, and identifying the key takeaways that you need to prioritize.
You apply that knowledge in order to make a decision, which means you have to adjust or pivot what you are doing.
You then take action to make changes based on those decisions – for example, you implement a new strategy or instructional tool because the data indicate that it will further support student learning.
How can we best set up schools to implement effective data cycles?
Click on the image to download a printable version of the Data Matters framework
Through the Data Matters study, we have identified what appear to be the key components of effective data use — these are the conditions under which schools are able to implement effective data cycles. The theory behind this framework is that in order for schools to integrate effective data cycles into their everyday operations and their general approach to learning, they must have support at two levels:
At the Organizational level – where the Culture and Infrastructure supports and promotes the use of data.
At the Practical level – where the Practices that teachers, administrators, and other school staff engage in also supports and promotes the use of data.
Within each domain, certain components are critical for ensuring that schools are set up to effectively use data in support of student learning:
CULTURE – the organizational components that build a culture that supports and promotes the use of data to drive decision-making.
SCHOOL LEADERSHIP: School leadership cultivates a culture of data use by setting transparent expectations about data, providing access to data, modeling good data use, and allowing time and space for staff to engage with data.
CONTINUOUS IMPROVEMENT MINDSET: School staff embrace data as a tool when there is a school-wide orientation toward learning, continuous improvement, and collaborative inquiry.
ORIENTATION TO ALIGNMENT: An orientation to alignment means that schools are embracing strategies that support deep collaboration, and syncing up actions with shared goals .
INFRASTRUCTURE – the organizational components that support data use practices, making sure that teams have access and capacity to use data to improve instruction.
HIGH QUALITY DATA: Data must be relevant, complete, secure, and actionable in order to inform changes to practice
DATA LITERACY: Educators must be confident in their knowledge and skills of data analysis and interpretation if they are to use data for decision-making purposes.
DATA FACILITATOR: A data facilitator provides staff with access to data, processes to guide reflection, guidance in interpreting results, and accountability to act on the data findings.
CO-CREATED GOALS: Schoolwide goals, co-created among staff, support the data cycle process by helping staff members collectively examine data, identify areas for growth, and agree upon goals.
PRACTICES - specific strategies that are critical for ensuring schools are truly engaging in effective data cycles.
REFLECTION ROUTINES: The routines that schools put in place to regularly reflect on data in order to make adjustments and appropriately respond to students’ needs. We’ve learned that effective reflection routines are:
Frequent and timely
Occurring at the classroom, grade, and school-wide levels
Guided by a protocol
Providing a space and time for dialogue, and
Always action-oriented
ANALYTIC PRACTICES: These are the actions that teams collectively and collaboratively engage in to examine data. While we could write an entire post on this one component, some key features of effective analytic practices are that you:
Allow the data to speak first – start by reading the numbers before making interpretations
Examine the data from multiple perspectives – zoom in from the school-wide level to the individual student level OR zoom out from the individual/classroom level to the bigger school-wide level
Summarize gaps or patterns in the data
FEEDBACK LOOPS: This is the process of sharing data with those who need to see it, ensuring decisions based on data lead directly to improved student outcomes. Effective feedback loops mean that:
Data is shared with school staff so that all staff can take ownership in adjusting and improving their work.
Data is shared with families so that parents can understand how their child is performing and how they can work together with the school to support their child’s learning.
Data is shared with students so that they have prompt and constructive feedback on their performance, and can feel empowered to set goals for their learning.
What do you think?
What other tools, frameworks, or considerations should evaluators and educators keep in mind to effectively support data use in schools?